Healthc Inform Res.  2024 Oct;30(4):324-332. 10.4258/hir.2024.30.4.324.

Evaluation of the Usability and User Experience of the Jaminan Kesehatan Nasional Mobile Application in Indonesia

• Handasari SP
• Wulandari R
• H

• KMID: 2560535
• DOI: http://doi.org/10.4258/hir.2024.30.4.324

Abstract

Objectives
The National Health Insurance (Jaminan Kesehatan Nasional or JKN) mobile application represents an administrative transformation of health social insurance in Indonesia. The application’s user base has quadrupled in the past 2 years. The study aimed to evaluate the usability and user experience of the JKN mobile application in Indonesia.
Methods
The study administered an online survey featuring the System Usability Scale (SUS) questionnaire alongside a user experience assessment. The survey was disseminated via social media and garnered responses from 370 participants. The variables examined included usability and user experience, specifically focusing on attractiveness, perspicuity, efficiency, dependability, stimulation, and novelty. Data analysis was performed descriptively, and the results were benchmarked.
Results
The JKN mobile application had an average SUS score of 60.9 ± 13.47 (adjective rating “good,” Grade “F,” acceptability rating of “low marginal,” and a percentile range of 34). User experience testing yielded the following results: attractiveness (1.09 ± 1.09), perspicuity (1.05 ± 1.17), efficiency (1.07 ± 1.20), dependability (1.03 ± 1.12), stimulation (0.88 ± 1.20), and novel-ty (0.58 ± 1.09). The benchmark for the JKN mobile application placed the user experience in the “below average” category.
Conclusions
The evaluation of the usability and user experience of the JKN mobile application was generally positive. However, there is a need to improve the usability and all parameters of the user experience. Broader-scale research is needed to explore the factors that influence usage and user experience to promote the adoption of the JKN application and increase user satisfaction.

Keyword

Mobile Applications, National Health Programs, Benchmarking, Indonesia, Surveys and Questionnaires

Figure

• 

  Figure 1 User interface display of the National Health Insurance (Jaminan Kesehatan Nasional or JKN) mobile application.

• 

  Figure 2 Users’ System Usability Scale score [18].

• 

  Figure 3 National Health Insurance (Jaminan Kesehatan Nasional or JKN) mobile application benchmark.

Reference

References

1. The Social Health Insurance Organizer. Participants of the National Health Insurance Program (Program JKN) [Internet]. Jakarta, Indonesia: The Social Health Insurance Organizer;2023. [cited at 2023 Jul 7]. Available from: https://www.bpjs-kesehatan.go.id/#/.

2. Ministry of Communication and Digital Republic of Indonesia. Access services at your fingertips via JKN Mobil [Internet]. Jakarta, Indonesia: Ministry of Communication and Digital Republic of Indonesia;2020. [cited 2024 Oct 28]. Available from: https://www.kominfo.go.id/berita/pengumuman/detail/mengaksespelayanan-dalam-genggaman-melalui-jkn-mobile/.

3. Bahri S, Amri A, Siregar AA. Analysis of service quality of the mobile JKN application by the social health insurance organizer using the service quality (Servqual) method. Ind Eng J. 2022; 11(2):1–7.

4. Nielsen J. Usability 101: introduction to usability [Internet]. Dover (DE): Nielsen Norman Group;2012. [cited at 2023 Jul 19]. Available from: https://www.nngroup.com/articles/usability-101-introduction-to-usability/.

5. Lewis JR. The system usability scale: past, present, and future. Int J Hum Comput Interact. 2018; 34(7):577–90. https://doi.org/10.1080/10447318.2018.1455307.

6. Muro-Culebras A, Escriche-Escuder A, Martin-Martin J, Roldan-Jimenez C, De-Torres I, Ruiz-Munoz M, et al. Tools for evaluating the content, efficacy, and usability of mobile health apps according to the consensus-based standards for the selection of health measurement instruments: systematic review. JMIR Mhealth Uhealth. 2021; 9(12):e15433. https://doi.org/10.2196/15433.

7. Hajesmaeel-Gohari S, Khordastan F, Fatehi F, Samzadeh H, Bahaadinbeigy K. The most used questionnaires for evaluating satisfaction, usability, acceptance, and quality outcomes of mobile health. BMC Med Inform Decis Mak. 2022; 22(1):22. https://doi.org/10.1186/s12911-022-01764-2.

8. Brooke J. SUS: a retrospective. J Usabil Stud. 2013; 8(2):29–40.

9. Weichbroth P. Usability of mobile applications: a systematic literature study. IEEE Access. 2020; 8:55563–77. https://doi.org/10.1109/ACCESS.2020.2981892.

10. Lobo EH, Abdelrazek M, Frolich A, Rasmussen LJ, Livingston PM, Islam SM, et al. Detecting user experience issues from mHealth apps that support stroke caregiver needs: an analysis of user reviews. Front Public Health. 2023; 11:1027667. https://doi.org/10.3389/fpubh.2023.1027667.

11. Hornbæk K, Hertzum M. Technology acceptance and user experience: a review of the experiential component in HCI. ACM Transact Comput Hum Interact. 2017; 24(5):1–30. https://doi.org/10.1145/3127358.

12. Sharp H, Rogers Y, Preece J. Interaction design: beyond human-computer interaction. 2nd ed. Chichester, UK: John Wiley & Sons;2007.

13. Wulandari R, Suwandono A, Kartasurya MI, Nugraheni SA. Postpartum care behavior improvement during COVID-19 pandemic in indonesia using mobile-health interactive message. Ethiop J Health Sci. 2022; 32(2):243–54. https://doi.org/10.4314/ejhs.v32i2.4.

14. Kushendriawan MA, Santoso HB, Putra PO, Putra PO, Schrepp M. Evaluating user experience of a mobile health application halodoc using user experience questionnaire and usability testing. J Sist Inf. 2021; 17(1):58–71. https://doi.org/10.21609/jsi.v17i1.1063.

15. Islam MN, Karim MM, Inan TT, Islam AK. Investigating usability of mobile health applications in Bangladesh. BMC Med Inform Decis Mak. 2020; 20(1):19. https://doi.org/10.1186/s12911-020-1033-3.

16. Bangor A, Kortum P, Miller J. Determining what individual SUS scores mean: adding an adjective rating scale. J Usabil Stud. 2009; 4(3):114–23.

17. Santoso HB, Schrepp M, Isal RY, Utomo AY, Priyogi B. Measuring user experience of the student-centered elearning environment. J Educ Online. 2016; 13(1):58–79.

18. Laugwitz B, Held T, Schrepp M. Construction and evaluation of a user experience questionnaire. Holzinger A, editor. HCI and usability for education and work. Heidelberg, Germany: Springer;2008. p. 63–7. https://doi.org/10.1007/978-3-540-89350-9_6.

19. Lewis JR, Sauro J. Item benchmarks for the system usability scale. J Usabil Stud. 2018; 13(3):158–67.

20. Parera G, Darwis M. Usability of JKN mobile application using system usability scale (SUS) method. J Bit-Tech. 2023; 5(3):138–45. https://doi.org/10.32877/bt.v5i3.691.

21. Lubs BO, Salim A, Jefi . Evaluation of usability of the mobile JKN application system using the use questionnaire. J Saintekom. 2020; 10(1):65–76.

22. Annisa N, Pradana DS, Suharso W. Evaluation of the National Health Insurance (JKN) mobile application in Malang City from the usability aspect. J Repos. 2020; 2(12):1689–99. https://doi.org/10.22219/repositor.v2i12.30999.

23. Wulandari R, Pangarsa EA, Andono PN, Rachmani E, Sidiq M, Setyowati M, et al. Evaluation of usability and user experience of Oncodoc’s M-health application for early detection of cancer. Asian Pac J Cancer Prev. 2022; 23(12):4169–4176. https://doi.org/10.31557/APJCP.2022.23.12.4169.

24. Iskandarsyah A, Shabrina A, Siswadi AG. Usability and acceptability of JAGA SEHAT: mobile application to improve knowledge about healthy lifestyle. J Multidiscip Healthc. 2022; 15:115–24. https://doi.org/10.2147/JMDH.S342913.

25. Dyayu AL, Beny YH, Yani H. Evaluation of the usability of the PeduliLindungi application using usability testing and the system usability scale (SUS) method. J Manaj Teknol Dan Sist Inf. 2023; 3(1):395–404. https://doi.org/10.33998/jms.2023.3.1.720.

26. Oakley-Girvan I, Yunis R, Fonda SJ, Neeman E, Liu R, Aghaee S, et al. A novel smartphone application for the informal caregivers of cancer patients: usability study. PLOS Digit Health. 2023; 2(3):e0000173. https://doi.org/10.1371/journal.pdig.0000173.

27. Hassenzahl M. The effect of perceived hedonic quality on product appealingness. Int J Hum Comput Interact. 2001; 13(4):481–99. https://doi.org/10.1207/S15327590IJHC1304_07.

28. Biduski D, Bellei EA, Rodriguez JP, Zaina LA, De Marchi AC. Assessing long-term user experience on a mobile health application through an in-app embedded conversation-based questionnaire. Comput Hum Behav. 2020; 104:106169. https://doi.org/10.1016/j.chb.2019.106169.

29. Pal S, Biswas B, Gupta R, Kumar A, Gupta S. Exploring the factors that affect user experience in mobile-health applications: a text-mining and machine-learning approach. J Bus Res. 2023; 156:113484. https://doi.org/10.1016/j.jbusres.2022.113484.

30. Ramachandran A, Snehalatha C, Ram J, Selvam S, Simon M, Nanditha A, et al. Effectiveness of mobile phone messaging in prevention of type 2 diabetes by lifestyle modification in men in India: a prospective, parallel-group, randomised controlled trial. Lancet Diabetes Endocrinol. 2013; 1(3):191–8. https://doi.org/10.1016/S2213-8587(13)70067-6.