Healthc Inform Res.  2023 Oct;29(4):367-376. 10.4258/hir.2023.29.4.367.

Estimating Cognitive Load in a Mobile Personal Health Record Application: A Cognitive Task Analysis Approach

• Zayim N
• Yildiz H
• Yüce YK

• KMID: 2547689
• DOI: http://doi.org/10.4258/hir.2023.29.4.367

Abstract

Objectives
Mobile health applications that are designed without considering usability criteria can lead to cognitive overload, resulting in the rejection of these apps. To avoid this problem, the user interface of mobile health applications should be evaluated for cognitive load. This evaluation can contribute to the improvement of the user interface and help prevent cognitive overload for the user.
Methods
In this study, we evaluated a mobile personal health records application using the cognitive task analysis method, specifically the goals, operators, methods, and selection rules (GOMS) approach, along with the related updated GOMS model and gesture-level model techniques. The GOMS method allowed us to determine the steps of the tasks and categorize them as physical or cognitive tasks. We then estimated the completion times of these tasks using the updated GOMS model and gesture-level model.
Results
All 10 identified tasks were split into 398 steps consisting of mental and physical operators. The time to complete all the tasks was 5.70 minutes and 5.45 minutes according to the updated GOMS model and gesture-level model, respectively. Mental operators covered 73% of the total fulfillment time of the tasks according to the updated GOMS model and 76% according to the gesture-level model. The inter-rater reliability analysis yielded an average of 0.80, indicating good reliability for the evaluation method.
Conclusions
The majority of the task execution times comprised mental operators, suggesting that the cognitive load on users is high. To enhance the application's implementation, the number of mental operators should be reduced.

Keyword

Mental Health, Personal Health Records, Mobile Applications, Cognitive Science, User-Centered Design

Figure

• 

  Figure 1 Menu structure diagram of e-Nabız.

• 

  Figure 2 Average execution time for mental and physical operators in tasks according to the updated GOMS model and gesture-level model. GOMS: goals, operators, methods, selection rules.

Reference

References

1. Tang PC, Lansky D.The missing link: bridging the patient-provider health information gap. Health Aff (Millwood). 2005; 24(5):1290–5. https://doi.org/10.1377/hlthaff.24.5.1290.

2. Showell C.Barriers to the use of personal health records by patients: a structured review. PeerJ. 2017; 5:e3268. https://doi.org/10.7717/peerj.3268.

3. Kemp S. Digital 2021 October Global Statshot Report [Internet]. Singapore: DataReportal;2022. [cited at 2023 Oct 31]. Available from: https://datareportal.com/reports/digital-2021-october-global-statshot?rq=health.

4. Franklin R. 11 surprising mobile health statistics [Internet]. San Antonio (TX): Mobius MD;[cited at 2023 Oct 31]. Available from: https://mobius.md/2021/10/25/11-mobile-health-statistics.

5. Pachunka E, Windle J, Schuetzler R, Fruhling A. Natural-setting PHR usability evaluation using the NASA TLX to measure cognitive load of patients. In : Proceedings of the 52nd Hawaii International Conference on System Sciences; 2016 Jan 8–11; Maui, HI. p. 3954–63.

6. Ehrler F, Weinhold T, Joe J, Lovis C, Blondon K.A Mobile app (BEDSide Mobility) to support nurses’ tasks at the patient’s bedside: usability study. JMIR Mhealth Uhealth. 2018; 6(3):e57. https://doi.org/10.2196/mhealth.9079.

7. International Organization for Standardization. Ergonomics of human-system interaction - Part 11: Usability: Definitions and concepts. Geneva, Switzerland: International Organization for Standardization;2018. (ISO 9241-11:2018).

8. Baddeley A.Working memory: theories, models, and controversies. Annu Rev Psychol. 2012; 63:1–29. https://doi.org/10.1146/annurev-psych-120710-100422.

9. Cooper G. Research into cognitive load theory and instructional design at UNSW. Sydney, Australia: School of Education Studies, University of New South Wales;1998.

10. Balogh J, Cohen M, Giangola JP. Voice user interface design: minimizing cognitive load [Internet]. Hoboken (NJ): InformIT;2004. [cited at 2023 Oct 31]. Available from: https://www.informit.com/articles/article.aspx?p=170792.

11. Zu T, Hutson J, Loschky LC, Rebello NS. Use of eye-tracking technology to investigate cognitive load theory [Internet]. Ithaca (NY): arXiv.org;2018. [cited at 2023 Oct 31]. Available from: https://arxiv.org/abs/1803.02499.

12. Tracy JP, Albers MJ. Measuring cognitive load to test the usability of web sites. In : Proceedings of the 53rd Annual Conference of the Society for Technical Communication; 2006 May 7–10; Las Vegas, NV. p. 256–60.

13. Militello LG, Hutton RJ.Applied cognitive task analysis (ACTA): a practitioner’s toolkit for understanding cognitive task demands. Ergonomics. 1998; 41(11):1618–41. https://doi.org/10.1080/001401398186108.

14. Saitwal H, Feng X, Walji M, Patel V, Zhang J.Assessing performance of an Electronic Health Record (EHR) using Cognitive Task Analysis. Int J Med Inform. 2010; 79(7):501–6. https://doi.org/10.1016/j.ijmedinf.2010.04.001.

15. Kushniruk AW, Patel VL. Cognitive approaches to the evaluation of healthcare information systems. Anderson JG, Aydin CE, editors. Evaluating the organizational impact of healthcare information systems. New York (NY): Springer;2005. p. 144–73. https://doi.org/10.1007/0-387-30329-4_6.

16. Klein G, Militello L. Some guidelines for conducting a cognitive task analysis. Advances in human performance and cognitive engineering research. Bingley, UK: Emerald Group Publishing Limited;2001. p. 163–99. https://doi.org/10.1016/S1479-3601(01)01006-2.

17. Card SK, Moran TP, Newell A. The psychology of human-computer interaction. Boca Raton (FL): CRC Press;2018.

18. Rasmussen R, Kushniruk A.The long and twisting path: an efficiency evaluation of an electronic whiteboard system. Stud Health Technol Inform. 2013; 183:174–8.

19. Gross M, Roberts C, Stinson K, Wiltman S.Improving medical device usability by reducing complexity using a novel predictive models-based user interface assessment tool. Hum Factors Healthc. 2023; 3:100041. https://doi.org/10.1016/j.hfh.2023.100041.

20. Setthawong P, Setthawong R.2019; Updated Goals Operators Methods and Selection Rules (GOMS) with touch screen operations for quantitative analysis of user interfaces. Int J Adv Sci Eng Inf Technol. 2019; 9(1):258–65. https://doi.org/10.18517/IJASEIT.9.1.7865.

21. Card SK, Moran TP, Newell A.The keystroke-level model for user performance time with interactive systems. Commun ACM. 1980; 23(7):396–410. https://doi.org/10.1145/358886.358895.

22. Nystrom A. Gesture-level model: a modified Keystroke-level model for tasks on mobile touchscreen devices [master’s thesis]. Uppsala, Sweden: Uppsala University;2018.

23. Kim D, Myung R.A unit touch gesture model of performance time prediction for mobile devices. J Ergon Soc Korea. 2016; 35(4):277–91. http://doi.org/10.5143/JESK.2016.35.4.277.

24. Li H, Liu Y, Liu J, Wang X, Li Y, Rau PL. Extended KLM for mobile phone interaction: a user study result. In : Proceedings of the Extended Abstracts on Human Factors in Computing Systems; 2010 Apr 10–15; Atlanta, GA. p. 3517–22. https://doi.org/10.1145/1753846.1754011.

25. Republic of Turkiye, Ministry of Health. General directorate of health information systems [Internet]. Ankara: Republic of Turkiye;2021. [cited at 2023 Oct 31]. Available from: https://sbsgm.saglik.gov.tr/TR,73567/her-4-kisiden-biri-e-nabiz-kullaniyor.html.

26. John BE, Kieras DE. Using GOMS for user interface design and evaluation: which technique? ACM Trans Comput Hum Interact. 1996; 3(4):287–319. https://doi.org/10.1145/235833.236050.

27. Al Ghalayini M, Antoun J, Moacdieh NM.Too much or too little? Investigating the usability of high and low data displays of the same electronic medical record. Health Informatics J. 2020; 26(1):88–103. https://doi.org/10.1177/1460458218813725.

28. Suebnukarn S, Rittipakorn P, Thongyoi B, Boonpitak K, Wongsapai M, Pakdeesan P.Usability assessment of an electronic health record in a comprehensive dental clinic. SpringerPlus. 2013; 2:220. https://doi.org/10.1186/2193-1801-2-220.