Healthc Inform Res.  2019 Oct;25(4):332-337. 10.4258/hir.2019.25.4.332.

Factors Associated with Daily Completion Rates in a Smartphone-Based Ecological Momentary Assessment Study

• Yang YS
• Ryu GW
• Choi M

• KMID: 2462232
• DOI: http://doi.org/10.4258/hir.2019.25.4.332

Abstract

OBJECTIVES
Ecological momentary assessment (EMA) methods are known to have validity for capturing momentary changes in variables over time. However, data quality relies on the completion rates, which are influenced by both participants' characteristics and study designs. This study applied an EMA method using a mobile application to assess momentary moods and stress levels in patients with Moyamoya disease to examine variables associated with EMA completion rates.
METHODS
Adults with Moyamoya disease were recruited from a tertiary hospital in Seoul. Patients with cognitive impairment were excluded. The EMA survey was loaded as a mobile application onto the participants' personal smartphones. Notifications were sent at semi-random intervals four times a day for seven consecutive days. Daily completion rates were calculated as the percentage of completed responses per day; overall completion rates were calculated as the proportion of completed responses per total of the 28 scheduled measures in the study and assessed through a descriptive analysis, t-test, ANOVA, and regression analysis, with mixed modeling to identify the point at which the daily completion rate significantly decreased.
RESULTS
A total of 98 participants responded (mean age, 41.00 ± 10.30 years; 69.4% female; 75.5% married). The overall completion rate was 70.66%, with no gender or age differences found. The daily completion rate decreased significantly after day 5 (p = 0.029).
CONCLUSIONS
Obtaining a good completion rate is essential for quality data in EMA methods. Strategic approaches to a study design should be established to encourage participants throughout a study to improve completion rates.

Keyword

Data Accuracy; Ecological Momentary Assessment; Epidemiologic Factors; Guideline Adherence; Mobile Applications

MeSH Terms

Adult
Cognition Disorders
Data Accuracy
Epidemiologic Factors
Female
Guideline Adherence
Humans
Methods
Mobile Applications
Moyamoya Disease
Seoul
Smartphone
Tertiary Care Centers

Figure

• 

  Figure 1 Distribution of completion rates.

• 

  Figure 2 Daily completion rates by days.

Reference

1. Shiffman S, Stone AA, Hufford MR. Ecological momentary assessment. Annu Rev Clin Psychol. 2008; 4:1–32.

2. Augustine AA, Larsen RJ. Emotion research. In : Mehl MR, Conner TS, editors. Handbook of research methods for studying daily life. New York (NY): Guilford Press;2012. p. 497–510.

3. Yang YS, Ryu GW, Choi M. Methodological strategies for ecological momentary assessment to evaluate mood and stress in adult patients using mobile phones: systematic review. JMIR Mhealth Uhealth. 2019; 7(4):e11215.

4. Stone AA, Shiffman S. Capturing momentary, self-report data: a proposal for reporting guidelines. Ann Behav Med. 2002; 24(3):236–243.

5. Courvoisier DS, Eid M, Lischetzke T. Compliance to a cell phone-based ecological momentary assessment study: the effect of time and personality characteristics. Psychol Assess. 2012; 24(3):713–720.

6. Messiah A, Grondin O, Encrenaz G. Factors associated with missing data in an experience sampling investigation of substance use determinants. Drug Alcohol Depend. 2011; 114(2-3):153–158.

7. Ono M, Schneider S, Junghaenel DU, Stone AA. What affects the completion of ecological momentary assessments in chronic pain research? An individual patient data meta-analysis. J Med Internet Res. 2019; 21(2):e11398.

8. Yang YS, Ryu GW, Han I, Oh S, Choi M. Ecological momentary assessment using smartphone-based mobile application for affect and stress assessment. Healthc Inform Res. 2018; 24(4):381–386.

9. Kang Y, Na DL, Hahn S. A validity study on the Korean Mini-Mental State Examination (K-MMSE) in dementia patients. J Korean Neurol Assoc. 1997; 15(2):300–307.

10. Verhagen SJ, Berben JA, Leue C, Marsman A, Delespaul PA, van Os J, et al. Demonstrating the reliability of transdiagnostic mHealth Routine Outcome Monitoring in mental health services using experience sampling technology. PLoS One. 2017; 12(10):e0186294.

11. Kramer I, Simons CJ, Hartmann JA, Menne-Lothmann C, Viechtbauer W, Peeters F, et al. A therapeutic application of the experience sampling method in the treatment of depression: a randomized controlled trial. World Psychiatry. 2014; 13(1):68–77.

12. Jacobs N, Menne-Lothmann C, Derom C, Thiery E, van Os J, Wichers M. Deconstructing the familiality of variability in momentary negative and positive affect. Acta Psychiatr Scand. 2013; 127(4):318–327.

13. Snippe E, Simons CJ, Hartmann JA, Menne-Lothmann C, Kramer I, Booij SH, et al. Change in daily life behaviors and depression: within-person and between-person associations. Health Psychol. 2016; 35(5):433–441.

14. van Os J, Verhagen S, Marsman A, Peeters F, Bak M, Marcelis M, et al. The experience sampling method as an mHealth tool to support self-monitoring, self-insight, and personalized health care in clinical practice. Depress Anxiety. 2017; 34(6):481–493.

15. Powell DJ, Schlotz W. Daily life stress and the cortisol awakening response: testing the anticipation hypothesis. PLoS One. 2012; 7(12):e52067.

16. Petrowski K, Paul S, Albani C, Brahler E. Factor structure and psychometric properties of the trier inventory for chronic stress (TICS) in a representative German sample. BMC Med Res Methodol. 2012; 12:42.

17. Petrowski K, Bastianon CD, Buhrer S, Brahler E. Air quality and chronic stress: a representative study of air pollution (PM2.5, PM10) in Germany. J Occup Environ Med. 2019; 61(2):144–147.

18. Delespaul PA. Assessing schizophrenia in daily life: the experience sampling method. Maastricht, the Netherlands: Maastricht University;1995.

19. Abdel-Kader K, Jhamb M, Mandich LA, Yabes J, Keene RM, Beach S, et al. Ecological momentary assessment of fatigue, sleepiness, and exhaustion in ESKD. BMC Nephrol. 2014; 15:29.

20. Moore RC, Kaufmann CN, Rooney AS, Moore DJ, Eyler LT, Granholm E, et al. Feasibility and acceptability of ecological momentary assessment of daily functioning among older adults with HIV. Am J Geriatr Psychiatry. 2017; 25(8):829–840.

21. Band R, Barrowclough C, Emsley R, Machin M, Wearden AJ. Significant other behavioural responses and patient chronic fatigue syndrome symptom fluctuations in the context of daily life: an experience sampling study. Br J Health Psychol. 2016; 21(3):499–514.

22. Band R, Barrowclough C, Caldwell K, Emsley R, Wearden A. Activity patterns in response to symptoms in patients being treated for chronic fatigue syndrome: an experience sampling methodology study. Health Psychol. 2017; 36(3):264–269.

23. Okifuji A, Bradshaw DH, Donaldson GW, Turk DC. Sequential analyses of daily symptoms in women with fibromyalgia syndrome. J Pain. 2011; 12(1):84–93.

24. Aaron LA, Mancl L, Turner JA, Sawchuk CN, Klein KM. Reasons for missing interviews in the daily electronic assessment of pain, mood, and stress. Pain. 2004; 109(3):389–398.