J Korean Med Sci.  2023 May;38(17):e133. 10.3346/jkms.2023.38.e133.

Impact of Criterion Versus NormReferenced Assessment on the Quality of Life in Korean Medical Students

• Park CH
• Kwon J
• Lee JT
• Ahn S

• KMID: 2542022
• DOI: http://doi.org/10.3346/jkms.2023.38.e133

Abstract

Background
Medical students are known to be subjected to immense stress under competitive curricula and have a high risk of depression, burnout, anxiety and sleep disorders. There is a global trend of switching from norm-referenced assessment (NRA) to criterion-referenced assessment (CRA), and these changes may have influenced the quality of life (QOL), sleep phase, sleep quality, stress, burnout, and depression of the medical students. We hypothesized that there is a significant difference of QOL between CRA and NRA and that sleep, stress, burnout, and depression are the main contributors.
Methods
By administering an online survey regarding QOL and its contributors to Korean medical students, 365 responses from 10 medical schools were recorded. To clarify the complex relationship between the multiple factors in play, we applied nonlinear machine learning algorithms and utilized causal structure learning techniques on the survey data.
Results
Students with CRA had lower scores in stress (68.16 ± 11.29, 76.03 ± 12.38, P< 0.001), burnout (48.09 ± 11.23, 55.93 ± 13.07, P < 0.001), depression (12.77 ± 9.82, 16.44 ± 11.27, P = 0.003) and higher scores in QOL (95.79 ± 16.20, 89.65 ± 16.28, P < 0.001) compared with students with NRA. Multiple linear regression, permutation importance of the random forest model and the causal structure model showed that depression, stress and burnout are the most influential factors of QOL of medical students.
Conclusion
Medical students from schools that use CRA showed higher QOL scores, as well as lower burnout, stress and depression when compared with students from schools that use NRA. These results may be used as a basis for granting justification for the transition to CRA.

Keyword

Criterion-Referenced Assessment; Medical Education; Quality of Life; Stress; Burnout; Depression

Figure

• 

  Fig. 1 Partial dependence plots (dotted line) and individual conditional expectations (solid lines) in (A) linear regression model and (B) random forest model. Boxplots present the permutation importance of each independent variable.QOL = quality of life, rMEQ = reduced Morningness–Eveningness Questionnaire, PSQI = Pittsburgh Sleep Quality Index, MSSIK = Medical Student Stress Inventory in Korea, K-MBI-SS = Korean Maslach Burnout Inventory Student Survey, CESD = Center for Epidemiologic Studies Depression Scale.

• 

  Fig. 2 Directed acyclic graph depicting causal structure network between group, five factors and QOL. Group is a variable that divides participants into two groups by the grading system (criterion-referenced assessment, norm-referenced assessment). Numbers on each edge indicate the weight value of the causal relationship.QOL = quality of life.

Reference

1. Bloodgood RA, Short JG, Jackson JM, Martindale JR. A change to pass/fail grading in the first two years at one medical school results in improved psychological well-being. Acad Med. 2009; 84(5):655–662. PMID: 19704204.

2. Reed DA, Shanafelt TD, Satele DW, Power DV, Eacker A, Harper W, et al. Relationship of pass/fail grading and curriculum structure with well-being among preclinical medical students: a multi-institutional study. Acad Med. 2011; 86(11):1367–1373. PMID: 21952063.

3. Kim IS, Yang EB, Jeon WT. Analysis of internal factors affecting learning of medical student in criterion-referenced assessment system. Korean J Educ Res. 2015; 53(4):283–303.

4. Swan Sein A, Rashid H, Meka J, Amiel J, Pluta W. Twelve tips for embedding assessment for and as learning practices in a programmatic assessment system. Med Teach. 2021; 43(3):300–306. PMID: 32658603.

5. Spring L, Robillard D, Gehlbach L, Simas TA. Impact of pass/fail grading on medical students’ well-being and academic outcomes. Med Educ. 2011; 45(9):867–877. PMID: 21848714.

6. Rohe DE, Barrier PA, Clark MM, Cook DA, Vickers KS, Decker PA. The benefits of pass-fail grading on stress, mood, and group cohesion in medical students. Mayo Clin Proc. 2006; 81(11):1443–1448. PMID: 17120399.

7. Association of American Medical Colleges. Grading systems use by US medical schools. Updated 2022. Accessed August 23, 2022. https://www.aamc.org/data-reports/curriculum-reports/interactive-data/grading-systems-use-us-medical-schools .

8. Chang HK, Lee SJ, Park CS, Kim BJ, Lee CS, Cha B, et al. Association between quality of life and eveningness as well as sleep quality among medical students. Sleep Med Psychophysiol. 2015; 22(2):64–69.

9. Park BS, Park KH. Quality of life and its related factors between medical students and non medical students. Asia Pac J Multimed Serv Converg Art Humanit Sociol. 2017; 7(9):577–588.

10. Yune S, Im S, Lee SY, Baek SY, Kam BS. Relationships among test anxiety, academic burnout, resilience, and academic achievement of medical school students. J Educ Innov Res. 2018; 28(4):173–188.

11. Kim B, Roh H. Depressive symptoms in medical students: prevalence and related factors. Korean J Med Educ. 2014; 26(1):53–58. PMID: 25805080.

12. Dyrbye LN, West CP, Satele D, Boone S, Tan L, Sloan J, et al. Burnout among U.S. medical students, residents, and early career physicians relative to the general U.S. population. Acad Med. 2014; 89(3):443–451. PMID: 24448053.

13. Loureiro F, Garcia-Marques T. Morning or evening person? Which type are you? Self-assessment of chronotype. Pers Individ Dif. 2015; 86:168–171.

14. Sohn SI, Kim DH, Lee MY, Cho YW. The reliability and validity of the Korean version of the Pittsburgh Sleep Quality Index. Sleep Breath. 2012; 16(3):803–812. PMID: 21901299.

15. Kim MJ, Park KH, Yoo HH, Park IB, Yim J. Development and validation of the medical student stress scale in Korea. Korean J Med Educ. 2014; 26(3):197–208. PMID: 25805262.

16. Lee S, Lee D. Validation of the MBI-SS scales-based on medical school students. Asian J Educ. 2013; 14(2):165–187.

17. Chon KK, Choi SC, Yang BC. Integrated adaptation of CES-D in Korea. Korean J Health Psychol. 2001; 6(1):59–76.

18. Min SK, Lee CI, Kim KI, Suh SY, Kim DK. Development of Korean version of WHO quality of life scale abbreviated version (WHOQOL-BREF). J Korean Neuropsychiatr Assoc. 2000; 39(3):571–579.

19. Krägeloh CU, Henning MA, Hawken SJ, Zhao Y, Shepherd D, Billington R. Validation of the WHOQOL-BREF quality of life questionnaire for use with medical students. Educ Health (Abingdon). 2011; 24(2):545. PMID: 22081657.

20. World Health Organization. WHOQOL-BREF: Introduction, Administration, Scoring and Generic Version of the Assessment: Field Trial Version, December 1996. Geneva, Switzerland: World Health Organization;1996.

21. Goldstein A, Kapelner A, Bleich J, Pitkin E. Peeking inside the black box: Visualizing statistical learning with plots of individual conditional expectation. J Comput Graph Stat. 2015; 24(1):44–65.

22. Zheng X, Aragam B, Ravikumar P, Xing EP. Dags with no tears: continuous optimization for structure learning. Adv Neural Inf Process Syst. Forthcoming 2023. DOI: 10.48550/arXiv.1803.01422.

23. Puthran R, Zhang MW, Tam WW, Ho RC. Prevalence of depression amongst medical students: a meta-analysis. Med Educ. 2016; 50(4):456–468. PMID: 26995484.

24. Jeong Y, Kim JY, Ryu JS, Lee KE, Ha EH, Park H. The associations between social support, health-related behaviors, socioeconomic status and depression in medical students. Epidemiol Health. 2010; 32:e2010009. PMID: 21191462.

25. Gupchup GV, Borrego ME, Konduri N. The impact of student life stress on health related quality of life among doctor of pharmacy students. Coll Stud J. 2004; 38(2):292–302.

26. Song WI, Hwang JW, Oh ES. Relations among leisure limitation, coping stress, and quality of life in college students. J Korea Soc Wellness. 2016; 11(3):159–167.

27. Pagnin D, de Queiroz V. Influence of burnout and sleep difficulties on the quality of life among medical students. Springerplus. 2015; 4(1):676. PMID: 26558179.

28. Kim JW, Han DH, Lee YS, Min KJ, Park JY, Lee K. The effect of depression, anxiety, self-esteem, temperament, and character on life satisfaction in college students. J Korean Neuropsychiatr Assoc. 2013; 52(3):150–156.

29. Arslan G, Ayranci U, Unsal A, Arslantas D. Prevalence of depression, its correlates among students, and its effect on health-related quality of life in a Turkish university. Ups J Med Sci. 2009; 114(3):170–177. PMID: 19736608.

30. Miranda-Ackerman RC, Barbosa-Camacho FJ, Sander-Möller MJ, Buenrostro-Jiménez AD, Mares-País R, Cortes-Flores AO, et al. Burnout syndrome prevalence during internship in public and private hospitals: a survey study in Mexico. Med Educ Online. 2019; 24(1):1593785. PMID: 30957686.

31. Zhou W, Pu J, Zhong X, Yang W, Teng T, Fan L, et al. Overlap of burnout-depression symptoms among Chinese neurology graduate students in a national cross-sectional study. BMC Med Educ. 2021; 21(1):83. PMID: 33530985.

32. Kim RB, Park KS, Lee JH, Kim BJ, Chun JH. Factors related to depression symptom and the influence of depression symptom on self-rated health status, outpatient health service utilization and quality of life. Korean J Health Educ Promot. 2011; 28(1):81–92.

33. Seo JH, Kim HJ, Kim BJ, Lee SJ, Bae HO. Educational and relational stressors associated with burnout in Korean medical students. Psychiatry Investig. 2015; 12(4):451–458.

34. Ryu S. Quality of life and quality of sleep in medical college students. J Korean Soc Biol Ther Psychiatry. 2009; 15(1):29–36.

35. Oginska H, Oginska-Bruchal K. Chronotype and personality factors of predisposition to seasonal affective disorder. Chronobiol Int. 2014; 31(4):523–531. PMID: 24397301.

36. Wolf MR, Rosenstock JB. Inadequate sleep and exercise associated with burnout and depression among medical students. Acad Psychiatry. 2017; 41(2):174–179. PMID: 26976402.

37. Grandner MA, Kripke DF, Yoon IY, Youngstedt SD. Criterion validity of the Pittsburgh Sleep Quality Index: investigation in a non-clinical sample. Sleep Biol Rhythms. 2006; 4(2):129–139. PMID: 22822303.

38. Lagnado DA, Sloman S. Learning causal structure. In : Proceedings of the Annual Meeting of the Cognitive Science Society; 2002 Aug 7–10; Fairfax, VA, USA. Seattle, WA, USA: Cognitive Science Society;2002.

39. Morgan TL, McFadden T, Fortier MS, Tomasone JR, Sweet SN. Positive mental health and burnout in first to fourth year medical students. Health Educ J. 2020; 79(8):948–962.

40. Cecil J, McHale C, Hart J, Laidlaw A. Behaviour and burnout in medical students. Med Educ Online. 2014; 19(1):25209. PMID: 25160716.

41. Park H, Lee YM, Ho MJ, Han HC. How the coronavirus disease 2019 pandemic changed medical education and deans’ perspectives in Korean medical schools. Korean J Med Educ. 2021; 33(2):65–74. PMID: 33957729.