J Korean Med Sci.  2024 Aug;39(33):e239. 10.3346/jkms.2024.39.e239.

Subtyping of Performance Trajectory During Medical School, Medical Internship, and the First Year of Residency in Training Physicians: A Longitudinal Cohort Study

• Yun JY
• Ryu H
• Kim JW
• Yoon HB
• Choi S
• Park WB
• Bae EJ
• Yim JJ
• Myung SJ

• KMID: 2559227
• DOI: http://doi.org/10.3346/jkms.2024.39.e239

Abstract

Background
Developmental trajectories of clinical skills in training physicians vary among tasks and show interindividual differences. This study examined the predictors of medical internship performance and residency entrance and found subtypes of performance trajectory in training physicians.
Methods
This retrospective cohort study involved 888 training physicians who completed a medical internship between 2015 and 2019. After the internship, 627 physicians applied for residency training between 2016 and 2020. Finally, 160 of them completed their first-year residency in internal medicine, surgery, pediatrics, and psychiatry departments between 2016 and 2020. Pearson’s correlation coefficients of internship performance and first year-residency performance (n = 160) were calculated. Latent profile analysis identified performance trajectory subtypes according to medical school grade point average (GPA), internship performance, English proficiency, and residency selection procedures. Multivariate logistic regression models of residency acceptance (n = 627) and performance in the top 30%/lower 10% in the first year of residency were also constructed.
Results
Medical internship performance showed a significant positive correlation with the medical school GPA (r = 0.194) and the written score for the medical licensing examination (r = 0.125). Higher scores in the interview (adjusted odds ratio [aOR], 2.57) and written examination (aOR, 1.45) of residency selection procedures and higher medical internship performance (aOR, 1.19) were associated with a higher chance of residency acceptance. The latent profile analyses identified three training physician subgroups: average performance, consistently high performance (top 30%), and adaptation to changes (lowest 10%). Higher scores in the interview for residency selection (aOR, 1.35) and lower scores for medical internship performance (aOR, 0.79) were associated with a higher chance of performing in the top 30% or lowest 10% in the first year of residency, respectively.
Conclusion
Performance in the interview and medical internship predicted being among the top 30% and lowest 10% of performers in the first year of residency training, respectively. Individualized educational programs to enhance the prospect of trainees becoming highfunctioning physicians are needed.

Keyword

Graduate Medical Education Training; Medical Internship; First Year of Medical Residency; Workplace-Based Assessment; Latent Profile Analysis; Multivariate Logistic Regression

Figure

• 

  Fig. 1 Receiver operating curves predicting residency acceptance in 627 training physicians who completed a medical internship between 2015 and 2019 and applied for the medical residency program between 2016 and 2020 based on medical school GPA, medical internship performance, English proficiency tests (TOEIC/TOEFL/TEPS), and written exam/practical exam/interview for residency selection.GPA = grade point average, TOEIC = Test of English for International Communication, TOEFL = Test of English as a Foreign Language, TEPS = Test of English Proficiency developed by Seoul National University.

• 

  Fig. 2 Subtypes of performance trajectory in training physicians. (A) Latent profile analysis divided 160 training physicians who completed both a medical internship between 2015 and 2019 and the first year of residency training between 2016 and 2020 into three subgroups: average (n = 106), consistently high achiever (n = 37), and adapting to challenges (n = 17). (B) Correlation matrix (Pearson’s correlation coefficients are shown and are marked with red rims when P < 0.05; n = 160) among the zGPA, zLICENSE, zINTERN, zENG, scores for residency selection procedures including the written examination (zIGOP), zINTERVIEW, and zPRACTICE, and zR1raw.zGPA = z-score–transformed medical school GPA, zLICENSE = z-score–transformed medical licensing examination score, zINTERN = z-score–transformed medical internship performance score, zENG = z-score–transformed TOEIC/TOEFL/TEPS scores, zIGOP = z-score–transformed internal medicine, general surgery, obstetrics and gynecology, and pediatrics, zINTERVIEW = z-score–transformed interview, zPRACTICE = z-score–transformed practical test, zR1raw = z-score–transformed first-year residency performance score.

• 

  Fig. 3 Associations between average residency performance in the first year of residency and (A) zGPA, (B) zLICENSE, and (C) zINTERN, as well as the (D) written examination (zIGOP) and (E) zINTERVIEW during the residency selection procedures (all x-axis), in 160 training physicians who completed both a medical internship between 2015 and 2019 and their first year of residency training between 2016 and 2020 (n = 66 from internal medicine [blue], n = 25 from general surgery [red], n = 46 from pediatrics [yellow], and n = 23 from psychiatry [purple]).zGPA = z-score–transformed medical school GPA, zLICENSE = z-score–transformed medical licensing examination score, zINTERN = z-score–transformed medical internship performance score, zIGOP = z-score–transformed internal medicine, general surgery, obstetrics and gynecology, and pediatrics, zINTERVIEW = z-score–transformed interview.

Reference

1. Farah G, Gonzalez MH, Choubey AS, Karam M, Igram C, Dougherty P, et al. Be a lighthouse for your medical students: how to help them navigate during changing times to a successful match. Instr Course Lect. 2024; 73:87–95. PMID: 38090889.

2. Tanaka P, Park YS, Roby J, Ahn K, Kakazu C, Udani A, et al. Milestone learning trajectories of residents at five anesthesiology residency programs. Teach Learn Med. 2021; 33(3):304–313. PMID: 33327788.

3. Colbert-Getz JM, Lappe K, Gerstenberger J, Milne CK, Raaum S. Capturing growth curves of medical students’ clinical skills performance. Clin Teach. 2023; 20(6):e13623. PMID: 37605795.

4. Rutgers DR, van Raamt F, Ten Cate TJ. Development of competence in volumetric image interpretation in radiology residents. BMC Med Educ. 2019; 19(1):122. PMID: 31046749.

5. Quick JA, Kudav V, Doty J, Crane M, Bukoski AD, Bennett BJ, et al. Surgical resident technical skill self-evaluation: increased precision with training progression. J Surg Res. 2017; 218:144–149. PMID: 28985841.

6. Cassidy DJ, Chakraborty S, Panda N, McKinley SK, Mansur A, Hamdi I, et al. The surgical knowledge “Growth Curve”: predicting ABSITE scores and identifying “At-Risk” residents. J Surg Educ. 2021; 78(1):50–59. PMID: 32694087.

7. Nazerali-Maitland A, Douglas C. Challenges with international medical graduate selection: finding positive attributes predictive of success in family medicine residency. BMC Prim Care. 2022; 23(1):256. PMID: 36175829.

8. Ju TR, Mikrut EE, Spinelli A, Romain AM, Brondolo E, Sundaram V, et al. Factors associated with burnout among resident physicians responding to the COVID-19 pandemic: a 2-month longitudinal observation study. Int J Environ Res Public Health. 2022; 19(15):9714. PMID: 35955071.

9. Seaberg PH, Kling JM, Klanderman MC, Mead-Harvey C, Williams KE, Labonte HR, et al. Resident factors associated with American board of internal medicine certification exam failure. Med Educ Online. 2023; 28(1):2152162. PMID: 36443907.

10. Mamede S, Zandbergen A, de Carvalho-Filho MA, Choi G, Goeijenbier M, van Ginkel J, et al. Role of knowledge and reasoning processes as predictors of resident physicians’ susceptibility to anchoring bias in diagnostic reasoning: a randomised controlled experiment. BMJ Qual Saf. 2024.

11. Jones AT, Brethauer SA, Dent DL, Desai DM, Jeyarajah R, Barry CL, et al. How does the sequence of the American board of surgery examinations impact pass/fail outcomes? Ann Surg. 2024; 279(1):187–190. PMID: 37470170.

12. Reisdorff EJ, Johnston MM, Kraus CK, Keim SM, Santen SA. Association between the American board of emergency medicine oral certifying examination and future state medical board disciplinary actions. J Am Coll Emerg Physicians Open. 2024; 5(1):e13119. PMID: 38322376.

13. Govindarajan V, Shah AH, Morell AA, Borowy V, Ingle SM, Mendez Valdez MJ, et al. Predicting academic career placement via development of novel intra-residency metrics. World Neurosurg. 2023; 174:e35–e43. PMID: 36841537.

14. Spurk D, Hirschi A, Wang M, Valero D, Kauffeld S. Latent profile analysis: a review and “how to” guide of its application within vocational behavior research. J Vocat Behav. 2020; 120:103445.

15. Scrucca L, Fop M, Murphy TB, Raftery AE. mclust 5: clustering, classification and density estimation using gaussian finite mixture models. R J. 2016; 8(1):289–317. PMID: 27818791.

16. Song X, Jia Y. Using latent class growth analysis to detect group developmental trajectories in preclinical medical education. Adv Health Sci Educ Theory Pract. 2024; 29(3):803–812. PMID: 37679596.

17. Grbic D, Gielissen KA, Obeso V, Amiel JM, Jayas A, Andriole DA. Core entrustable professional activities for entering residency: a national survey of graduating medical students’ self-assessed skills by specialty. J Am Coll Surg. 2022; 235(6):940–951. PMID: 36102502.

18. Herzog TL, Sawatsky AP, Kelm DJ, Nelson DR, Park JG, Niven AS. The resident learning journey in the medical intensive care unit. ATS Sch. 2023; 4(2):177–190. PMID: 37533538.

19. Silkens ME, Chahine S, Lombarts KM, Arah OA. From good to excellent: improving clinical departments’ learning climate in residency training. Med Teach. 2018; 40(3):237–243. PMID: 29172795.

20. Sorenson JM, Khan NR, Michael LM 2nd, Nguyen V, Baughman B, Boop FA, et al. Case curve: a novel web-based platform and mobile phone application to evaluate surgical competence in graduate medical education. Neurosurgery. 2024; 94(6):1237–1245.

21. Nguyen ET, Berler MH, Gonzales PA, Greenberg AL, Lebares CC, Divino C, et al. Flourishing and the prioritization of workplace elements in general surgery residents. J Surg Res. 2023; 291:488–495. PMID: 37536190.

22. Ryan MS, Gielissen KA, Shin D, Perera RA, Gusic M, Ferenchick G, et al. How well do workplace-based assessments support summative entrustment decisions? A multi-institutional generalisability study. Med Educ. 2024; 58(7):825–837. PMID: 38167833.

23. Zavodnick J, Doroshow J, Rosenberg S, Banks J, Leiby BE, Mingioni N. Hawks and doves: perceptions and reality of faculty evaluations. J Med Educ Curric Dev. 2023; 10:23821205231197079. PMID: 37692558.

24. Tooley AA, Law J, Lelli GJ, Sun G, Godfrey KJ, Tran AQ, et al. Predictors of ophthalmology resident performance from medical student application materials. J Surg Educ. 2024; 81(1):151–160. PMID: 38036387.

25. Hemrajani R, Vettese T, Law K, Turbow S. Association of interview and holistic review metrics with resident performance-related difficulties in an internal medicine residency program. J Grad Med Educ. 2023; 15(5):564–571. PMID: 37781425.

26. Wang A, Karunungan KL, Story JD, Shlobin NA, Woo J, Ha EL, et al. Reimagining a pass/fail clinical core clerkship: a US residency program director survey and meta-analysis. BMC Med Educ. 2023; 23(1):788. PMID: 37875929.

27. Chen F, Arora H, Martinelli SM, Teeter E, Mayer D, Zvara DA, et al. The predictive value of pre-recruitment achievement on resident performance in anesthesiology. J Clin Anesth. 2017; 39:139–144. PMID: 28494890.

28. Brothers TE, Wetherholt S. Importance of the faculty interview during the resident application process. J Surg Educ. 2007; 64(6):378–385. PMID: 18063273.

29. Alterman DM, Jones TM, Heidel RE, Daley BJ, Goldman MH. The predictive value of general surgery application data for future resident performance. J Surg Educ. 2011; 68(6):513–518. PMID: 22000538.

30. Bhat R, Takenaka K, Levine B, Goyal N, Garg M, Visconti A, et al. Predictors of a top performer during emergency medicine residency. J Emerg Med. 2015; 49(4):505–512. PMID: 26242925.

31. Quinn JK, Mongelluzzo J, Nip A, Graterol J, Chen EH. Perception of quiet students in emergency medicine: an exploration of narratives in the standardized letter of evaluation. West J Emerg Med. 2023; 24(4):728–731. PMID: 37527382.

32. Drum B, Shi J, Peterson B, Lamb S, Hurdle JF, Gradick C. Using natural language processing and machine learning to identify internal medicine-pediatrics residency values in applications. Acad Med. 2023; 98(11):1278–1282. PMID: 37506388.

33. Drum B, Lamb S, Gradick C. Values-based resident selection in an internal medicine-pediatrics residency program. J Gen Intern Med. 2023; 38(6):1410–1416. PMID: 36344647.