J Educ Eval Health Prof.  2019;16:28. 10.3352/jeehp.2019.16.28.

Goodness of fit of the items used in the 2nd cycle of evaluation and accreditation of medical schools by the Korea Institute of Medical Education and Evaluation based on the Rasch model

• Lim MS
• Huh S

• KMID: 2502171
• DOI: http://doi.org/10.3352/jeehp.2019.16.28

Abstract

Purpose
Since 2004, the Korea Institute of Medical Education and Evaluation has been responsible for the evaluation and accreditation of medical schools in Korea. The 2nd cycle of evaluations was conducted from 2007 to 2011. The present study aimed at testing the goodness of fit of the items used in the 2nd cycle of evaluation and accreditation based on the Rasch model.
Methods
Dichotomous data on 40 medical schools were analyzed using Winsteps, a tool based on the Rasch model that includes goodness-of-fit testing.
Results
Two of the 109 items had an outfit mean square exceeding 2.0. The other 107 items showed a goodness of fit in the acceptable range for the outfit mean square. All items were in the acceptable range in terms of the infit mean square. Furthermore, 1 school had an outfit mean square exceeding 2.0, while all schools were in the acceptable range for the infit mean square. An outfit mean square value over 2.0 means that an item is a outlier. Therefore, 2 items showed an extreme response relative to the overall response. Meanwhile, the finding of an outfit mean square over 2.0 for 1 school means that it showed extraordinary responses to specific items, despite its excellent overall competency.
Conclusion
The goodness of fit of the items used for evaluation and accreditation by the Korea Institute of Medical Education and Evaluation should be checked so that they can be revised appropriately. Furthermore, the outlier school should be investigated to determine why it showed such an inappropriate goodness of fit.

Keyword

Medical education; Accreditation; Academies and institutes; Republic of Korea

Figure

• 

  Fig. 1. Correlation between the score obtained using classical test theory and measure (difficulty parameter) obtained using the Rasch model for the items (number of items=109; r=-0.9427; 95% confidence interval, -0.9605 to -0.0.9172; degrees of freedom=107; t=1.9824; P<0.0001).

• 

  Fig. 2. Correlation between the score obtained using classical test theory and measure (difficulty parameter) obtained using the Rasch model for 40 medical schools in Korea (number of items=40; 95% confidence interval, 0.09948 to 0.9986; r=0.9973; degrees of freedom=38; t=2.0244; P<0.0001).

Reference

References

1. Korean Institute of Medical Education and Evaluation. KIMEE leads to desirable medical education [Internet]. Seoul: Korean Institute of Medical Education and Evaluation;[cited 2109 Sep 15]. Available from: http://kimee.or.kr/en/kimee-2/.

2. Boone WJ. Rasch analysis for instrument development: why, when, and how? CBE Life Sci Educ. 2016; 15:rm4. https://doi.org/10.1187/cbe.16-04-0148.

3. Measured Progress. DIMPACK [Internet]. Dover (NH): Measured Progress;c2019. [cited 2019 Sep 15]. Available from: https://psychometrics.onlinehelp.measuredprogress.org/tools/dim/.

4. Linacre JM. What do infit and outfit, mean-square and standardized mean. Rasch Meas Trans. 2002; 16:878.