J Educ Eval Health Prof.  2006;3:2.

Test Equating of the Medical Licensing Examination in 2003 and 2004 Based on the Item Response Theory

Affiliations
  • 1National Health Personnel Licensing Examination Board, Seoul, Korea.
  • 2Department of Parasitology, College of Medicine and Institute of Medical Education, Hallym University, Chuncheon, Korea. shuh@hallym.ac.kr

Abstract

The passing rate of the Medical Licensing Examination has been variable, which probably originated from the difference in the difficulty of items and/or difference in the ability level of examinees. We tried to explain the origin of the difference using the test equating method based on the item response theory. The number of items and examinees were 500, 3,647 in 2003 and 550, 3,879 in 2004. Common item nonequivalent group design was used for 30 common items. Item and ability parameters were calculated by three parametric logistic models using ICL. Scale transformation and true score equating were executed using ST and PIE. The mean of difficulty index of the year 2003 was -0.957 (SD 2.628) and that of 2004 after equating was -1.456 (SD 3.399). The mean of discrimination index of year 2003 was 0.487 (SD 0.242) and that of 2004 was 0.363 (SD 0.193). The mean of ability parameter of year 2003 was 0.00617 (SD 0.96605) and that of year 2004 was 0.94636 (SD 1.32960). The difference of the equated true score at the same ability level was high at the range of score of 200-350. The reason for the difference in passing rates over two consecutive years was due to the fact that the Examination in 2004 was easier and the abilities of the examinees in 2004 were higher. In addition, the passing rates of examinees with score of 270-294 in 2003, and those with 322-343 in 2004, were affected by the examination year.

Keyword

Test Equating; Education, Medical; licensing Examination; Item Response Theory

MeSH Terms

Discrimination (Psychology)
Education, Medical
Licensure*
Logistic Models

Figure

  • Fig. 1. Frequency distribution of discrimination parameter of Medical Licensing Examination analyzed based on item response theory in 2003 and 2004 after scale transformation.

  • Fig. 2. Frequency distribution of difficulty parameter of Medical Licensing Examination analyzed based on item response theory in 2003 and 2004 after scale transformation.

  • Fig. 3. Frequency distribution of ability parameter of examinees who took Medical Licensing Examination analyzed based on item response theory in 2003 and 2004 after scale transformation.

  • Fig. 4. Linear transformation of true score of year 2004 after equating to the score of year 2003 of Medical Licensing Examination.

  • Fig. 5. Gap of true score between true score of year 2004 after equating and the true score of year 2003 at the same ability level of Medical Licensing Examination.


Reference

1. Kolen MJ, Brennan RL. Test equating: method and practices. New York: Springer;1995.
2. Stocking ML, Lord FM. Developing a common metric in item response theory. Appl Psychol Meas. 1983; 7:201–10.
Article
3. Hanson BA. IRT command language [Internet] Version 0.020301. [cited 2005 Feb 2]. Available from: hhttp://www.b-a-h.com/software/irt/icl/.
4. A computer program for IRT scale transformation [Internet] Version 1.0. Iowa (IA): The University of Iowa, College of Education;Center for Advanced Studies in Measurement and Assessment: [date unknown]-[cited 2005 Feb 4]. Available from http://www.education.uiowa.edu/casma.
5. A computer program for Equating [Internet] Version 1.0. Iowa (IA): The University of Iowa, College of Education;Center for Advanced Studies in Measurement and Assessment: [date unknown]-[cited 2005 Feb 4]. Available from http://www.education.uiowa.edu/casma.
6. Hendrickson AM, Kolen MJ. IRT equating of MCAT [Internet]. Washington, DC: Association of American Medical College;c2003. [cited 2005 Feb 2]. Available form http://www.aamc.org/students/mcat/research/monograph4.pdf.
Full Text Links
  • JEEHP
Actions
Cited
CITED
export Copy
Close
Share
  • Twitter
  • Facebook
Similar articles
Copyright © 2024 by Korean Association of Medical Journal Editors. All rights reserved.     E-mail: koreamed@kamje.or.kr