J Educ Eval Health Prof.  2007;4:1.

Reconsidering the Cut Score of Korean National Medical Licensing Examination

• Ahn DS
• Ahn S

Abstract

After briefly reviewing theories of standard setting we analyzed the problems of the current cut scores. Then, we reported the results of need assessment on the standard setting among medical educators and psychometricians. Analyses of the standard setting methods of developed countries were reported as well. Based on these findings, we suggested the Bookmark and the modified Angoff methods as alternative methods for setting standard. Possible problems and challenges were discussed when these methods were applied to the National Medical Licensing Examination.

Keyword

Cut Score; Standard Setting; Bookmark Method; Modified Angoff Method; Psychometrics

MeSH Terms

Developed Countries
Licensure*
Psychometrics

Figure

• 

  Fig. 1. An example of the application of the current 60% cut score.

Reference

1. Norcini JJ. Setting standards on educational tests. Med Educ. 2003; 37:464–9.

2. Zieky MJ. So much has changed: How the setting of cutscores has evolved since the 1980s. Cizek GJ, editor. Setting performance standards. Mahwah (NJ): Lawrence Erlbaum Assoociates;2001. p. 19–52.

3. Mehrens WA. Methodological issues in standard setting for educational exams. In : Crocker L, Zieky M, editors. Proceedings of Joint Conference on Standard Setting for Large-Scale Assessments; 1994 Oct 5–7; Washington, DC. Washington, DC: U.S. Government Printing Office;c2005. p. 221–63.

4. Lewis DM, Mitzel HC, Green DR. Standard setting: a bookmark approach. Paper presented at: IRT-based standard setting procedures utilizing behavioral anchoring. The 1996 Council of Chief State School Officers National Conference on Large-Scale Assessment. 1996 June; Pheonix, AZ.

5. Lee G. A psychometric approach to setting a passing score on Korean National Medical Licensing Examination. J Educ Eval Health Prof. 2004; 1:5–14.

6. Mehrens WA, Popham WJ. How to evaluate the legal defensibility of high-stakes tests. Appl Meas Educ. 1992; 5:265–83.

7. Cizek GJ. An NCME instructional module on: Setting passing scores. Educa Meas: Issues Pract. 1996; 15:20–31.

8. Norcini JJ, Shea JA. The credibility and comparability of standards. Appl Meas Educ. 1997; 10:39–59.

9. Zieky MJ, Livingston SA. Manual for setting standards on the basic skills assessment tests. Princeton (NJ): Educational Testing Service;1977.

10. Bahn JC. Scale and test equating of National Assessment of Educational Achievement. Paper presented at: 2004 Annual Fall Meeting of the Korean Society for the Study of Education. 2004 October 29–30; Daejeon, KR.Korean.

11. Kolen MJ, Brennan RL. Test equating: methods and practices. New York (NY): Springer-Verlag;1995.

12. Lee G, Lewis DM. A generalizability theory approach toward estimating standard errors of cut scores set using the Bookmark standard setting procedure. Paper presented at: 2001 annual meeting of the National Council on Measurement in Education. 2001 April 11–13; Seattle, WA.