Korean J Med Educ.  2013 Dec;25(4):327-336.

Experience of clinical skills assessment in the Busan-Gyeongnam Consortium

• Kam B
• Oh YR
• Lee SH
• Roh HR
• Hahm JR
• Im SJ

Abstract

PURPOSE
The purpose of this study is to judge the quality of clinical skills assessment in Busan-Gyeongnam Consortium.
METHODS
Fourth grade medical school students (n=350 in 2012 and n=419 in 2013) in the Busan-Gyeongnam Consortium were included in the study. The examination was consisted of 6 clinical performance examination (CPX) and 6 objective structured clinical examination (OSCE) stations. The students were divided into groups to take the exam in 4 sites during 3 days. The overall reliability was estimated by Cronbach alpha coefficient across the stations and the case reliability was by alpha across checklist items. Analysis of variance and between-group variation were used to evaluate the variation of examinee performance across different days and sites.
RESULTS
The mean total CPX/OSCE score was 67.0 points. The overall alpha across-stations was 0.66 in 2012 and 0.61 in 2013. The alpha across-items within a station was 0.54 to 0.86 in CPX, 0.51 to 0.92 in OSCE. There was no significant increase in scores between the different days. The mean scores over sites were different in 30 out of 48 stations but between-group variances were under 30%, except 2 cases.
CONCLUSION
The overall reliability was below 0.70 and standardization of exam sites was unclear. To improve the quality of exam, case development, item design, training of standardized patients and assessors, and standardization of sites are necessary. Above of all, we need to develop the well-organized matrix to measure the quality of the exam.

Keyword

Outcome and process assessment; Reliability; Psychometrics

MeSH Terms

Checklist
Clinical Competence*
Humans
Outcome and Process Assessment (Health Care)
Psychometrics
Schools, Medical