Inter-rater Reliability in a Clinical Performance Examination Using Multiple Standardized Patients for the Same Case

Ko, Jinkyung; Yoon, Tai Young; Park, Jaehyun

Abstract

PURPOSE
The "standardization" of standardized patients (SP) is one of the most crucial factors for a successful clinical performance examination (CPX). This study aimed to examine the inter-rater reliability among SPs who portrayed the same case during a CPX. METHODS: The context was a CPX conducted under the supervision of CPX Seoul-Gyeonggi Consortium in K medical school in August 2007. K medical school ran 12 stations consisting of duplicated sets of 6 cases. In total, thirty SPs participated with 5 SPs acting each of the 6 cases. The SPs evaluated the student's performances in addition to portraying the cases. ANCOVA (analysis of covariance) was used to compare scores rated by the different SPs. The dependent variables were the case scores and the 4 subcomponent (history taking, physical examination, Clinical courtesy, and Patient-physician interaction) scores for each case; the independent variable was the SPs; and the covariate was the CPX total score. RESULTS: The Headache and Cough stations showed an acceptable level of reliability. Otherwise, Weight Loss and Facial Flushing failed to show consistent scores in all 4 subcomponents. Diarrhea and Lt. hemiparesis showed partial consistency. In terms of the subcomponents, the physical exam scores were most consistent and the patient-physician interaction scores were most inconsistent. CONCLUSION: This study tested the level of "standardization" of one set of CPX cases with mixed results. The authors hope that our results will contribute to quality assurance of CPX.