J Educ Eval Health Prof.  2017;14:1. 10.3352/jeehp.2017.14.1.

Reliability of a viva assessment of clinical reasoning in an Australian pre-professional osteopathy program assessed using generalizability theory

• Vaughan B
• Orrock P
• Grace S

• KMID: 2406664
• DOI: http://doi.org/10.3352/jeehp.2017.14.1

Abstract

Clinical reasoning is situation-dependent and case-specific; therefore, assessments incorporating different patient presentations are warranted. The present study aimed to determine the reliability of a multi-station case-based viva assessment of clinical reasoning in an Australian pre-registration osteopathy program using generalizability theory. Students (from years 4 and 5) and examiners were recruited from the osteopathy program at Southern Cross University, Lismore, Australia. The study took place on a single day in the student teaching clinic. Examiners were trained before the examination. Students were allocated to 1 of 3 rounds consisting of 5 10-minute stations in an objective structured clinical examination-style. Generalizability analysis was used to explore the reliability of the examination. Fifteen students and 5 faculty members participated in the study. The examination produced a generalizability coefficient of 0.53, with 18 stations required to achieve a generalizability coefficient of 0.80. The reliability estimations were acceptable and the psychometric findings related to the marking rubric and overall scores were acceptable; however, further work is required in examiner training and ensuring consistent case difficulty to improve the reliability of the examination.

Keyword

Australia; Osteopathic medicine; Physical examination; Psychometrics; Reproducibility of results

MeSH Terms

Australia
Humans
Osteopathic Medicine
Physical Examination
Psychometrics
Reproducibility of Results

Reference

References

1. Rencic J, Durning S, Holmboe E, Gruppen L. Understanding the assessment of clinical reasoning. In : Wimmers PF, Mentkowski M, editors. Assessing competence in professional performance across disciplines and professions. New York (NY): Springer;2016. p. 209–235.

2. Thomson OP, Petty NJ, Moore AP. A qualitative grounded theory study of the conceptions of clinical practice in osteopathy: a continuum from technical rationality to professional artistry. Man Ther. 2014; 19:37–43. https://doi.org/10.1016/j.math.2013.06.005.

3. Grace S, Orrock P, Vaughan B, Blaich R, Coutts R. Understanding clinical reasoning in osteopathy: a qualitative research approach. Chiropr Man Therap. 2016; 24:6. https://doi.org/10.1186/s12998-016-0087-x.

4. Kasiri-Martino H, Bright P. Osteopathic educators’ attitudes towards osteopathic principles and their application in clinical practice: a qualitative inquiry. Man Ther. 2016; 21:233–240. https://doi.org/10.1016/j.math.2015.09.003.

5. Orrock P, Grace S, Vaughan B, Coutts R. Developing a viva exam to assess clinical reasoning in pre-registration osteopathy students. BMC Med Educ. 2014; 14:193. https://doi.org/10.1186/1472-6920-14-193.

6. Moore K, Grace S, Orrock P, Coutts R, Blaich R, Vaughan B. Benchmarking the strategies for assessing clinical reasoning in osteopathic curricula. Int J Osteopath Med. 2014; 17:187–198. https://doi.org/10.1016/j.ijosm.2014.03.001.

7. Bloch R, Norman G. Generalizability theory for the perplexed: a practical introduction and guide: AMEE Guide No. 68. 68. 2012; 34:960–992. https://doi.org/10.3109/0142159X.2012.703791.