J Educ Eval Health Prof.  2018;15:25. 10.3352/jeehp.2018.15.25.

Benefits of focus group discussions beyond online surveys in course evaluations by medical students in the United States: a qualitative study

• Brandl K
• Rabadia SV
• Chang A
• Mandel J

• KMID: 2439281
• DOI: http://doi.org/10.3352/jeehp.2018.15.25

Abstract

In addition to online questionnaires, many medical schools use supplemental evaluation tools such as focus groups to evaluate their courses. Although some benefits of using focus groups in program evaluation have been described, it is unknown whether these inperson data collection methods provide sufficient additional information beyond online evaluations to justify them. In this study, we analyze recommendations gathered from student evaluation team (SET) focus group meetings and analyzed whether these items were captured in open-ended comments within the online evaluations. Our results indicate that online evaluations captured only 49% of the recommendations identified via SETs. Surveys to course directors identified that 74% of the recommendations exclusively identified via the SETs were implemented within their courses. Our results indicate that SET meetings provided information not easily captured in online evaluations and that these recommendations resulted in actual course changes.

Keyword

Qualitative analysis; Program evaluation; Evaluation studies

MeSH Terms

Data Collection
Focus Groups*
Humans
Program Evaluation
Schools, Medical
Students, Medical*
United States*

Figure

• 

  Fig. 1. Flowchart of study design. SET, student evaluation team.

Reference

References

1. Brandl K, Mandel J. What else is happening?: a more holistic view of programme evaluation. Med Educ. 2018; 52:352–354. https://doi.org/10.1111/medu.13524.

2. Colbert-Getz JM, Baumann S. Changing medical students’ perception of the evaluation culture: is it possible? J Educ Eval Health Prof. 2016; 13:8. https://doi.org/10.3352/jeehp.2016.13.8.

3. Koufakou A, Gosselin J, Guo D. Using data mining to extract knowledge from student evaluation comments in undergraduate courses. In : Proceedings of the 2016 International Joint Conference on Neural Networks (IJCNN); 2016 Jul 24-29; Vancouver, Canada. Piscataway (NJ). Institute of Electrical and Electronics Engineers. 2016. 3138–3142.

4. Uijtdehaage S, O’Neal C. A curious case of the phantom professor: mindless teaching evaluations by medical students. Med Educ. 2015; 49:928–932. https://doi.org/10.1111/medu.12647.

5. Brandl K, Mandel J, Winegarden B. Student evaluation team focus groups increase students’ satisfaction with the overall course evaluation process. Med Educ. 2017; 51:215–227. https://doi.org/10.1111/medu.13104.

6. Hsih KW, Iscoe MS, Lupton JR, Mains TE, Nayar SK, Orlando MS, Parzuchowski AS, Sabbagh MF, Schulz JC, Shenderov K, Simkin DJ, Vakili S, Vick JB, Xu T, Yin O, Goldberg HR. The Student Curriculum Review Team: how we catalyze curricular changes through a student-centered approach. Med Teach. 2015; 37:1008–1012. https://doi.org/10.3109/0142159X.2014.990877.

7. Alhija FN, Fresko B. Student evaluation of instruction: what can be learned from students’ written comments? Stud Educ Eval. 2009; 35:37–44. https://doi.org/10.1016/j.stueduc.2009.01.002.