Kosin Med J.  2022 Sep;37(3):228-235. 10.7180/kmj.22.118.

How does quiz activity affect summative assessment outcomes? An analysis of three consecutive years’ data on self-directed learning

• Oh CE
• Hwang H

• KMID: 2538802
• DOI: http://doi.org/10.7180/kmj.22.118

Abstract

Background
We investigated how quiz activities can improve summative assessment outcomes by analyzing the relationship between them.
Methods
We used 217 first-year medical students’ medical informatics data from 3 consecutive years. We analyzed summative assessment outcomes between quiz completion and incompletion groups, one-time and multiple-time quiz learning groups, and three combined comparisons between subgroups of quiz learning activity frequencies: 1 versus 2, 3, 4, and 6 (group 1), 1 and 2 versus 3, 4, and 6 (group 2), and 1, 2, and 3 versus 4 and 6 (group 3). We then analyzed correlations between the final quiz scores and summative assessment outcomes.
Results
The summative assessment means for students who completed quizzes and those who did not were 87.16±8.73 and 83.22±8.31, respectively (p=0.001). The means for the one-time and multiple-time quiz learning groups were 86.54±8.94 and 88.71±8.10, respectively (p=0.223). The means for combined subgroups were not significantly different between groups (p>0.05), although a statistically significant increasing trend was found from groups 1 to 3 (0.223>0.203>0.075 using the t-test and 0.225>0.150>0.067 using the Mann-Whitney test, respectively). Summative assessment scores were not significantly correlated with quiz scores (r=0.115, p=0.213).
Conclusions
Quizzes helped students who used self-directed learning obtain better summative assessment outcomes. Formative quizzes presumably did not provide students with direct knowledge, but showed them their weak points and motivated them to work on areas where their knowledge was insufficient.

Keyword

Correlation; Formative assessment; Quiz; Self-directed learning; Summative assessment

Figure

• 

  Fig. 1. Formative quizzes provided for students’ self-directed learning. Formative quizzes were made using the “Quiz” function on the Moodle platform. Students could select questions and were allowed unlimited attempts to solve them. The highest grade was applied for a question with multiple attempts. EMR, electronic medical record; EHR, electronic health record.

• 

  Fig. 2. Summative assessment outcomes for quiz completion and incompletion groups. The thick horizontal line in the middle of the box is the mean for each group’s summative assessment score. The mean scores were significantly different using the t-test (t=–3.377, p=0.001).

• 

  Fig. 3. Correlation between the final quiz score and the summative assessment outcome. We plotted a total of 119 students’ summative assessment and quiz scores. No significant correlation is visually observed. Pearson correlation analysis resulted in the same results (r=0.115, p=0.213).

Reference

References

1. Towle A, Cottrell D. Self directed learning. Arch Dis Child. 1996; 74:357–9.

2. Hauer KE, Iverson N, Quach A, Yuan P, Kaner S, Boscardin C. Fostering medical students’ lifelong learning skills with a dashboard, coaching and learning planning. Perspect Med Educ. 2018; 7:311–7.

3. Rocker N, Lottspeich C, Braun LT, Lenzer B, Frey J, Fischer MR, et al. Implementation of self-directed learning within clinical clerkships. GMS J Med Educ. 2021; 38:Doc43.

4. Kostons D, van Gog T, Paas F. Self-assessment and task selection in learner-controlled instruction: differences between effective and ineffective learners. Comput Educ. 2010; 54:932–40.

5. Lim YS. Students’ perception of formative assessment as an instructional tool in medical education. Med Sci Educ. 2019; 29:255–63.

6. Jain V, Agrawal V, Biswas S. Use of formative assessment as an educational tool. J Ayub Med Coll Abbottabad. 2012; 24:68–70.

7. Gavriel J. Assessment for learning: a wider (classroom-researched) perspective is important for formative assessment and self-directed learning in general practice. Educ Prim Care. 2013; 24:93–6.

8. Evans DJ, Zeun P, Stanier RA. Motivating student learning using a formative assessment journey. J Anat. 2014; 224:296–303.

9. Hwang H. A computer-assisted, real-time feedback system for medical students as a tool for web-based learning. Kosin Med J. 2016; 31:134–45.

10. Kibble J. Use of unsupervised online quizzes as formative assessment in a medical physiology course: effects of incentives on student participation and performance. Adv Physiol Educ. 2007; 31:253–60.

11. Azzi AJ, Ramnanan CJ, Smith J, Dionne E, Jalali A. To quiz or not to quiz: formative tests help detect students at risk of failing the clinical anatomy course. Anat Sci Educ. 2015; 8:413–20.

12. Kunzler E, Graham J, Mostow E. Motivating medical students by utilizing dermatology-oriented online quizzes. Dermatol Online J. 2016; 22:13030/qt0p31j0z8.

13. Larsen DP, Butler AC, Roediger HL 3rd. Repeated testing improves long-term retention relative to repeated study: a randomised controlled trial. Med Educ. 2009; 43:1174–81.

14. Karpicke JD, Roediger HL 3rd. The critical importance of retrieval for learning. Science. 2008; 319:966–8.

15. Larsen DP, Dornan T. Quizzes and conversations: exploring the role of retrieval in medical education. Med Educ. 2013; 47:1236–41.

16. Zhang N, Henderson CN. Can formative quizzes predict or improve summative exam performance? J Chiropr Educ. 2015; 29:16–21.

17. McNulty JA, Espiritu BR, Hoyt AE, Ensminger DC, Chandrasekhar AJ. Associations between formative practice quizzes and summative examination outcomes in a medical anatomy course. Anat Sci Educ. 2015; 8:37–44.

18. Vinall R, Kreys E. Use of end-of-class quizzes to promote pharmacy student self-reflection, motivate students to improve study habits, and to improve performance on summative examinations. Pharmacy (Basel). 2020; 8:167.

19. Kang G, Kim SE. How to write an original article in medicine and medical science. Kosin Med J. 2022; 37:96–101.

20. Lee SS, Lee H, Hwang H. New approach to learning medical procedures using a smartphone and the Moodle platform to facilitate assessments and written feedback. Kosin Med J. 2022; 37:75–82.

21. Carrillo-de-la-Pena MT, Bailles E, Caseras X, Martinez A, Ortet G, Perez J. Formative assessment and academic achievement in pre-graduate students of health sciences. Adv Health Sci Educ Theory Pract. 2009; 14:61–7.

22. Velan GM, Jones P, McNeil HP, Kumar RK. Integrated online formative assessments in the biomedical sciences for medical students: benefits for learning. BMC Med Educ. 2008; 8:52.

23. Wardman MJ, Yorke VC, Hallam JL. Evaluation of a multi-methods approach to the collection and dissemination of feedback on OSCE performance in dental education. Eur J Dent Educ. 2018; 22:e203–11.

24. Kim JY, Na BJ, Yun J, Kang J, Han S, Hwang W, et al. What kind of feedback do medical students want? Korean J Med Educ. 2014; 26:231–4.

25. Sterz J, Linßen S, Stefanescu MC, Schreckenbach T, Seifert LB, Ruesseler M. Implementation of written structured feedback into a surgical OSCE. BMC Med Educ. 2021; 21:192.

26. Halim J, Jelley J, Zhang N, Ornstein M, Patel B. The effect of verbal feedback, video feedback, and self-assessment on laparoscopic intracorporeal suturing skills in novices: a randomized trial. Surg Endosc. 2021; 35:3787–95.

27. Rolfe I, McPherson J. Formative assessment: how am I doing? Lancet. 1995; 345:837–9.

28. Grande RA, Berdida DJ, Cruz JP, Cometa-Manalo RJ, Balace AB, Ramirez SH. Academic motivation and self-directed learning readiness of nursing students during the COVID-19 pandemic in three countries: a cross-sectional study. Nurs Forum. 2022; 57:382–92.

29. Nagandla K, Sulaiha S, Nalliah S. Online formative assessments: exploring their educational value. J Adv Med Educ Prof. 2018; 6:51–7.

30. Palmen LN, Vorstenbosch MA, Tanck E, Kooloos JG. What is more effective: a daily or a weekly formative test? Perspect Med Educ. 2015; 4:73–8.

31. Roediger HL, Karpicke JD. Test-enhanced learning: taking memory tests improves long-term retention. Psychol Sci. 2006; 17:249–55.

32. Dobson JL. The use of formative online quizzes to enhance class preparation and scores on summative exams. Adv Physiol Educ. 2008; 32:297–302.

33. Baig M, Gazzaz ZJ, Farouq M. Blended learning: the impact of blackboard formative assessment on the final marks and students’ perception of its effectiveness. Pak J Med Sci. 2020; 36:327–32.

34. Guilding C, Pye RE, Butler S, Atkinson M, Field E. Answering questions in a co-created formative exam question bank improves summative exam performance, while students perceive benefits from answering, authoring, and peer discussion: a mixed methods analysis of PeerWise. Pharmacol Res Perspect. 2021; 9:e00833.

35. Walsh JL, Harris BH, Denny P, Smith P. Formative student-authored question bank: perceptions, question quality and association with summative performance. Postgrad Med J. 2018; 94:97–103.

36. Kibble JD, Johnson TR, Khalil MK, Nelson LD, Riggs GH, Borrero JL, et al. Insights gained from the analysis of performance and participation in online formative assessment. Teach Learn Med. 2011; 23:125–9.