J Educ Eval Health Prof.  2015;12:55. 10.3352/jeehp.2015.12.55.

Smartphone-based evaluations of clinical placements-a useful complement to web-based evaluation tools

• Hessius J
• Johansson

• KMID: 2402062
• DOI: http://doi.org/10.3352/jeehp.2015.12.55

Abstract

PURPOSE
Web-based questionnaires are currently the standard method for course evaluations. The high rate of smartphone adoption in Sweden makes possible a range of new uses, including course evaluation. This study examines the potential advantages and disadvantages of using a smartphone app as a complement to web-based course evaluationsystems.
METHODS
An iPhone app for course evaluations was developed and interfaced to an existing web-based tool. Evaluations submitted using the app were compared with those submitted using the web between August 2012 and June 2013, at the Faculty of Medicine at Uppsala University, Sweden.
RESULTS
At the time of the study, 49% of the students were judged to own iPhones. Over the course of the study, 3,340 evaluations were submitted, of which 22.8% were submitted using the app. The median of mean scores in the submitted evaluations was 4.50 for the app (with an interquartile range of 3.70-5.20) and 4.60 (3.70-5.20) for the web (P=0.24). The proportion of evaluations that included a free-text comment was 50.5% for the app and 49.9% for the web (P=0.80).
CONCLUSION
An app introduced as a complement to a web-based course evaluation system met with rapid adoption. We found no difference in the frequency of free-text comments or in the evaluation scores. Apps appear to be promising tools for course evaluations. web-based course evaluation system met with rapid adoption. We found no difference in the frequency of free-text comments or in the evaluation scores. Apps appear to be promising tools for course evaluations.

Keyword

Clinical education; Course evaluation; Education evaluation; Smartphone based tools

MeSH Terms

Complement System Proteins*
Humans
Sweden
Smartphone
Complement System Proteins

Figure

• 

  Fig. 1. The homepage of the web based tool (A) and the app’s opening screen (B).

• 

  Fig. 2. (A) Radio buttons in the web-based tool and (B)radio buttons in the app, adapted for greater ease of use on a mobile device. Translation of end points: ‘dålig/obefintlig’-‘bad/non-existent’ and ‘mycket bra’-‘very good’.

• 

  Fig. 3. Distribution of responses on the Likert scale, averaged over all questions answered per evaluation.

Reference

1. Aleamoni LM. Student rating myths versus research facts from 1924 to 1998. J Pers Eval Educ. 1999; 13:153–166.

2. Nulty DD. The adequacy of response rates to online and paper surveys: what can be done? Assess Eval High Educ. 2008; 33:301–314. http://dx.doi.org/10.1080/02602930701293231.

3. Carini RM, Hayek JC, Kuh GD, Kennedy JM, Ouimet JA. College student responses to web and paper surveys: does mode matter? Res High Educ. 2003; 44:19.

4. Avery RJ, Bryant WK, Mathios A, Kang H, Bell D. Electronic course evaluations: does an online delivery system influence student evaluations? J Econ Educ. 2006; 37:21–37. http://dx.doi.org/10.3200/jece.37.1.21-37.

5. Leung DY, Kember D. Comparability of data gathered from evaluation questionnaires on paper and through the internet. Res High Educ. 2005; 46:571–591. http://dx.doi.org/10.1007/s11162-005-3365-3.

6. Anderson HM, Cain J, Bird E. Online student course evaluations: review of literature and a pilot study. Am J Pharm Educ. 2005; 69:34–43. http://dx.doi.org/10.5688/aj690105.

7. Donovan J, Mader CE, Shinsky J. Constructive student feedback: online vs traditional course evaluations. J Interact Online Learn. 2010; 9:283–296.

8. Struthers J, Irvine P, Jackson C. Can text messaging be used as an effective method for collecting quality and detailed evaluation data from students on clinical placements? . Med Teach. 2013; 35:678–683. http://dx.doi.org/10.3109/0142159X.2013.801549.

9. Coulby C, Hennessey S, Davies N, Fuller R. The use of mobile technology for work‐based assessment: the student experience. Br J Educ Technol. 2011; 42:251–265. http://dx.doi.org/10.1111/j.1467-8535.2009.01022.x.

10. Farago P. iOS and android adoption explodes internationally [Internet]. Flurry Insights Blog. 2012. [cited 2015 Aug 27]. Available from: http://blog.flurry.com//bid/88867/ios-and-android-adoption-explodes-internationally.

11. Payne KB, Wharrad H, Watts K. Smartphone and medical related App use among medical students and junior doctors in the United Kingdom (UK): a regional survey. BMC Med Inform Decis Mak. 2012; 12:121. http://dx.doi.org/10.1186/1472-6947-12-121.

12. Kihlberg P, Perzon M, Gedeborg R, Blomqvist P, Johansson J. Uniform evaluation of clinical teaching: an instrument for specific feedback and cross comparison between departments. Hogre Utbildning. 2011; 1:139–150.

13. Lewisson N, Hellgren L, Johansson J. Quality improvement in clinical teaching through student evaluations of rotations and feedback to departments. Med Teach. 2013; 35:820–825. http://dx.doi.org/10.3109/0142159X.2013.802302.

14. Berk RA. The secret to the “best” ratings from any evaluation scale. J Fac Dev. 2010; 24:37–39.

15. Champagne MV. Student use of mobile devices in course evaluation: a longitudinal study. Educ Res Eval. 2013; 19:636–646. http://dx.doi.org/10.1080/13803611.2013.834618.

16. Johnson TD. Online student ratings: will students respond? New Dir Teach Learn. 2003; 2003:49–59. http://dx.doi.org/10.1002/tl.122.

17. McNulty JA, Gruener G, Chandrasekhar A, Espiritu B, Hoyt A, Ensminger D. Are online student evaluations of faculty influenced by the timing of evaluations? Adv Physiol Educ. 2010; 34:213–216. http://dx.doi.org/10.1152/advan.00079.2010.