J Educ Eval Health Prof.  2018;15:33. 10.3352/jeehp.2018.15.33.

The relationship of examinees' individual characteristics and perceived acceptability of smart device-based testing to test scores on the practice test of the Korea Emergency Medicine Technician Licensing Examination

• Lim EY
• Yim MK
• Huh S

• KMID: 2439287
• DOI: http://doi.org/10.3352/jeehp.2018.15.33

Abstract

PURPOSE
Smart device-based testing (SBT) is being introduced into the Republic of Korea's high-stakes examination system, starting with the Korean Emergency Medicine Technician Licensing Examination (KEMTLE) in December 2017. In order to minimize the effects of variation in examinees' environment on test scores, this study aimed to identify any associations of variables related to examinees' individual characteristics and their perceived acceptability of SBT with their SBT practice test scores.
METHODS
Of the 569 candidate students who took the KEMTLE on September 12, 2015, 560 responded to a survey questionnaire on the acceptability of SBT after the examination. The questionnaire addressed 8 individual characteristics and contained 2 satisfaction, 9 convenience, and 9 preference items. A comparative analysis according to individual variables was performed. Furthermore, a generalized linear model (GLM) analysis was conducted to identify the effects of individual characteristics and perceived acceptability of SBT on test scores.
RESULTS
Among those who preferred SBT over paper-and-pencil testing, test scores were higher for male participants (mean± standard deviation [SD], 4.36± 0.72) than for female participants (mean± SD, 4.21± 0.73). According to the GLM, no variables evaluated"” including gender and experience with computer-based testing, SBT, or using a tablet PC"”showed a statistically significant relationship with the total score, scores on multimedia items, or scores on text items.
CONCLUSION
Individual characteristics and perceived acceptability of SBT did not affect the SBT practice test scores of emergency medicine technician students in Korea. It should be possible to adopt SBT for the KEMTLE without interference from the variables examined in this study.

Keyword

Emergency medical technicians; Licensure; Linear models; Personal satisfaction; Students; Republic of Korea

MeSH Terms

Emergencies*
Emergency Medical Technicians*
Emergency Medicine*
Female
Humans
Korea*
Licensure
Linear Models
Male
Multimedia
Personal Satisfaction
Republic of Korea

Figure

• 

  Fig. 1. Diagram of the study process. KEMTLE, Korea Emergency Medicine Technician Licensing Examination.

Reference

References

1. Kim CH. Presidential address: launching the Korea Health Personnel Licensing Examination Institute, a government-supported special foundation from December 23, 2015. J Educ Eval Health Prof. 2016; 13:20. https://doi.org/10.3352/jeehp.2016.13.20.

2. Kim CH. Presidential address: introduction of smart device-based testing and item exposure policy for Korea Health Personnel Licensing Examination. J Educ Eval Health Prof. 2017; 14:5. https://doi.org/10.3352/jeehp.2017.14.5.

3. Lim EY, Yim MK, Huh S. Smart device-based testing for medical students in Korea: satisfaction, convenience, and advantages. J Educ Eval Health Prof. 2017; 14:7. https://doi.org/10.3352/jeehp.2017.14.7.

4. Kim KJ, Hwang JY. Ubiquitous testing using tablets: its impact on medical student perceptions of and engagement in learning. Korean J Med Educ. 2016; 28:57–66. https://doi.org/10.3946/kjme.2016.10.

5. Lee G, Weerakoon P. The role of computer-aided assessment in health professional education: a comparison of student performance in computer-based and paper-and-pen multiple-choice tests. Med Teach. 2001; 23:152–157. https://doi.org/10.1080/01421590020031066.

6. Clariana R, Wallace P. Paper–based versus computer–based assessment: key factors associated with the test mode effect. Br J Educ Technol. 2002; 33:593–602. https://doi.org/10.1111/1467-8535.00294.