J Educ Eval Health Prof.  2015;12:43. 10.3352/jeehp.2015.12.43.

An assessment blueprint for the Advanced Medical Life Support two-day prehospital emergency medical services training program in the United States

• Becker LR
• Vassar M

• KMID: 2402050
• DOI: http://doi.org/10.3352/jeehp.2015.12.43

Abstract

PURPOSE
Traditional approaches to blueprint creation may focus on fine-grained detail at the expense of important foundational concepts. The purpose of this study was to develop a method for constructing an assessment blueprint to guide the creation of a new post-test for a two-day prehospital emergency medical services training program.
METHODS
In order to create the blueprint, we first determined the proportions of the total classroom and home-study minutes associated with the lower- and higher-order cognitive objectives of each chapter of the textbook and the two-day classroom activities during training courses conducted from January to April 2015. These proportions were then applied to a 50-question test structure in order to calculate the number of desired questions by chapter and content type.
RESULTS
Our blueprint called for the test to contain an almost even split of lower- and higher-order cognitive questions. One-best-answer multiple choice items and extended matching-type items were written to assess lower- and higher-order cognitive content, respectively.
CONCLUSION
We report the first known application of an assessment blueprint to a prehospital professional development education program. Our approach to blueprint creation is computationally straightforward and could be easily adopted by a group of instructors with a basic understanding of lower- and higher-order cognitive constructs. By blueprinting at the chapter level, as we have done, item-writers should be more inclined to construct questions that focus on important central themes or procedures.

Keyword

Choice behavior; Cognition; Emergency medical services; Test writing

MeSH Terms

Choice Behavior
Cognition
Education*
Emergencies*
Emergency Medical Services*
United States*

Figure

• 

  Figure 1. Twelve-lead electrocardiogram of the patient in item 1.Answer: I – Pericarditis.

• 

  Figure 2. Twelve-lead electrocardiogram of the patient in Item 2.Answer: L – Stable angina.

• 

  Figure 3. Twelve-lead electrocardiogram of the patient in Item 3.Answer: D – Congestive heart failure.

Reference

1. Advanced Medical Life Support Committee of the National Association of Emergency Medcal Technicians. Advanced Medical Life Support: An assessment-based approach. Burlington, MA: Jones & Bartlett Learning;2011. p. 545.

2. DiDonato-Barnes N, Fives H, Krause ES. Using a Table of Specifications to improve teacher-constructed traditional tests: an experimental design. Assess Educ Princ Policy Pract. 2014; 21:90–108. http://dx.doi.org/10.1080/0969594X.2013.808173.

3. Fives H, DiDonato-Barnes N. Classroom test construction: The power of a table of specifications. Pract Assess Res Eval. 2013; 18:3. Available from: http://pareonline.net/getvn.asp?v=18&n=3.

4. Bridge PD, Musial J, Frank R, Roe T, Sawilowsky S. Measurement practices: methods for developing content-valid student examinations. Med Teach. 2003; 25:414–421. http://dx.doi.org/10.1080/0142159031000100337.

5. Case S, Swanson DB. Constructing written test questions for the basic and clinical sciences. Philadelphia, (PA): National Board of Medical Examiners;2002. p. 180.

6. McLaughlin K, Lemaire J, Coderre S. Creating a reliable and valid blueprint for the internal medicine clerkship evaluation. Med Teach. 2005; 27:544–547. http://dx.doi.org/10.1080/01421590500136113.

7. Roberts C, Newble D, Jolly B, Reed M, Hampton K. Assuring the quality of high-stakes undergraduate assessments of clinical competence. Med Teach. 2006; 28:535–543. http://dx.doi.org/10.1080/01421590600711187.

8. Ahmad RG, Hamed OAE. Impact of adopting a newly developed blueprinting method and relating it to item analysis on students’ performance. Med Teach. 2014; 36 Suppl 1:S55–S61. http://dx.doi.org/10.3109/0142159x.2014.886014.

9. Notar C, Zuelke D, Wilson J, Yunker B. The table of specifications: Insuring accountability in teacher made tests. J Instr Psychol. 2004; 31:115–129. https://www.questia.com/library/journal/1G1-119611686/the-table-of-specifications-insuring-accountability.