J Educ Eval Health Prof.  2017;14:20. 10.3352/jeehp.2017.14.20.

Item development process and analysis of 50 case-based items for implementation on the Korean Nursing Licensing Examination

• Park IS
• Suh YO
• Park HS
• Kang SY
• Kim KS
• Kim GH
• Choi YH
• Kim HJ

• KMID: 2406683
• DOI: http://doi.org/10.3352/jeehp.2017.14.20

Abstract

PURPOSE
The purpose of this study was to improve the quality of items on the Korean Nursing Licensing Examination by developing and evaluating case-based items that reflect integrated nursing knowledge.
METHODS
We conducted a cross-sectional observational study to develop new case-based items. The methods for developing test items included expert workshops, brainstorming, and verification of content validity. After a mock examination of undergraduate nursing students using the newly developed case-based items, we evaluated the appropriateness of the items through classical test theory and item response theory.
RESULTS
A total of 50 case-based items were developed for the mock examination, and content validity was evaluated. The question items integrated 34 discrete elements of integrated nursing knowledge. The mock examination was taken by 741 baccalaureate students in their fourth year of study at 13 universities. Their average score on the mock examination was 57.4, and the examination showed a reliability of 0.40. According to classical test theory, the average level of item difficulty of the items was 57.4% (80%-100% for 12 items; 60%-80% for 13 items; and less than 60% for 25 items). The mean discrimination index was 0.19, and was above 0.30 for 11 items and 0.20 to 0.29 for 15 items. According to item response theory, the item discrimination parameter (in the logistic model) was none for 10 items (0.00), very low for 20 items (0.01 to 0.34), low for 12 items (0.35 to 0.64), moderate for 6 items (0.65 to 1.34), high for 1 item (1.35 to 1.69), and very high for 1 item (above 1.70). The item difficulty was very easy for 24 items (below −2.0), easy for 8 items (−2.0 to −0.5), medium for 6 items (−0.5 to 0.5), hard for 3 items (0.5 to 2.0), and very hard for 9 items (2.0 or above). The goodness-of-fit test in terms of the 2-parameter item response model between the range of 2.0 to 0.5 revealed that 12 items had an ideal correct answer rate.
CONCLUSION
We surmised that the low reliability of the mock examination was influenced by the timing of the test for the examinees and the inappropriate difficulty of the items. Our study suggested a methodology for the development of future case-based items for the Korean Nursing Licensing Examination.

Keyword

Case studies; Koreans; Nursing; Licensing

MeSH Terms

Asian Continental Ancestry Group
Discrimination (Psychology)
Education
Humans
Licensure*
Nursing*
Observational Study
Students, Nursing

Figure

• 

  Fig. 1. Item characteristic curve of moderate goodness of fit.

Reference

References

1. Lee HY, Kim CJ, Lee SJ, Park HR, Lee IS, Kim HJ, Park YM. A study of the validity of the Korean Nurses’ Licensing Examination. J Educ Eval Health Prof. 2005; 2:31–42. https://doi.org/10.3352/jeehp.2005.2.1.31.

2. Kim KS, Kang YH, Koo HY, Kwan MJ, Kim NC, Kim OS, Park JS, Shin HS, Ahn OH, Jang SO. Systematical improvement on the subjects of Korea Nursing License Examination. Seoul: Korea Health Personnel Licensing Examination Institute;2013.

3. Park HR, Kim KH, Kang YS, Kim SY, Park EH. Comparative study on the educational contents of Korea Nursing License Examination by school system. Seoul: Korea Health Personnel Licensing Examination Institute;2011.

4. Korea Health Personnel Licensing Examination Institute. Specific area for developing test item pool of Korean Nursing Licensing Examination. Seoul: Korea Health Personnel Licensing Examination Institute;2014.

5. Kang SY, Kim KS, Kim GH, Park IS, Park HS, Suh YO, Ahn SY. Development of nursing knowledge statement based on job description for new graduate nurses in Korea. In : Proceedings of the 4th China-JapanKorea Nursing Conference; 2016 Nov 12-14; Beijing, China.

6. Park IS, Park IH, Kim SJ, Suh YO, Park HS, Park EH. Job analysis of the new nurses for national licensing examination. Seoul: Korean Nurses Association;2010.

7. Lim NY, Song JH. Delphi study on introduction of practical skills test in national examination for nursing licensure. J Korean Acad Fundam Nurs. 2007; 14:157–165.

8. Suh YO, Kang SY, Park IS, Shin SJ, Hwang SY. Status of the Korean nursing licensure examination and suggestion for improvement. In : Proceedings of the Academic Seminar for 24th Korea Health Personnel Licensing Examination Institute; 2016 Nov 2-3; Seoul, Korea. Seoul. In : Korea Health Personnel Licensing Examination Institute; 2016.

9. Park IS, Suh YO, Park HS, Ahn SY, Kang SY, Ko IS. The job analysis of Korean nurses as a strategy to improve the Korean Nursing Licensing Examination. J Educ Eval Health Prof. 2016; 13:24. https://doi.org/10.3352/jeehp.2016.13.24.

10. Park IS, Suh YO, Park HS, Ahn SY, Kang SY, Kim KS. Relevance of the test content of the Korean Nursing Licensing Examination to nursing job. J Educ Eval Health Prof. 2016; 13:23. https://doi.org/10.3352/jeehp.2016.13.23.

11. National Board of Medical Examiners. Constructing written test questions for the basic and clinical sciences. 3rd ed. Philadelphia (PA): National Board of Medical Examiners;2002.

12. Harding MM, Snyder JS, Preusser BA. Winningham’s critical thinking cases in nursing. 6th ed. St. Louis (MO): Elsevier;2016.