J Educ Eval Health Prof.  2017;14:32. 10.3352/jeehp.2017.14.32.

Usefulness of the DETECT program for assessing the internal structure of dimensionality in simulated data and results of the Korean nursing licensing examination

• Seo DG
• Choi Y
• Huh S

• KMID: 2406695
• DOI: http://doi.org/10.3352/jeehp.2017.14.32

Abstract

PURPOSE
The dimensionality of examinations provides empirical evidence of the internal test structure underlying the responses to a set of items. In turn, the internal structure is an important piece of evidence of the validity of an examination. Thus, the aim of this study was to investigate the performance of the DETECT program and to use it to examine the internal structure of the Korean nursing licensing examination.
METHODS
Non-parametric methods of dimensional testing, such as the DETECT program, have been proposed as ways of overcoming the limitations of traditional parametric methods. A non-parametric method (the DETECT program) was investigated using simulation data under several conditions and applied to the Korean nursing licensing examination.
RESULTS
The DETECT program performed well in terms of determining the number of underlying dimensions under several different conditions in the simulated data. Further, the DETECT program correctly revealed the internal structure of the Korean nursing licensing examination, meaning that it detected the proper number of dimensions and appropriately clustered the items within each dimension.
CONCLUSION
The DETECT program performed well in detecting the number of dimensions and in assigning items for each dimension. This result implies that the DETECT method can be useful for examining the internal structure of assessments, such as licensing examinations, that possess relatively many domains and content areas.

Keyword

Dimensionality; Korea; Licensure; Nursing; Simulation

MeSH Terms

Korea
Licensure*
Methods
Nursing*

Reference

References

1. McDonald RP. Nonlinear factor analysis. Richmond (VA): William Byrd Press;1967.

2. Stout W, Habing B, Douglas J, Kim HR, Roussos L, Zhang J. Conditional covariance-based nonparametric multidimensionality assessment. Appl Psychol Meas. 1996; 20:331–354.

3. Mroch AA, Bolt DM. A simulation comparison of parametric and nonparametric dimensionality detection procedures. Appl Meas Educ. 2006; 19:67–91.

4. Reckase MD. The difficulty of test items that measure more than one ability. Appl Psychol Meas. 1985; 9:401–412.

5. R Development Core Team. R: a language and environment for statistical computing [Internet]. Vienna: R Foundation for Statistical Computing;2008. [cited 2017 Dec 20]. Available from: http://www.Rproject.org.

6. Robitzsch A. Sirt: supplementary item response theory models [Internet]. R package version 2.4-20. Vienna: R Foundation for Statistical Computing;2018. [cited 2017 Dec 20]. Available from: https://CRAN.R-project.org/package=sirt.

7. DeMars CE. “Guessing” parameter estimates for multidimensional item response theory models. Educ Psychol Meas. 2007; 67:433–446.

8. Seo DG, Kim JK, Kim K. Characteristics of item parameter estimation for the multidimensional item response theory (MIRT). Korean J Psychol Gen. 2015; 34:619–640.

9. Kim HR. New techniques for the dimensionality assessment of standardized test data. [dissertation]. Urbana (IL): University of Illinois at Urbana-Champaign;1994.

10. Zhang J, Stout W. The theoretical DETECT index of dimensionality and its application to approximate simple structure. Psychometrika. 1999; 64:213–249.