J Korean Acad Nurs.  2020 Aug;50(4):621-630. 10.4040/jkan.20137.

External Validation of Carbapenem-Resistant Enterobacteriaceae Acquisition Risk Prediction Model in a Medium Sized Hospital

• Seo SM
• Jeong IS

• KMID: 2505822
• DOI: http://doi.org/10.4040/jkan.20137

Abstract

Purpose
This study was aimed to evaluate the external validity of a carbapenem-resistant Enterobacteriaceae (CRE) acquisition risk prediction model (the CREP-model) in a medium-sized hospital.
Methods
This retrospective cohort study included 613 patients (CRE group: 69, no-CRE group: 544) admitted to the intensive care units of a 453-beds secondary referral general hospital from March 1, 2017 to September 30, 2019 in South Korea. The performance of the CREP-model was analyzed with calibration, discrimination, and clinical usefulness.
Results
The results showed that those higher in age had lower presence of multidrug resistant organisms (MDROs), cephalosporin use ≥ 15 days, Acute Physiology and Chronic Health Evaluation II (APACHE II) score ≥ 21 points, and lower CRE acquisition rates than those of CREP-model development subjects. The calibration-in-the-large was 0.12 (95% CI: - 0.16~0.39), while the calibration slope was 0.87 (95% CI: 0.63~1.12), and the concordance statistic was .71 (95% CI: .63~.78). At the predicted risk of .10, the sensitivity, specificity, and correct classification rates were 43.5%, 84.2%, and 79.6%, respectively. The net true positive according to the CREP-model were 3 per 100 subjects. After adjusting the predictors’ cutting points, the concordance statistic increased to .84 (95% CI: .79~.89), and the sensitivity and net true positive was improved to 75.4%. and 6 per 100 subjects, respectively.
Conclusion
The CREP-model’s discrimination and clinical usefulness are low in a medium sized general hospital but are improved after adjusting for the predictors. Therefore, we suggest that institutions should only use the CREP-model after assessing the distribution of the predictors and adjusting their cutting points.

Keyword

Carbapenem-Resistant Enterobacteriaceae; Model; Statistical; Calibration; Sensitivity and Specificity