Healthc Inform Res.  2020 Jan;26(1):13-19. 10.4258/hir.2020.26.1.13.

Machine Learning and Initial Nursing Assessment-Based Triage System for Emergency Department

• Yu JY
• Jeong GY
• Jeong OS
• Chang DK
• Cha WC

• KMID: 2469396
• DOI: http://doi.org/10.4258/hir.2020.26.1.13

Abstract

OBJECTIVES
The aim of this study was to develop machine learning (ML) and initial nursing assessment (INA)-based emergency department (ED) triage to predict adverse clinical outcome.
METHODS
The retrospective study included ED visits between January 2016 and December 2017 that resulted in either intensive care unit admission or emergency room death. We trained four classifiers using logistic regression and a deep learning model on INA and low dimensional (LD) INA, logistic regression on the Korea Triage and acuity scale (KTAS) and Sequential Related Organ Failure Assessment (SOFA). We varied the outcome ratio for external validation. Finally, variables of importance were identified using the random forest model's information gain. The four most influential variables were used for LD modeling for efficiency.
RESULTS
A total of 86,304 patient visits were included, with an overall outcome rate of 3.5%. The area under the curve (AUC) values for the KTAS model were 76.8 (74.9-78.6) with logistic regression and 74.0 (72.1-75.9) for the SOFA model, while the AUC values of the INA model were 87.2 (85.9-88.6) and 87.6 (86.3-88.9) with logistic regression and deep learning, suggesting that the ML and INA-based triage system result more accurately predicted the outcomes. The AUC values for the LD model were 81.2 (79.4-82.9) and 80.7 (78.9-82.5) for logistic regression and deep learning, respectively.
CONCLUSIONS
We developed an ML and INA-based triage system for EDs. The novel system was able to predict clinical outcomes more accurately than existing triage systems, KTAS and SOFA.

Keyword

Triage; Machine Learning; Deep Learning; Hospital Emergency Service; Efficiency

MeSH Terms

Area Under Curve
Emergencies*
Emergency Service, Hospital*
Forests
Humans
Intensive Care Units
Korea
Learning
Logistic Models
Machine Learning*
Nursing Assessment
Nursing*
Retrospective Studies
Triage*

Figure

• 

  Figure 1 Study inclusion criteria for the emergency room cohort. DOA: dead on arrival, CPR: cardiopulmonary resuscitation, NA: not available (missing), GCS: Glasgow Coma Scale.

• 

  Figure 2 Overview of workflow. Clinical data warehouse (CDW) with clinical outcome is split into three parts (training, validation, and testing). Three different methods (logistic regression, random forest, and deep learning) and two different types of input (KTAS and INA without KTAS) were combined. Model evaluation was judged by AUROC. KTAS: Korea Triage and Acuity Scale, SOFA: Sequential Organ Failure Assessment, INA: initial nursing assessment, AUROC: area under the receiver operating characteristic curve.

• 

  Figure 3 Ratio for nursing Korea Triage and Acuity Scale (KTAS) and machine learning (ML)-based KTAS.

Reference

1. Kang SW, Park HS. Emergency department visit volume variability. Clin Exp Emerg Med. 2015; 2(3):150–154.

2. Moore BJ, Stocks C, Owens PL. Trends in emergency department visits, 2006–2014 [Internet]. Rockville (MD): Agency for Healthcare Research and Quality;2017. cited 2020 Jan 10. Available from: https://www.hcup-us.ahrq.gov/reports/statbriefs/sb227-Emergency-Department-Visit-Trends.pdf.

3. Kasteridis P, Street A, Dolman M, Gallier L, Hudson K, Martin J, et al. Who would most benefit from improved integrated care? Implementing an analytical strategy in South Somerset. Int J Integr Care. 2015; 15:e001.

4. Combes JR, Arespacochaga E. Appropriate use of medical resources [Internet]. Chicago (IL): American Hospital Association's Physician Leadership Forum;2013. cited at 2020 Jan 10. Available from: http://www.ahaphysicianforum.org/files/pdf/appropusewhiteppr.pdf.

5. Robertson-Steel I. Evolution of triage systems. Emerg Med J. 2006; 23(2):154–155.

6. Schull MJ, Kiss A, Szalai JP. The effect of low-complexity patients on emergency department waiting times. Ann Emerg Med. 2007; 49(3):257–264.e1.

7. Silva JA, Emi AS, Leao ER, Lopes MC, Okuno MF, Batista RE. Emergency Severity Index: accuracy in risk classification. Einstein (Sao Paulo). 2017; 15(4):421–427.

8. Ajani K. Triage: a literature review of key concepts. J Pak Med Assoc. 2012; 62(5):487–489.

9. FitzGerald G, Jelinek GA, Scott D, Gerdtz MF. Emergency department triage revisited. Emerg Med J. 2010; 27(2):86–92.

10. Dugas AF, Kirsch TD, Toerper M, Korley F, Yenokyan G, France D, et al. An electronic emergency triage system to improve patient distribution by critical outcomes. J Emerg Med. 2016; 50(6):910–918.

11. Levin S, Toerper M, Hamrock E, Hinson JS, Barnes S, Gardner H, et al. Machine-learning-based electronic triage more accurately differentiates patients with respect to clinical outcomes compared with the Emergency Severity Index. Ann Emerg Med. 2018; 71(5):565–574.e2.

12. Edmonds J. Initial assessment of young people in the emergency department. Emerg Nurse. 2011; 19(8):32–36.

13. Kwon JM, Lee Y, Lee Y, Lee S, Park H, Park J. Validation of deep-learning-based triage and acuity score using a large national dataset. PLoS One. 2018; 13(10):e0205836.

14. Hwang SY, Jo IJ, Lee SU, Lee TR, Yoon H, Cha WC, Sim MS, Shin TG. Low accuracy of positive qSOFA criteria for predicting 28-day mortality in critically ill septic patients during the early period after emergency department presentation. Ann Emerg Med. 2018; 71(1):1–9.e2.

15. Baig MA, Sheikh S, Hussain E, Bakhtawar S, Subhan Khan M, Mujtaba S, et al. Comparison of qSOFA and SOFA score for predicting mortality in severe sepsis and septic shock patients in the emergency department of a low middle income country. Turk J Emerg Med. 2018; 18(4):148–151.

16. Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP. SMOTE: synthetic minority over-sampling technique. J Artif Intell Res. 2002; 16:321–357.