Acute Crit Care.  2024 Feb;39(1):186-191. 10.4266/acc.2023.01424.

Development of a deep learning model for predicting critical events in a pediatric intensive care unit

• Lee IK
• Lee B
• Park JD

• KMID: 2555238
• DOI: http://doi.org/10.4266/acc.2023.01424

Abstract

Background
Identifying critically ill patients at risk of cardiac arrest is important because it offers the opportunity for early intervention and increased survival. The aim of this study was to develop a deep learning model to predict critical events, such as cardiopulmonary resuscitation or mortality. Methods: This retrospective observational study was conducted at a tertiary university hospital. All patients younger than 18 years who were admitted to the pediatric intensive care unit from January 2010 to May 2023 were included. The main outcome was prediction performance of the deep learning model at forecasting critical events. Long short-term memory was used as a deep learning algorithm. The five-fold cross validation method was employed for model learning and testing. Results: Among the vital sign measurements collected during the study period, 11,660 measurements were used to develop the model after preprocessing; 1,060 of these data points were measurements that corresponded to critical events. The prediction performance of the model was the area under the receiver operating characteristic curve (95% confidence interval) of 0.988 (0.9751.000), and the area under the precision-recall curve was 0.862 (0.700–1.000). Conclusions: The performance of the developed model at predicting critical events was excellent. However, follow-up research is needed for external validation.

Keyword

cardiac arrest, cardiopulmonary resuscitation, machine learning; mortality, pediatric intensive care unit

Figure

• 

  Figure 1. Participant flowchart. PICU: pediatric intensive care unit.

• 

  Figure 2. The area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC) of the long short-term memory network model. Five-fold cross validation was performed on the developed prediction model, and each fold is displayed in a different color. CI: confidence interval.

• 

  Figure 3. Line plots of the loss values over several training epochs. The epoch number was 10,000, and the learning rate was 0.001. Five-fold cross validation was performed on the developed prediction model, and each fold is displayed in a different color.

Reference

1. Buckley TA, Short TG, Rowbottom YM, Oh TE. Critical incident reporting in the intensive care unit. Anaesthesia. 1997; 52:403–9.

2. Berg RA, Nadkarni VM, Clark AE, Moler F, Meert K, Harrison RE, et al. Incidence and outcomes of cardiopulmonary resuscitation in PICUs. Crit Care Med. 2016; 44:798–808.

3. Girotra S, Nallamothu BK, Spertus JA, Li Y, Krumholz HM, Chan PS, et al. Trends in survival after in-hospital cardiac arrest. N Engl J Med. 2012; 367:1912–20.

4. Sandroni C, Nolan J, Cavallaro F, Antonelli M. In-hospital cardiac arrest: incidence, prognosis and possible measures to improve survival. Intensive Care Med. 2007; 33:237–45.

5. Thorén A, Rawshani A, Herlitz J, Engdahl J, Kahan T, Gustafsson L, et al. ECG-monitoring of in-hospital cardiac arrest and factors associated with survival. Resuscitation. 2020; 150:130–8.

6. Pollack MM, Patel KM, Ruttimann UE. PRISM III: an updated Pediatric Risk of Mortality score. Crit Care Med. 1996; 24:743–52.

7. Straney L, Clements A, Parslow RC, Pearson G, Shann F, Alexander J, et al. Paediatric index of mortality 3: an updated model for predicting mortality in pediatric intensive care. Pediatr Crit Care Med. 2013; 14:673–81.

8. Choi RY, Coyner AS, Kalpathy-Cramer J, Chiang MF, Campbell JP. Introduction to machine learning, neural networks, and deep learning. Transl Vis Sci Technol. 2020; 9:14.

9. Meiring C, Dixit A, Harris S, MacCallum NS, Brealey DA, Watkinson PJ, et al. Optimal intensive care outcome prediction over time using machine learning. PLoS One. 2018; 13:e0206862.

10. Delahanty RJ, Kaufman D, Jones SS. Development and evaluation of an automated machine learning algorithm for in-hospital mortality risk adjustment among critical care patients. Crit Care Med. 2018; 46:e481–8.

11. Lee B, Kim K, Hwang H, Kim YS, Chung EH, Yoon JS, et al. Development of a machine learning model for predicting pediatric mortality in the early stages of intensive care unit admission. Sci Rep. 2021; 11:1263.

12. Fleming S, Thompson M, Stevens R, Heneghan C, Plüddemann A, Maconochie I, et al. Normal ranges of heart rate and respiratory rate in children from birth to 18 years of age: a systematic review of observational studies. Lancet. 2011; 377:1011–8.

13. Bae W, Kim K, Lee B. Distribution of pediatric vital signs in the emergency department: a nationwide study. Children (Basel). 2020; 7:89.

14. Hwang S, Lee B. Machine learning-based prediction of critical illness in children visiting the emergency department. PLoS One. 2022; 17:e0264184.

15. Rigby RA, Stasinopoulos DM. Smooth centile curves for skew and kurtotic data modelled using the Box-Cox power exponential distribution. Stat Med. 2004; 23:3053–76.

16. Rigby RA, Stasinopoulos DM. Automatic smoothing parameter selection in GAMLSS with an application to centile estimation. Stat Methods Med Res. 2014; 23:318–32.

17. Cole TJ, Donaldson MD, Ben-Shlomo Y. SITAR: a useful instrument for growth curve analysis. Int J Epidemiol. 2010; 39:1558–66.

18. Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, et al. Pytorch: an imperative style, high-performance deep learning library. arXiv [Preprint]. 2019; [cited 2024 Jan 15]. Available from: https://doi.org/10.48550/arXiv.1912.01703.

19. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, et al. Scikit-learn: machine learning in Python. J Mach Learn Res. 2011; 12:2825–30.

20. Matam BR, Duncan H, Lowe D. Machine learning based framework to predict cardiac arrests in a paediatric intensive care unit: prediction of cardiac arrests. J Clin Monit Comput. 2019; 33:713–24.

21. Kennedy CE, Aoki N, Mariscalco M, Turley JP. Using time series analysis to predict cardiac arrest in a PICU. Pediatr Crit Care Med. 2015; 16:e332–9.

22. Aczon MD, Ledbetter DR, Laksana E, Ho LV, Wetzel RC. Continuous prediction of mortality in the PICU: a recurrent neural network model in a single-center dataset. Pediatr Crit Care Med. 2021; 22:519–29.

23. Rust LO, Gorham TJ, Bambach S, Bode RS, Maa T, Hoffman JM, et al. The deterioration risk index: developing and piloting a machine learning algorithm to reduce pediatric inpatient deterioration. Pediatr Crit Care Med. 2023; 24:322–33.

24. Dewan M, Soberano B, Sosa T, Zackoff M, Hagedorn P, Brady PW, et al. Assessment of a situation awareness quality improvement intervention to reduce cardiac arrests in the PICU. Pediatr Crit Care Med. 2022; 23:4–12.