Acute Crit Care.  2024 Nov;39(4):621-629. 10.4266/acc.2024.01200.

A deep learning model for estimating sedation levels using heart rate variability and vital signs: a retrospective cross-sectional study at a center in South Korea

• Kim YS
• Lee B
• Jang W
• Jeon Y
• Park JD

• KMID: 2563551
• DOI: http://doi.org/10.4266/acc.2024.01200

Abstract

Background
Optimal sedation assessment in critically ill children remains challenging due to the subjective nature of behavioral scales and intermittent evaluation schedules. This study aimed to develop a deep learning model based on heart rate variability (HRV) parameters and vital signs to predict effective and safe sedation levels in pediatric patients.
Methods
This retrospective cross-sectional study was conducted in a pediatric intensive care unit at a tertiary children’s hospital. We developed deep learning models incorporating HRV parameters extracted from electrocardiogram waveforms and vital signs to predict Richmond Agitation-Sedation Scale (RASS) scores. Model performance was evaluated using the area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve (AUPRC). The data were split into training, validation, and test sets (6:2:2), and the models were developed using a 1D ResNet architecture.
Results
Analysis of 4,193 feature sets from 324 patients achieved excellent discrimination ability, with AUROC values of 0.867, 0.868, 0.858, 0.851, and 0.811 for whole number RASS thresholds of −5 to −1, respectively. AUPRC values ranged from 0.928 to 0.623, showing superior performance in deeper sedation levels. The HRV metric SDANN2 showed the highest feature importance, followed by systolic blood pressure and heart rate.
Conclusions
A combination of HRV parameters and vital signs can effectively predict sedation levels in pediatric patients, offering the potential for automated and continuous sedation monitoring in pediatric intensive care settings. Future multi-center validation studies are needed to establish broader applicability.

Keyword

conscious sedation; deep learning; heart rate variability; machine learning; patient monitoring; pediatric intensive care unit; vital signs

Figure

• 

  Figure 1. Receiver operating characteristic curves for sedation-level prediction at different Richmond Agitation-Sedation Scale (RASS) thresholds. Each curve represents the performance of a binary classifier distinguishing between scores at or below a threshold versus those above that threshold (e.g., “RASS >−5” distinguishes between RASS=−5 vs. RASS >−5). The model shows consistent discrimination ability across thresholds, with area under the curve (AUC) values and 95% CIs shown in parentheses. The dashed gray line represents random prediction (AUC=0.5).

• 

  Figure 2. Precision-recall curves for sedation-level prediction at different Richmond Agitation-Sedation Scale (RASS) thresholds. Each curve represents the performance of a binary classifier distinguishing between scores at or below a threshold versus those above that threshold (e.g., “RASS >−5” distinguishes between RASS=−5 versus RASS >−5). The model demonstrates varying performance levels across different thresholds, with area under the precision-recall curve (AUPRC) values and 95% CIs shown in parentheses. Higher AUPRC values were observed for deeper sedation levels.

• 

  Figure 3. SHapley Additive exPlanations (SHAP) feature importance for binary classification model distinguishing between Richmond Agitation-Sedation Scale (RASS)=−5 vs. RASS >−5. The bar plot shows the mean absolute SHAP values for the top 20 features, indicating their relative importance in the model’s predictions. Heart rate variability (heart rate variability) parameters and vital signs are ranked by their average impact on model output magnitude. HRV_SDANN2 showed the highest importance, followed by vital signs (SBP, HR, DBP, RR) and other HRV parameters. SDANN2: standard deviation of average NN intervals over 5-minute segments; NN: normal-to-normal; SBP: systolic blood pressure; HR: heart rate; pNN20: proportion of NN intervals differing by more than 20 ms; IQRNN: interquartile range of NN intervals; DBP: diastolic blood pressure; RR: respiratory rate; HTI: heart rate turbulence index; SDNN: standard deviation of NN intervals; SDNNI1: successive difference of NN intervals first-order; SDNNI2: SDNNI second-order, SDSD: standard deviation of successive differences; TINN: triangular interpolation of NN interval histogram; CVSD: coefficient of variation of successive differences; RMSSD: root mean square of successive differences; CVNN: coefficient of variation of NN intervals; MedianNN: median of NN intervals.

• 

  Figure 4. SHapley Additive exPlanations (SHAP) value distribution for features in binary classification model distinguishing between Richmond Agitation-Sedation Scale (RASS)=−5 vs. RASS >−5. The plot shows the impact of each feature on model output, with colors indicating the feature value (blue=low, red=high). Each point represents a single prediction, and the horizontal position shows the SHAP value (negative values push predictions toward RASS=−5, positive values toward RASS >−5). Features are ordered by their mean absolute SHAP value, with HRV_SDANN2 having the strongest impact on predictions. HRV: heart rate variability; SDANN2: standard deviation of average NN intervals over 5-minute segments; NN: normal-to-normal; SBP: systolic blood pressure; HR: heart rate; pNN20: proportion of NN intervals differing by more than 20 ms; IQRNN: interquartile range of NN intervals; DBP: diastolic blood pressure; RR: respiratory rate; HTI: heart rate turbulence index; SDNN: standard deviation of NN intervals; SDNNI1: successive difference of NN intervals first-order; SDNNI2: SDNNI second-order, SDSD: standard deviation of successive differences; TINN: triangular interpolation of NN interval histogram; CVSD: coefficient of variation of successive differences; RMSSD: root mean square of successive differences; CVNN: coefficient of variation of NN intervals; MedianNN: median of NN intervals.

Reference

1. Pomeranz B, Macaulay RJ, Caudill MA, Kutz I, Adam D, Gordon D, et al. Assessment of autonomic function in humans by heart rate spectral analysis. Am J Physiol. 1985; 248(1 Pt 2):H151–3.

2. Anand KJ, Willson DF, Berger J, Harrison R, Meert KL, Zimmerman J, et al. Tolerance and withdrawal from prolonged opioid use in critically ill children. Pediatrics. 2010; 125:e1208–25.

3. Best KM, Boullata JI, Curley MA. Risk factors associated with iatrogenic opioid and benzodiazepine withdrawal in critically ill pediatric patients: a systematic review and conceptual model. Pediatr Crit Care Med. 2015; 16:175–83.

4. de Wit M, Pedram S, Best AM, Epstein SK. Observational study of patient-ventilator asynchrony and relationship to sedation level. J Crit Care. 2009; 24:74–80.

5. Devlin JW, Skrobik Y, Gélinas C, Needham DM, Slooter AJ, Pandharipande PP, et al. Clinical practice guidelines for the prevention and management of pain, agitation/sedation, delirium, immobility, and sleep disruption in adult patients in the ICU. Crit Care Med. 2018; 46:e825–73.

6. Thille AW, Rodriguez P, Cabello B, Lellouche F, Brochard L. Patient-ventilator asynchrony during assisted mechanical ventilation. Intensive Care Med. 2006; 32:1515–22.

7. Balit CR, LaRosa JM, Ong JS, Kudchadkar SR. Sedation protocols in the pediatric intensive care unit: fact or fiction? Transl Pediatr. 2021; 10:2814–24.

8. Nagaraj SB, McClain LM, Zhou DW, Biswal S, Rosenthal ES, Purdon PL, et al. Automatic classification of sedation levels in ICU patients using heart rate variability. Crit Care Med. 2016; 44:e782–9.

9. Spencer EM, Green JL, Willatts SM. Continuous monitoring of depth of sedation by EEG spectral analysis in patients requiring mechanical ventilation. Br J Anaesth. 1994; 73:649–54.

10. Shaffer F, Ginsberg JP. An overview of heart rate variability metrics and norms. Front Public Health. 2017; 5:258.

11. Bradley BD, Green G, Ramsay T, Seely AJ. Impact of sedation and organ failure on continuous heart and respiratory rate variability monitoring in critically ill patients: a pilot study. Crit Care Med. 2013; 41:433–44.

12. Bae W, Kim K, Lee B. Distribution of pediatric vital signs in the emergency department: a nationwide study. Children (Basel). 2020; 7:89.

13. Goo S, Jang W, Kim YS, Ji S, Park T, Park JD, et al. Streamlining pediatric vital sign assessment: innovations and insights. Sci Rep. 2024; 14:22542.

14. Kim L, Yun KS, Park JD, Lee B. Effect of diurnal variation of heart rate and respiratory rate on activation of rapid response system and clinical outcome in hospitalized children. Children (Basel). 2023; 10:167.

15. Lee HC, Jung CW. Vital Recorder: a free research tool for automatic recording of high-resolution time-synchronised physiological data from multiple anaesthesia devices. Sci Rep. 2018; 8:1527.

16. Tapia R, López-Herce J, Arias Á, Del Castillo J, Mencía S. Validity and reliability of the Richmond agitation-sedation scale in pediatric intensive care patients: a multicenter study. Front Pediatr. 2022; 9:795487.

17. Smith HA, Besunder JB, Betters KA, Johnson PN, Srinivasan V, Stormorken A, et al. 2022 Society of Critical Care Medicine clinical practice guidelines on prevention and management of pain, agitation, neuromuscular blockade, and delirium in critically Ill pediatric patients with consideration of the ICU environment and early mobility. Pediatr Crit Care Med. 2022; 23:e74–110.

18. Boerlage AA, Ista E, de Jong M, Tibboel D, van Dijk M. The COMFORT behavior scale: is a shorter observation period feasible? Pediatr Crit Care Med. 2012; 13:e124–5.

19. Kerson AG, DeMaria R, Mauer E, Joyce C, Gerber LM, Greenwald BM, et al. Validity of the Richmond Agitation-Sedation Scale (RASS) in critically ill children. J Intensive Care. 2016; 4:65.

20. Nagaraj SB, Ramaswamy SM, Biswal S, Boyle EJ, Zhou DW, Mcclain LM, et al. Heart rate variability as a biomarker for sedation depth estimation in ICU patients. Annu Int Conf IEEE Eng Med Biol Soc. 2016; 2016:6397–400.

21. Nagaraj SB, Biswal S, Boyle EJ, Zhou DW, McClain LM, Bajwa EK, et al. Patient-specific classification of ICU sedation levels from heart rate variability. Crit Care Med. 2017; 45:e683–90.

22. Toyofuku A, Ehrler M, Naef N, Schmid AS, Kretschmar O, Latal B, et al. Heart rate variability and cognitive functions in adolescents with complex congenital heart disease. Pediatr Res. 2024; Jul. 30. [Epub]. https://doi.org/10.1038/s41390-024-03432-9.

23. Goldenberg I, Goldkorn R, Shlomo N, Einhorn M, Levitan J, Kuperstein R, et al. Heart rate variability for risk assessment of myocardial ischemia in patients without known coronary artery disease: the HRV-DETECT (Heart Rate Variability for the Detection of Myocardial Ischemia) study. J Am Heart Assoc. 2019; 8:e014540.

24. Kong SD, Gordon CJ, Hoyos CM, Wassing R, D'Rozario A, Mowszowski L, et al. Heart rate variability during slow wave sleep is linked to functional connectivity in the central autonomic network. Brain Commun. 2023; 5:fcad129.

25. Ernst G. Hidden signals-the history and methods of heart rate variability. Front Public Health. 2017; 5:265.