J Korean Med Sci.  2017 Feb;32(2):365-370. 10.3346/jkms.2017.32.2.365.

Validation of the Pediatric Index of Mortality 3 in a Single Pediatric Intensive Care Unit in Korea

• Lee OJ
• Jung M
• Kim M
• Yang HK
• Cho J

• KMID: 2364184
• DOI: http://doi.org/10.3346/jkms.2017.32.2.365

Abstract

To compare mortality rate, the adjustment of case-mix variables is needed. The Pediatric Index of Mortality (PIM) 3 score is a widely used case-mix adjustment system of a pediatric intensive care unit (ICU), but there has been no validation study of it in Korea. We aim to validate the PIM3 in a Korean pediatric ICU, and extend the validation of the score from those aged 0-16 to 0-18 years, as patients aged 16-18 years are admitted to pediatric ICU in Korea. A retrospective cohort study of 1,710 patients was conducted in a tertiary pediatric ICU. To validate the score, the discriminatory power was assessed by calculating the area under the receiver-operating characteristic (ROC) curve, and calibration was evaluated by the Hosmer-Lemeshow goodness-of-fit (GOF) test. The observed mortality rate was 8.47%, and the predicted mortality rate was 6.57%. For patients aged < 18 years, the discrimination was acceptable (c-index = 0.76) and the calibration was good, with a χ² of 9.4 in the GOF test (P = 0.313). The observed mortality rate in the hemato-oncological subgroup was high (18.73%), as compared to the predicted mortality rate (7.13%), and the discrimination was unacceptable (c-index = 0.66). In conclusion, the PIM3 performed well in a Korean pediatric ICU. However, the application of the PIM3 to a hemato-oncological subgroup needs to be cautioned. Further studies on the performance of PIM3 in pediatric patients in adult ICUs and pediatric ICUs of primary and secondary hospitals are needed.

Keyword

Mortality; Critical Care; Risk Adjustment; Child; Validation Study; Benchmarking

MeSH Terms

Adult
Benchmarking
Calibration
Child
Cohort Studies
Critical Care*
Discrimination (Psychology)
Humans
Intensive Care Units*
Korea*
Mortality*
Retrospective Studies
Risk Adjustment

Figure

• 

  Fig. 1 Calibration curves of the pediatric index of mortality 3 (PIM3). (A) PIM3, age group < 18 years. (B) PIM3, age group < 16 years. “Expected” hospital mortality (%) calculated by the PIM3 model and “observed” hospital mortality (%), together with the corresponding patient number, are presented. The patients were divided into 10 similar sized groups based on predicted mortality.

Reference

1. Garland A. Improving the ICU: part 1. Chest. 2005; 127:2151–2164.

2. Pollack MM, Alexander SR, Clarke N, Ruttimann UE, Tesselaar HM, Bachulis AC. Improved outcomes from tertiary center pediatric intensive care: a statewide comparison of tertiary and nontertiary care facilities. Crit Care Med. 1991; 19:150–159.

3. Marcin JP, Pollack MM. Review of the methodologies and applications of scoring systems in neonatal and pediatric intensive care. Pediatr Crit Care Med. 2000; 1:20–27.

4. Straney L, Clements A, Parslow RC, Pearson G, Shann F, Alexander J, Slater A; ANZICS Paediatric Study Group and the Paediatric Intensive Care Audit Network. Paediatric index of mortality 3: an updated model for predicting mortality in pediatric intensive care*. Pediatr Crit Care Med. 2013; 14:673–681.

5. Pollack MM, Patel KM, Ruttimann UE. PRISM III: an updated pediatric risk of mortality score. Crit Care Med. 1996; 24:743–752.

6. Rosenberg AL. Recent innovations in intensive care unit risk-prediction models. Curr Opin Crit Care. 2002; 8:321–330.

7. Lim SY, Koh SO, Jeon K, Na S, Lim CM, Choi WI, Lee YJ, Kim SC, Chon GR, Kim JH, et al. Validation of SAPS3 admission score and its customization for use in Korean intensive care unit patients: a prospective multicentre study. Respirology. 2013; 18:989–995.

8. Kim JY, Lim SY, Jeon K, Koh Y, Lim CM, Koh SO, Na S, Lee KM, Lee BH, Kwon JY, et al. External validation of the acute physiology and chronic health evaluation II in Korean intensive care units. Yonsei Med J. 2013; 54:425–431.

9. Park SK, Chun HJ, Kim DW, Im TH, Hong HJ, Yi HJ. Acute physiology and chronic health evaluation II and simplified acute physiology score II in predicting hospital mortality of neurosurgical intensive care unit patients. J Korean Med Sci. 2009; 24:420–426.

10. Shann F, Pearson G, Slater A, Wilkinson K. Paediatric index of mortality (PIM): a mortality prediction model for children in intensive care. Intensive Care Med. 1997; 23:201–207.

11. Slater A, Shann F, Pearson G; Paediatric Index of Mortality (PIM) Study Group. PIM2: a revised version of the paediatric index of mortality. Intensive Care Med. 2003; 29:278–285.

12. Lemeshow S, Hosmer DW Jr. A review of goodness of fit statistics for use in the development of logistic regression models. Am J Epidemiol. 1982; 115:92–106.

13. Murphy-Filkins R, Teres D, Lemeshow S, Hosmer DW. Effect of changing patient mix on the performance of an intensive care unit severity-of-illness model: how to distinguish a general from a specialty intensive care unit. Crit Care Med. 1996; 24:1968–1973.

14. Ng DK, Miu TY, Chiu WK, Hui NT, Chan CH. Validation of pediatric index of mortality 2 in three pediatric intensive care units in Hong Kong. Indian J Pediatr. 2011; 78:1491–1494.

15. Eulmesekian PG, Pérez A, Minces PG, Ferrero H. Validation of pediatric index of mortality 2 (PIM2) in a single pediatric intensive care unit of Argentina. Pediatr Crit Care Med. 2007; 8:54–57.

16. Ciofi degli Atti ML. Cuttini M, Ravà L, Rinaldi S, Brusco C, Cogo P, Pirozzi N, Picardo S, Schiavi F, Raponi M. Performance of the pediatric index of mortality 2 (PIM-2) in cardiac and mixed intensive care units in a tertiary children’s referral hospital in Italy. BMC Pediatr. 2013; 13:100.

17. Salamati P, Talaee S, Eghbalkhah A, Chaman R, Mokhtari Z, Azarshahin M. Validation of pediatric index of mortality-2 scoring system in a single pediatric intensive care unit in iran. Iran J Pediatr. 2012; 22:481–486.

18. Hariharan S, Krishnamurthy K, Grannum D. Validation of pediatric index of mortality-2 scoring system in a pediatric intensive care unit, Barbados. J Trop Pediatr. 2011; 57:9–13.

19. Imamura T, Nakagawa S, Goldman RD, Fujiwara T. Validation of pediatric index of mortality 2 (PIM2) in a single pediatric intensive care unit in Japan. Intensive Care Med. 2012; 38:649–654.

20. Taori RN, Lahiri KR, Tullu MS. Performance of PRISM (pediatric risk of mortality) score and PIM (pediatric index of mortality) score in a tertiary care pediatric ICU. Indian J Pediatr. 2010; 77:267–271.

21. Wolfler A, Osello R, Gualino J, Calderini E, Vigna G, Santuz P, Amigoni A, Savron F, Caramelli F, Rossetti E, et al. The importance of mortality risk assessment: validation of the pediatric index of mortality 3 score. Pediatr Crit Care Med. 2016; 17:251–256.

22. van Veen A, Karstens A, van der Hoek AC, Tibboel D, Hählen K, van der Voort E. The prognosis of oncologic patients in the pediatric intensive care unit. Intensive Care Med. 1996; 22:237–241.

23. Hayes C, Lush RJ, Cornish JM, Foot AM, Henderson J, Jenkins I, Murphy P, Oakhill A, Pamphilon DH, Steward CG, et al. The outcome of children requiring admission to an intensive care unit following bone marrow transplantation. Br J Haematol. 1998; 102:666–670.

24. Jacobe SJ, Hassan A, Veys P, Mok Q. Outcome of children requiring admission to an intensive care unit after bone marrow transplantation. Crit Care Med. 2003; 31:1299–1305.

25. Tomaske M, Bosk A, Eyrich M, Bader P, Niethammer D. Risks of mortality in children admitted to the paediatric intensive care unit after haematopoietic stem cell transplantation. Br J Haematol. 2003; 121:886–891.

26. Jackson SR, Tweeddale MG, Barnett MJ, Spinelli JJ, Sutherland HJ, Reece DE, Klingemann HG, Nantel SH, Fung HC, Toze CL, et al. Admission of bone marrow transplant recipients to the intensive care unit: outcome, survival and prognostic factors. Bone Marrow Transplant. 1998; 21:697–704.

27. Schneider DT, Cho J, Laws HJ, Dilloo D, Göbel U, Nürnberger W. Serial evaluation of the oncological pediatric risk of mortality (O-PRISM) score following allogeneic bone marrow transplantation in children. Bone Marrow Transplant. 2002; 29:383–389.

28. Flaatten H. The present use of quality indicators in the intensive care unit. Acta Anaesthesiol Scand. 2012; 56:1078–1083.

29. Kramer AA, Higgins TL, Zimmerman JE. Comparing observed and predicted mortality among ICUs using different prognostic systems: why do performance assessments differ? Crit Care Med. 2015; 43:261–269.

30. Afessa B, Gajic O, Morales IJ, Keegan MT, Peters SG, Hubmayr RD. Association between ICU admission during morning rounds and mortality. Chest. 2009; 136:1489–1495.

31. Kramer AA, Higgins TL, Zimmerman JE. The association between ICU readmission rate and patient outcomes. Crit Care Med. 2013; 41:24–33.

32. Slater A, Shann F; ANZICS Paediatric Study Group. The suitability of the pediatric index of mortality (PIM), PIM2, the pediatric risk of mortality (PRISM), and PRISM III for monitoring the quality of pediatric intensive care in Australia and New Zealand. Pediatr Crit Care Med. 2004; 5:447–454.

33. Arias Lopez MP, Fernández AL, Ratto ME, Saligari L, Serrate AS, Ko IJ, Troster E, Schnitzler E; ValidarPIM2 Latin American Group. Pediatric index of mortality 2 as a predictor of death risk in children admitted to pediatric intensive care units in Latin America: a prospective, multicenter study. J Crit Care. 2015; 30:1324–1330.

34. Pearson GA, Stickley J, Shann F. Calibration of the paediatric index of mortality in UK paediatric intensive care units. Arch Dis Child. 2001; 84:125–128.