Yonsei Med J.  2013 Mar;54(2):425-431. 10.3349/ymj.2013.54.2.425.

External Validation of the Acute Physiology and Chronic Health Evaluation II in Korean Intensive Care Units

• Kim JY
• Lim SY
• Jeon K
• Koh Y
• Lim CM
• Koh SO
• Na S
• Lee KM
• Lee BH
• Kwon JY
• Lee KH
• Yoon SH
• Park J
• Suh GY

• KMID: 1503906
• DOI: http://doi.org/10.3349/ymj.2013.54.2.425

Abstract

PURPOSE
This study was designed to validate the usefulness of the Acute Physiology and Chronic Health Evaluation (APACHE) II for predicting hospital mortality of critically ill Korean patients.
MATERIALS AND METHODS
We analyzed data on 826 patients who had been admitted to nine intensive care units and were included in the Fever and Antipyretics in Critical Illness Evaluation study cohort.
RESULTS
Among the patients enrolled, 62% (512/826) were medical and 38% (314/826) were surgical patients. The median APACHE II score was 17 (11 to 23 interquartile range), and the hospital mortality rate was 19.5%. Age, underlying diseases, medical patients, mechanical ventilation, and renal replacement therapy were independently associated with hospital mortality. The calibration of APACHE II was poor (H=57.54, p<0.0001; C=55.99, p<0.0001), and the discrimination was modest [area under the receiver operating characteristic (aROC)=0.729]. Calibration was poor for both medical and surgical patients (H=63.56, p<0.0001; C=73.83, p<0.0001, and H=33.92, p<0.0001; C=33.34, p=0.0001, respectively), while discrimination was poor for medical patients (aROC=0.651) and modest for surgical patients (aROC=0.704). At the predicted risk of 50%, APACHE II had a sensitivity of 36.6% and a specificity of 87.4% for hospital mortality.
CONCLUSION
For Koreans, the APACHE II exhibits poor calibration and modest discrimination for hospital mortality. Therefore, a new model is needed to accurately predict mortality in critically ill Korean patients.

Keyword

APACHE II; calibration; discrimination; intensive care units; illness severity

MeSH Terms

*APACHE
Aged
Cohort Studies
Critical Illness/mortality
Hospital Mortality
Humans
*Intensive Care Units
Middle Aged
Risk Factors

Reference

1. Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med. 1985. 13:818–829.

2. Knaus WA, Wagner DP, Draper EA, Zimmerman JE, Bergner M, Bastos PG, et al. The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest. 1991. 100:1619–1636.

3. Le Gall JR, Lemeshow S, Saulnier F. A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. JAMA. 1993. 270:2957–2963.

4. Metnitz PG, Moreno RP, Almeida E, Jordan B, Bauer P, Campos RA, et al. SAPS 3--from evaluation of the patient to evaluation of the intensive care unit. Part 1: objectives, methods and cohort description. Intensive Care Med. 2005. 31:1336–1344.

5. Moreno RP, Metnitz PG, Almeida E, Jordan B, Bauer P, Campos RA, et al. SAPS 3--from evaluation of the patient to evaluation of the intensive care unit. Part 2: development of a prognostic model for hospital mortality at ICU admission. Intensive Care Med. 2005. 31:1345–1355.

6. Afessa B, Gajic O, Keegan MT. Severity of illness and organ failure assessment in adult intensive care units. Crit Care Clin. 2007. 23:639–658.

7. Lee BH, Inui D, Suh GY, Kim JY, Kwon JY, Park J, et al. Association of body temperature and antipyretic treatments with mortality of critically ill patients with and without sepsis: multi-centered prospective observational study. Crit Care. 2012. 16:R33.

8. Hosmer DW, Lemeshow S. Confidence interval estimates of an index of quality performance based on logistic regression models. Stat Med. 1995. 14:2161–2172.

9. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982. 143:29–36.

10. Steyerberg EW. Clinical prediction models: a practical approach to development, validation, and updating (statistics for biology and health). 2010. New York: Springer.

11. Glance LG, Osler T, Shinozaki T. Effect of varying the case mix on the standardized mortality ratio and W statistic: a simulation study. Chest. 2000. 117:1112–1117.

12. Harrison DA, Brady AR, Parry GJ, Carpenter JR, Rowan K. Recalibration of risk prediction models in a large multicenter cohort of admissions to adult, general critical care units in the United Kingdom. Crit Care Med. 2006. 34:1378–1388.

13. Timsit JF, Fosse JP, Troché G, De Lassence A, Alberti C, Garrouste-Orgeas M, et al. Accuracy of a composite score using daily SAPS II and LOD scores for predicting hospital mortality in ICU patients hospitalized for more than 72 h. Intensive Care Med. 2001. 27:1012–1021.

14. Strand K, Flaatten H. Severity scoring in the ICU: a review. Acta Anaesthesiol Scand. 2008. 52:467–478.

15. Capuzzo M, Valpondi V, Sgarbi A, Bortolazzi S, Pavoni V, Gilli G, et al. Validation of severity scoring systems SAPS II and APACHE II in a single-center population. Intensive Care Med. 2000. 26:1779–1785.

16. Fedullo AJ, Swinburne AJ, Wahl GW, Bixby KR. APACHE II score and mortality in respiratory failure due to cardiogenic pulmonary edema. Crit Care Med. 1988. 16:1218–1221.

17. Cerra FB, Negro F, Abrams J. APACHE II score does not predict multiple organ failure or mortality in postoperative surgical patients. Arch Surg. 1990. 125:519–522.

18. Vincent JL, Bruzzi de Carvalho F. Severity of illness. Semin Respir Crit Care Med. 2010. 31:31–38.

19. Lim SY, Ham CR, Park SY, Kim S, Park MR, Jeon K, et al. Validation of the Simplified Acute Physiology Score 3 scoring system in a Korean intensive care unit. Yonsei Med J. 2011. 52:59–64.

20. Greiner M, Pfeiffer D, Smith RD. Principles and practical application of the receiver-operating characteristic analysis for diagnostic tests. Prev Vet Med. 2000. 45:23–41.