Diabetes Metab J.  2020 Jun;44(3):458-469. 10.4093/dmj.2019.0061.

Validation of Risk Prediction Models for Atherosclerotic Cardiovascular Disease in a Prospective Korean Community-Based Cohort

• Bae JH
• Moon MK
• Oh S
• Koo BK
• Cho NH
• Lee MK

• KMID: 2513003
• DOI: http://doi.org/10.4093/dmj.2019.0061

Abstract

To investigate the performance of the 2013 American College of Cardiology/American Heart Association Pooled Cohort Equations (PCE) in a large, prospective, community-based cohort in Korea and to compare it with that of the Framingham Global Cardiovascular Disease Risk Score (FRS-CVD) and the Korean Risk Prediction Model (KRPM).

In the Korean Genome and Epidemiology Study (KOGES)-Ansan and Ansung study, we evaluated calibration and discrimination of the PCE for non-Hispanic whites (PCE-WH) and for African Americans (PCE-AA) and compared their predictive abilities with the FRS-CVD and the KRPM.

The present study included 7,932 individuals (3,778 men and 4,154 women). The PCE-WH and PCE-AA moderately overestimated the risk of atherosclerotic cardiovascular disease (ASCVD) for men (6% and 13%, respectively) but underestimated the risk for women (−49% and −25%, respectively). The FRS-CVD overestimated ASCVD risk for men (91%) but provided a good risk prediction for women (3%). The KRPM underestimated ASCVD risk for men (−31%) and women (−31%). All the risk prediction models showed good discrimination in both men (C-statistic 0.730 to 0.735) and women (C-statistic 0.726 to 0.732). Recalibration of the PCE using data from the KOGES-Ansan and Ansung study substantially improved the predictive accuracy in men.

In the KOGES-Ansan and Ansung study, the PCE overestimated ASCVD risk for men and underestimated the risk for women. The PCE-WH and the FRS-CVD provided an accurate prediction of ASCVD in men and women, respectively.

Keyword

Atherosclerosis; Calibration; Cardiovascular diseases; Cohort studies; Epidemiology; Primary prevention; Risk assessment; Risk factors

Figure

• 

  Fig. 1 Ten-year probabilities of atherosclerotic cardiovascular disease (ASCVD) in men from the Korean Genome and Epidemiology Study (KOGES)-Ansan and Ansung study. (A) The Pooled Cohort Equations for non-Hispanic whites. (B) The Pooled Cohort Equations for African Americans. (C) The Framingham Global Cardiovascular Risk Score. (D) The Korean Risk Prediction Model.

• 

  Fig. 2 Ten-year probabilities of atherosclerotic cardiovascular disease (ASCVD) in women from the Korean Genome and Epidemiology Study (KOGES)-Ansan and Ansung study. (A) The Pooled Cohort Equations for non-Hispanic whites. (B) The Pooled Cohort Equations for African Americans. (C) The Framingham Global Cardiovascular Risk Score. (D) The Korean Risk Prediction Model.

• 

  Fig. 3 Ten-year probabilities of atherosclerotic cardiovascular disease (ASCVD) in men and women from the Korean Genome and Epidemiology Study (KOGES)-Ansan and Ansung study using the recalibrated and new Pooled Cohort Equations. (A) The recalibrated Pooled Cohort Equations for non-Hispanic whites in men. (B) The recalibrated Pooled Cohort Equations for African Americans in men. (C) The recalibrated Pooled Cohort Equations for non-Hispanic whites in women. (D) The recalibrated Pooled Cohort Equations for African Americans in women. (E) The new Pooled Cohort Equations in men. (F) The new Pooled Cohort Equations in women.

Reference

1. World Health Organization. World health statistics 2016: monitoring health for the sustainable development goals (SDGs). Geneva: World Health Organization;2016.

2. Lusis AJ. Atherosclerosis. Nature. 2000; 407:233–241. PMID: 11001066.

3. Wong ND. Epidemiological studies of CHD and the evolution of preventive cardiology. Nat Rev Cardiol. 2014; 11:276–289. PMID: 24663092.

4. Blood Pressure Lowering Treatment Trialists Collaboration. Blood pressure-lowering treatment based on cardiovascular risk: a meta-analysis of individual patient data. Lancet. 2014; 384:591–598. PMID: 25131978.

5. Goff DC Jr, Lloyd-Jones DM, Bennett G, Coady S, D'Agostino RB, Gibbons R, Greenland P, Lackland DT, Levy D, O'Donnell CJ, Robinson JG, Schwartz JS, Shero ST, Smith SC Jr, Sorlie P, Stone NJ, Wilson PW, Jordan HS, Nevo L, Wnek J, Anderson JL, Halperin JL, Albert NM, Bozkurt B, Brindis RG, Curtis LH, DeMets D, Hochman JS, Kovacs RJ, Ohman EM, Pressler SJ, Sellke FW, Shen WK, Smith SC Jr, Tomaselli GF. American College of Cardiology/American Heart Association Task Force on Practice Guidelines. 2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation. 2014; 129:S49–S73. PMID: 24222018.

6. Stone NJ, Robinson JG, Lichtenstein AH, Bairey Merz CN, Blum CB, Eckel RH, Goldberg AC, Gordon D, Levy D, Lloyd-Jones DM, McBride P, Schwartz JS, Shero ST, Smith SC Jr, Watson K, Wilson PW, Eddleman KM, Jarrett NM, LaBresh K, Nevo L, Wnek J, Anderson JL, Halperin JL, Albert NM, Bozkurt B, Brindis RG, Curtis LH, DeMets D, Hochman JS, Kovacs RJ, Ohman EM, Pressler SJ, Sellke FW, Shen WK, Smith SC Jr, Tomaselli GF. American College of Cardiology/American Heart Association Task Force on Practice Guidelines. 2013 ACC/AHA guideline on the treatment of blood cholesterol to reduce atherosclerotic cardiovascular risk in adults: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation. 2014; 129:S1–S45. PMID: 24222016.

7. Grundy SM, Stone NJ, Bailey AL, Beam C, Birtcher KK, Blumenthal RS, Braun LT, de Ferranti S, Faiella-Tommasino J, Forman DE, Goldberg R, Heidenreich PA, Hlatky MA, Jones DW, Lloyd-Jones D, Lopez-Pajares N, Ndumele CE, Orringer CE, Peralta CA, Saseen JJ, Smith SC Jr, Sperling L, Virani SS, Yeboah J. 2018 AHA/ACC/AACVPR/AAPA/ABC/ACPM/ADA/AGS/APhA/ASPC/NLA/PCNA guideline on the management of blood cholesterol: a report of the American College of Cardiology/ American Heart Association Task Force on Clinical Practice Guidelines. Circulation. 2019; 139:e1082–e1143. PMID: 30586774.

8. Whelton PK, Carey RM, Aronow WS, Casey DE Jr, Collins KJ, Dennison Himmelfarb C, DePalma SM, Gidding S, Jamerson KA, Jones DW, MacLaughlin EJ, Muntner P, Ovbiagele B, Smith SC Jr, Spencer CC, Stafford RS, Taler SJ, Thomas RJ, Williams KA Sr, Williamson JD, Wright JT Jr. 2017 ACC/AHA/AAPA/ABC/ACPM/AGS/APhA/ASH/ASPC/NMA/PCNA guideline for the prevention, detection, evaluation, and management of high blood pressure in adults: a report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. J Am Coll Cardiol. 2018; 71:e127–e248. PMID: 29146535.

9. DeFilippis AP, Young R, McEvoy JW, Michos ED, Sandfort V, Kronmal RA, McClelland RL, Blaha MJ. Risk score overestimation: the impact of individual cardiovascular risk factors and preventive therapies on the performance of the American Heart Association-American College of Cardiology-Atherosclerotic Cardiovascular Disease risk score in a modern multi-ethnic cohort. Eur Heart J. 2017; 38:598–608. PMID: 27436865.

10. Kavousi M, Leening MJ, Nanchen D, Greenland P, Graham IM, Steyerberg EW, Ikram MA, Stricker BH, Hofman A, Franco OH. Comparison of application of the ACC/AHA guidelines, Adult Treatment Panel III guidelines, and European Society of Cardiology guidelines for cardiovascular disease prevention in a European cohort. JAMA. 2014; 311:1416–1423. PMID: 24681960.

11. Rana JS, Tabada GH, Solomon MD, Lo JC, Jaffe MG, Sung SH, Ballantyne CM, Go AS. Accuracy of the atherosclerotic cardiovascular risk equation in a large contemporary, multiethnic population. J Am Coll Cardiol. 2016; 67:2118–2130. PMID: 27151343.

12. Volgman AS, Palaniappan LS, Aggarwal NT, Gupta M, Khandelwal A, Krishnan AV, Lichtman JH, Mehta LS, Patel HN, Shah KS, Shah SH, Watson KE. American Heart Association Council on Epidemiology and Prevention. ardiovascular Disease and Stroke in Women and Special Populations Committee of the Council on Clinical Cardiology. Council on Cardiovascular and Stroke Nursing. Council on Quality of Care and Outcomes Research. Stroke Council. Atherosclerotic cardiovascular disease in South Asians in the United States: epidemiology, risk factors, and treatments: a scientific statement from the American Heart Association. Circulation. 2018; 138:e1–e34. PMID: 29794080.

13. Hutchinson RN, Shin S. Systematic review of health disparities for cardiovascular diseases and associated factors among American Indian and Alaska Native populations. PLoS One. 2014; 9:e80973. PMID: 24454685.

14. The ARIC investigators. The Atherosclerosis Risk in Communities (ARIC) Study: design and objectives. Am J Epidemiol. 1989; 129:687–702. PMID: 2646917.

15. Fried LP, Borhani NO, Enright P, Furberg CD, Gardin JM, Kronmal RA, Kuller LH, Manolio TA, Mittelmark MB, Newman A, O'Leary DH, Psaty B, Rautaharju P, Tracy RP, Weiler PG. The Cardiovascular Health Study: design and rationale. Ann Epidemiol. 1991; 1:263–276. PMID: 1669507.

16. Friedman GD, Cutter GR, Donahue RP, Hughes GH, Hulley SB, Jacobs DR Jr, Liu K, Savage PJ. CARDIA: study design, recruitment, and some characteristics of the examined subjects. J Clin Epidemiol. 1988; 41:1105–1116. PMID: 3204420.

17. Lloyd-Jones DM, Leip EP, Larson MG, D'Agostino RB, Beiser A, Wilson PW, Wolf PA, Levy D. Prediction of lifetime risk for cardiovascular disease by risk factor burden at 50 years of age. Circulation. 2006; 113:791–798. PMID: 16461820.

18. Pencina MJ, D'Agostino RB Sr, Larson MG, Massaro JM, Vasan RS. Predicting the 30-year risk of cardiovascular disease: the Framingham Heart Study. Circulation. 2009; 119:3078–3084. PMID: 19506114.

19. Kim Y, Han BG. KoGES group. Cohort profile: the Korean Genome and Epidemiology Study (KoGES) Consortium. Int J Epidemiol. 2017; 46:e20. PMID: 27085081.

20. Suh S, Baek J, Bae JC, Kim KN, Park MK, Kim DK, Cho NH, Lee MK. Sex factors in the metabolic syndrome as a predictor of cardiovascular disease. Endocrinol Metab (Seoul). 2014; 29:522–529. PMID: 25559575.

21. D'Agostino RB Sr, Vasan RS, Pencina MJ, Wolf PA, Cobain M, Massaro JM, Kannel WB. General cardiovascular risk profile for use in primary care: the Framingham Heart Study. Circulation. 2008; 117:743–753. PMID: 18212285.

22. Dawber TR, Meadors GF, Moore FE Jr. Epidemiological approaches to heart disease: the Framingham Study. Am J Public Health Nations Health. 1951; 41:279–281. PMID: 14819398.

23. Kannel WB, Feinleib M, McNamara PM, Garrison RJ, Castelli WP. An investigation of coronary heart disease in families. The Framingham offspring study. Am J Epidemiol. 1979; 110:281–290. PMID: 474565.

24. Jung KJ, Jang Y, Oh DJ, Oh BH, Lee SH, Park SW, Seung KB, Kim HK, Yun YD, Choi SH, Sung J, Lee TY, Kim SH, Koh SB, Kim MC, Chang Kim H, Kimm H, Nam C, Park S, Jee SH. The ACC/AHA 2013 pooled cohort equations compared to a Korean Risk Prediction Model for atherosclerotic cardiovascular disease. Atherosclerosis. 2015; 242:367–375. PMID: 26255683.

25. Jee SH, Batty GD, Jang Y, Oh DJ, Oh BH, Lee SH, Park SW, Seung KB, Kimm H, Kim SY, Mok Y, Kim HS, Lee DC, Choi SH, Kim MJ, Lee GJ, Sung J, Cho B, Kim ES, Yu BY, Lee TY, Kim JS, Lee YJ, Oh JK, Kim SH, Park JK, Koh SB, Park SB, Lee SY, Yoo CI, Kim MC, Kim HK, Park JS, Yun YD, Baek SJ, Samet JM, Woodward M. The Korean Heart Study: rationale, objectives, protocol, and preliminary results for a new prospective cohort study of 430,920 men and women. Eur J Prev Cardiol. 2014; 21:1484–1492. PMID: 23864362.

26. Appelman Y, van Rijn BB, Ten Haaf ME, Boersma E, Peters SA. Sex differences in cardiovascular risk factors and disease prevention. Atherosclerosis. 2015; 241:211–218. PMID: 25670232.

27. Mosca L, Barrett-Connor E, Wenger NK. Sex/gender differences in cardiovascular disease prevention: what a difference a decade makes. Circulation. 2011; 124:2145–2154. PMID: 22064958.

28. Demler OV, Paynter NP, Cook NR. Tests of calibration and goodness-of-fit in the survival setting. Stat Med. 2015; 34:1659–1680. PMID: 25684707.

29. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina MJ, Kattan MW. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010; 21:128–138. PMID: 20010215.

30. D'Agostino RB Sr, Grundy S, Sullivan LM, Wilson P. CHD Risk Prediction Group. Validation of the Framingham coronary heart disease prediction scores: results of a multiple ethnic groups investigation. JAMA. 2001; 286:180–187. PMID: 11448281.

31. DeFilippis AP, Young R, Carrubba CJ, McEvoy JW, Budoff MJ, Blumenthal RS, Kronmal RA, McClelland RL, Nasir K, Blaha MJ. An analysis of calibration and discrimination among multiple cardiovascular risk scores in a modern multiethnic cohort. Ann Intern Med. 2015; 162:266–275. PMID: 25686167.

32. WHO Expert Consultation. Appropriate body-mass index for Asian populations and its implications for policy and intervention strategies. Lancet. 2004; 363:157–163. PMID: 14726171.

33. Karthikeyan G, Teo KK, Islam S, McQueen MJ, Pais P, Wang X, Sato H, Lang CC, Sitthi-Amorn C, Pandey MR, Kazmi K, Sanderson JE, Yusuf S. Lipid profile, plasma apolipoproteins, and risk of a first myocardial infarction among Asians: an analysis from the INTERHEART Study. J Am Coll Cardiol. 2009; 53:244–253. PMID: 19147041.

34. Thom T, Haase N, Rosamond W, Howard VJ, Rumsfeld J, Manolio T, Zheng ZJ, Flegal K, O'Donnell C, Kittner S, Lloyd-Jones D, Goff DC Jr, Hong Y, Adams R, Friday G, Furie K, Gorelick P, Kissela B, Marler J, Meigs J, Roger V, Sidney S, Sorlie P, Steinberger J, Wasserthiel-Smoller S, Wilson M, Wolf P. American Heart Association Statistics Committee and Stroke Statistics Subcommittee. Heart disease and stroke statistics: 2006 update: a report from the American Heart Association Statistics Committee and Stroke Statistics Subcommittee. Circulation. 2006; 113:e85–e151. PMID: 16407573.

35. Lee SW, Kim HC, Lee HS, Suh I. Thirty-year trends in mortality from cardiovascular diseases in Korea. Korean Circ J. 2015; 45:202–209. PMID: 26023308.

36. Kim HJ, Kim Y, Cho Y, Jun B, Oh KW. Trends in the prevalence of major cardiovascular disease risk factors among Korean adults: results from the Korea National Health and Nutrition Examination Survey, 1998-2012. Int J Cardiol. 2014; 174:64–72. PMID: 24742812.

37. Garcia M, Mulvagh SL, Merz CN, Buring JE, Manson JE. Cardiovascular disease in women: clinical perspectives. Circ Res. 2016; 118:1273–1293. PMID: 27081110.

38. EUGenMed Cardiovascular Clinical Study Group. Regitz-Zagrosek V, Oertelt-Prigione S, Prescott E, Franconi F, Gerdts E, Foryst-Ludwig A, Maas AH, Kautzky-Willer A, Knappe-Wegner D, Kintscher U, Ladwig KH, Schenck-Gustafsson K, Stangl V. Gender in cardiovascular diseases: impact on clinical manifestations, management, and outcomes. Eur Heart J. 2016; 37:24–34. PMID: 26530104.

39. Yusuf S, Hawken S, Ounpuu S, Dans T, Avezum A, Lanas F, McQueen M, Budaj A, Pais P, Varigos J, Lisheng L. INTERHEART Study Investigators. Effect of potentially modifiable risk factors associated with myocardial infarction in 52 countries (the INTERHEART study): case-control study. Lancet. 2004; 364:937–952. PMID: 15364185.

40. Camici PG, d'Amati G, Rimoldi O. Coronary microvascular dysfunction: mechanisms and functional assessment. Nat Rev Cardiol. 2015; 12:48–62. PMID: 25311229.

41. Reynolds HR, Srichai MB, Iqbal SN, Slater JN, Mancini GB, Feit F, Pena-Sing I, Axel L, Attubato MJ, Yatskar L, Kalhorn RT, Wood DA, Lobach IV, Hochman JS. Mechanisms of myocardial infarction in women without angiographically obstructive coronary artery disease. Circulation. 2011; 124:1414–1425. PMID: 21900087.

42. von Mering GO, Arant CB, Wessel TR, McGorray SP, Bairey Merz CN, Sharaf BL, Smith KM, Olson MB, Johnson BD, Sopko G, Handberg E, Pepine CJ, Kerensky RA. National Heart, Lung, and Blood Institute. Abnormal coronary vasomotion as a prognostic indicator of cardiovascular events in women: results from the National Heart, Lung, and Blood Institute-Sponsored Women's Ischemia Syndrome Evaluation (WISE). Circulation. 2004; 109:722–725. PMID: 14970106.

43. de Las Heras Gala T, Geisel MH, Peters A, Thorand B, Baumert J, Lehmann N, Jockel KH, Moebus S, Erbel R, Meisinger C, Mahabadi AA, Koenig W. KORA Study Group and the Heinz Nixdorf Recall Study Group. Recalibration of the ACC/AHA risk score in two population-based German cohorts. PLoS One. 2016; 11:e0164688. PMID: 27732641.

44. Yadlowsky S, Hayward RA, Sussman JB, McClelland RL, Min YI, Basu S. Clinical implications of revised pooled cohort equations for estimating atherosclerotic cardiovascular disease risk. Ann Intern Med. 2018; 169:20–29. PMID: 29868850.

45. Lee CH, Woo YC, Lam JK, Fong CH, Cheung BM, Lam KS, Tan KC. Validation of the pooled cohort equations in a long-term cohort study of Hong Kong Chinese. J Clin Lipidol. 2015; 9:640–646. PMID: 26350809.

46. Majed B, Tafflet M, Kee F, Haas B, Ferrieres J, Montaye M, Ruidavets JB, Arveiler D, Yarnell J, Amouyel P, Ducimetiere P, Empana JP. PRIME study group. External validation of the 2008 Framingham cardiovascular risk equation for CHD and stroke events in a European population of middle-aged men. The PRIME study. Prev Med. 2013; 57:49–54. PMID: 23603213.

47. Chia YC, Gray SY, Ching SM, Lim HM, Chinna K. Validation of the Framingham general cardiovascular risk score in a multiethnic Asian population: a retrospective cohort study. BMJ Open. 2015; 5:e007324.