0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Original Investigation |

Validation of the Atherosclerotic Cardiovascular Disease Pooled Cohort Risk Equations FREE

Paul Muntner, PhD1,2; Lisandro D. Colantonio, MD1; Mary Cushman, MD3; David C. Goff Jr, MD, PhD4; George Howard, DrPh5; Virginia J. Howard, PhD1; Brett Kissela, MD, MS6; Emily B. Levitan, ScD1; Donald M. Lloyd-Jones, MD, ScM7; Monika M. Safford, MD2
[+] Author Affiliations
1Department of Epidemiology, University of Alabama at Birmingham
2Department of Medicine, University of Alabama at Birmingham
3Department of Medicine, University of Vermont, Burlington
4Department of Epidemiology, Colorado School of Public Health, Aurora
5Department of Biostatistics, University of Alabama at Birmingham
6Department of Neurology and Rehabilitation Medicine, University of Cincinnati, Cincinnati, Ohio
7Department of Preventive Medicine, Northwestern University Feinberg School of Medicine, Chicago, Illinois
JAMA. 2014;311(14):1406-1415. doi:10.1001/jama.2014.2630.
Text Size: A A A
Published online

Importance  The American College of Cardiology/American Heart Association (ACC/AHA) Pooled Cohort risk equations were developed to estimate atherosclerotic cardiovascular disease (CVD) risk and guide statin initiation.

Objective  To assess calibration and discrimination of the Pooled Cohort risk equations in a contemporary US population.

Design, Setting, and Participants  Adults aged 45 to 79 years enrolled in the Reasons for Geographic and Racial Differences in Stroke (REGARDS) study between January 2003 and October 2007 and followed up through December 2010. We studied participants for whom atherosclerotic CVD risk may trigger a discussion of statin initiation (those without clinical atherosclerotic CVD or diabetes, low-density lipoprotein cholesterol level between 70 and 189 mg/dL, and not taking statins; n = 10 997).

Main Outcomes and Measures  Predicted risk and observed adjudicated atherosclerotic CVD incidence (nonfatal myocardial infarction, coronary heart disease [CHD] death, nonfatal or fatal stroke) at 5 years because REGARDS participants have not been followed up for 10 years. Additional analyses, limited to Medicare beneficiaries (n = 3333), added atherosclerotic CVD events identified in Medicare claims data.

Results  There were 338 adjudicated events (192 CHD events, 146 strokes). The observed and predicted 5-year atherosclerotic CVD incidence per 1000 person-years for participants with a 10-year predicted atherosclerotic CVD risk of less than 5% was 1.9 (95% CI, 1.3-2.7) and 1.9, respectively, risk of 5% to less than 7.5% was 4.8 (95% CI, 3.4-6.7) and 4.8, risk of 7.5% to less than 10% was 6.1 (95% CI, 4.4-8.6) and 6.9, and risk of 10% or greater was 12.0 (95% CI, 10.6-13.6) and 15.1 (Hosmer-Lemeshow χ2 = 19.9, P = .01). The C index was 0.72 (95% CI, 0.70-0.75). There were 234 atherosclerotic CVD events (120 CHD events, 114 strokes) among Medicare-linked participants and the observed and predicted 5-year atherosclerotic CVD incidence per 1000 person-years for participants with a predicted risk of less than 7.5% was 5.3 (95% CI, 2.8-10.1) and 4.0, respectively, risk of 7.5% to less than 10% was 7.9 (95% CI, 4.6-13.5) and 6.4, and risk of 10% or greater was 17.4 (95% CI, 15.3-19.8) and 16.4 (Hosmer-Lemeshow χ2 = 5.4, P = .71). The C index was 0.67 (95% CI, 0.64-0.71).

Conclusions and Relevance  In this cohort of US adults for whom statin initiation is considered based on the ACC/AHA Pooled Cohort risk equations, observed and predicted 5-year atherosclerotic CVD risks were similar, indicating that these risk equations were well calibrated in the population for which they were designed to be used, and demonstrated moderate to good discrimination.

Figures in this Article

The American College of Cardiology (ACC) and the American Heart Association (AHA) recently published the 2013 Guideline on the Assessment of Cardiovascular Risk.1 As part of this guideline, a working group developed new equations for the prediction of 10-year atherosclerotic cardiovascular disease risk, the Pooled Cohort risk equations. These equations were derived in several population-based cohorts that included large samples of blacks and whites and were aimed at estimating 10-year risk for nonfatal myocardial infarction (MI), coronary heart disease (CHD) death, and nonfatal or fatal stroke (hard atherosclerotic cardiovascular disease events). The 2013 ACC/AHA cholesterol treatment guidelines recommend using the Pooled Cohort risk equations to estimate atherosclerotic cardiovascular disease risk and to help guide the decision to initiate statin therapy for primary prevention in adults without clinical atherosclerotic cardiovascular disease or diabetes, and with a low-density lipoprotein cholesterol (LDL-C) level between 70 and 189 mg/dL.2

Many of the studies used to develop the new atherosclerotic cardiovascular disease risk equations recruited and followed up participants before 2000, and there have been marked declines in CHD and stroke incidence over the past 2 decades.3,4 In light of the need for new prediction models to undergo external validation, we evaluated the calibration and discrimination of these equations for predicting atherosclerotic cardiovascular disease risk in a contemporary population-based cohort, the Reasons for Geographic and Racial Differences in Stroke (REGARDS) study. Calibration provides assessment of whether a risk prediction model accurately estimates the absolute observed risk level. Discrimination provides assessment of whether a risk prediction model accurately rank orders individuals (ie, are individuals with higher predicted risk more likely to have events). In addition to analyses of the overall REGARDS population without atherosclerotic cardiovascular disease at baseline, we conducted analyses restricted to the population for which the Pooled Cohorts risk equations are intended to inform discussions about initiating statin use (participants without clinical atherosclerotic cardiovascular disease or diabetes, with LDL-C levels between 70 and 189 mg/dL, and not taking statins). To maximize event surveillance, we also conducted analyses including atherosclerotic cardiovascular disease events identified in Medicare claims data.

Study Population

The REGARDS study was designed to investigate reasons underlying the higher stroke mortality in blacks compared with whites and residents of the southeastern region vs other regions of the United States.5 Coronary heart disease events are being identified and adjudicated in an ancillary study.6 The REGARDS participants were selected from a commercially available nationwide list purchased through Genesys Inc. Criteria for inclusion in the sample include having a name, telephone number, and address in the Genesys database. A letter and study brochure were sent to potential participants and telephone contact was attempted approximately 2 weeks later. A total of 30 239 adults aged 45 years or older from all 48 contiguous US states and the District of Columbia were enrolled between January 2003 and October 2007.

The inclusion criteria for the current analyses were chosen to match those used in the development of the Pooled Cohort risk equations. We restricted the analysis to participants aged 45 to 79 years without a history of CHD, stroke, heart failure, or atrial fibrillation at baseline. A history of heart failure was not ascertained during the REGARDS baseline study visit. As a proxy for heart failure, we excluded participants taking digoxin, which was determined through baseline pill bottle review. We further excluded participants with missing data on components of the Pooled Cohort risk equations. The REGARDS study protocol was approved by the institutional review boards governing research in human subjects at the participating centers; all participants provided written informed consent.

Data Collected at Baseline

Computer-assisted telephone interviews were administered by trained staff and were used to collect information on participants’ age, race, sex, smoking status, prior diagnosed comorbid conditions, and use of antihypertensive and antidiabetes medications. Race was self-reported. Following the interview, trained health professionals conducted in-home examinations that included blood pressure measurements, an electrocardiogram, collection of a blood sample, plus a review of prescription and over-the-counter medications used during the 2-week period prior to the study visit. Blood pressure was measured 2 times following a standardized protocol and averaged for analysis. Levels of total and high-density lipoprotein cholesterol (HDL-C), serum triglycerides, and glucose were measured using the Ortho Vitros Clinical Chemistry System 950IRC instrument (Johnson & Johnson Clinical Diagnostics).

For participants who fasted prior to their study visit with serum levels of triglycerides lower than 400 mg/dL, LDL-C was calculated using the Friedewald equation.7 For all other participants, non–HDL-C was calculated. Diabetes was defined as a glucose level of 126 mg/dL or higher for participants who had fasted prior to their blood draw, a glucose level of 200 mg/dL or higher for those nonfasting participants, or self-report of a prior diagnosis of diabetes with current use of insulin or oral hypoglycemic medications.

Through the pill bottle review, we identified the use of statins at baseline. Digoxin use was identified and served as a marker for prevalent heart failure.8 Atrial fibrillation was defined based on the study electrocardiogram or self-report. History of CHD at baseline was defined by self-report of MI or revascularization procedure or evidence on the study electrocardiogram of MI. History of stroke was defined by self-report.

Data Collected During Follow-up

Living participants or their proxies were contacted every 6 months via telephone to assess new-onset stroke and CHD events. Consistent with the definition used to derive the Pooled Cohort risk equations, the outcome for our primary analyses was defined as the first atherosclerotic cardiovascular disease event, which included a nonfatal or fatal stroke, a nonfatal MI, or CHD death. Adjudication of stroke and CHD events are currently available through December 31, 2010.

Stroke

Stroke events were identified via self-report or proxy report of stroke, transient ischemic attack, or stroke symptoms.9,10 For reported events, hospital charts and physician office records were retrieved for adjudication. Stroke events were confirmed by a panel of experts according to the World Health Organization (WHO) definition.11 Events not meeting the WHO definition but characterized by symptoms lasting less than 24 hours with neuroimaging consistent with acute infarct or hemorrhage were classified as clinical strokes. The present analysis included WHO-defined stroke events as well as clinical stroke.12

Coronary Heart Disease

The CHD events (nonfatal MI or CHD death) were detected during the follow-up telephone interviews. Medical records were retrieved and events were adjudicated by trained clinicians following published guidelines.13,14 Records were examined for the presence of signs or symptoms suggestive of ischemia, an increasing and/or decreasing pattern in levels of cardiac troponin or creatine kinase-MB over 6 or more hours with a peak value greater than or equal to twice the upper limit of normal, and electrocardiogram changes consistent with ischemia or MI (guided by the Minnesota code and classified as evolving diagnostic, positive, nonspecific, or not consistent with ischemia).15,16 For deceased participants, interviews with next of kin or proxies, medical records from the last year of life, death certificates, and autopsy reports were reviewed to determine if stroke or CHD was the main underlying cause of death. Only definite or probable events were included.

Events Identified in Medicare Claims

The cohorts used to develop the Pooled Cohort risk equations had surveillance components (eg, review of hospital discharges and obituaries in local newspapers) to detect atherosclerotic cardiovascular disease events not reported by participants. The REGARDS study did not have this active surveillance and, therefore, some atherosclerotic cardiovascular disease events were possibly missed. To address this limitation, we used Medicare-linked claims data to identify atherosclerotic cardiovascular disease events not detected through routine cohort follow-up. Medicare provides health insurance to adults aged 65 years or older, and to those with end-stage renal disease or disability.

The REGARDS participants were linked to Medicare enrollment and claims data from 1999 through 2010 by Social Security number, sex, and date of birth. Myocardial infarctions were defined by an overnight hospitalization in an acute care facility with an International Classification of Diseases, Ninth Revision (ICD-9) discharge diagnosis code of 410.xx (except 410.x2, which indicates a subsequent episode of care) in any position and stroke events were defined by an ICD-9 discharge diagnosis code of 430.xx, 431.xx, 433.xx, 434.xx, or 436.x in the primary position. These definitions have positive predictive values of greater than 90%.1719

Statistical Analysis

Analyses were performed for the overall population and, separately, in the subgroup for whom the 2013 cholesterol treatment guidelines recommend using 10-year atherosclerotic cardiovascular disease risk from the Pooled Cohort risk equations to inform discussions about initiating statin use for the primary prevention of atherosclerotic cardiovascular disease (ie, participants without atherosclerotic cardiovascular disease or diabetes, with an LDL-C level between 70 and 189 mg/dL or, if LDL-C was not available (n = 1255), non–HDL-C level between 100 and 219 mg/dL, and not already taking statins).2 We calculated each participant’s predicted 10-year atherosclerotic cardiovascular disease risk using the Pooled Cohort risk equations.

Participants were categorized into 4 groups according to their 10-year predicted atherosclerotic cardiovascular disease risk: less than 5%, 5% to less than 7.5%, 7.5% to less than 10%, and 10% or greater. Participant characteristics, including age, race, sex, current smoking, diabetes, systolic blood pressure, use of antihypertensive medication, total cholesterol, HDL-C, LDL-C, and use of statins were calculated within each 10-year predicted atherosclerotic cardiovascular disease risk group.

Because the REGARDS study has not yet completed 10 years of follow-up, we calculated observed and predicted atherosclerotic cardiovascular disease incidence rates at 5 years within the 4 atherosclerotic cardiovascular disease risk groups described above. Observed rates were calculated using adjudicated events. Participants were censored at the time the first of the following events occurred: (1) an atherosclerotic cardiovascular disease event; (2) death; (3) their last REGARDS follow-up interview; (4) 5 years of follow-up; or (5) December 31, 2010. Overall, 53.6% of REGARDS participants were censored at 5 years of follow-up free of atherosclerotic cardiovascular disease events. Predicted atherosclerotic cardiovascular disease incidence at 5 years of follow-up was calculated using the Pooled Cohort risk equations and (S0(t)) at 5 years (eTable 1 in Supplement). The observed number of atherosclerotic cardiovascular disease events at 5 years was adjusted for variable follow-up time using the Kaplan-Meier estimate.20

The predicted number of events was calculated based on the mean predicted atherosclerotic cardiovascular disease incidence at 5 years. Next, participants were grouped into deciles of predicted atherosclerotic cardiovascular disease risk. The calibration of the Pooled Cohort risk equations was determined using the observed and predicted number of atherosclerotic cardiovascular disease events at 5 years of follow-up in each decile and a modified Hosmer-Lemeshow χ2 statistic.21 A χ2 value of greater than 20 or a P value of less than .05 indicates poor calibration. We calculated the C index to estimate discrimination of the atherosclerotic cardiovascular disease risk equation.22,23 Although no thresholds exist, a C index between 0.70 and 0.80 is considered moderate to good and 0.80 or greater is considered excellent.24

The above analyses were performed for the overall population, and separately for men and women and for whites and blacks. Analyses were also repeated for participants residing in the stroke belt (North Carolina, South Carolina, Georgia, Alabama, Mississippi, Louisiana, Tennessee, and Arkansas) and non–stroke-belt regions of the continental United States. Analyses were repeated and limited to participants aged 65 years or older at baseline with Medicare Part A coverage and included adjudicated atherosclerotic cardiovascular disease events plus events identified in the Medicare claims. Two-sided P values of less than .05 were considered statistically significant. Analyses were conducted using SAS statistical software version 9.3 (SAS Institute Inc).

Study Population

We restricted the analyses to REGARDS participants aged 45 to 79 years (n = 28 044, eFigure in Supplement). After excluding 7326 participants with a history of CHD, stroke, atrial fibrillation, or heart failure, there were missing data for the Pooled Cohort risk equations for 1929 participants and an additional 291 participants without follow-up data; therefore, 18 498 participants were available for analysis. After further excluding participants with diabetes, a LDL-C level of less than 70 mg dL or of 190 mg/dL or greater, or with statin use at baseline, there were 10 997 participants available in the subgroup for whom the 2013 cholesterol treatment guidelines recommend consideration of statin initiation based on their estimated atherosclerotic cardiovascular disease risk.

At baseline, 25.0% of REGARDS participants were taking statins. Pooled Cohort risk equation components are provided by 10-year predicted risk of atherosclerotic cardiovascular disease in Table 1. Additional characteristics of REGARDS study participants are presented in eTable 2 in Supplement. In both populations, participants with higher 10-year predicted atherosclerotic cardiovascular disease risk were older and a higher percentage had black race, were men, were current smokers, and were taking an antihypertensive medication.

Table Graphic Jump LocationTable 1.  Baseline Characteristics of REGARDS Study Participants

Mean systolic blood pressure and LDL-C level were higher and mean HDL-C level was lower for those with higher 10-year predicted atherosclerotic cardiovascular disease risk. In the overall population, a higher prevalence of diabetes and statin use at baseline was present and total cholesterol level was lower in participants with higher 10-year predicted risk of atherosclerotic cardiovascular disease. Level of total cholesterol was similar across atherosclerotic cardiovascular disease risk groups among participants considered for statin initiation based on their atherosclerotic cardiovascular disease risk.

Pooled Cohort Risk Equations in the Overall Population

There were 674 adjudicated atherosclerotic cardiovascular disease events (382 CHD events and 292 strokes) over 79 321 person-years of follow-up in the overall cohort. For participants with a 10-year predicted atherosclerotic cardiovascular disease risk of less than 5%, observed and predicted 5-year incidence rates were 2.2 (95% CI, 1.7-3.0) and 2.0, respectively, per 1000 person-years (Table 2). In higher 10-year predicted atherosclerotic cardiovascular disease risk strata, 5-year observed risk was lower than predicted risk. For example, for those with predicted risk of 10% or greater, the observed and predicted risk were 12.6 (95% CI, 12.0-14.2) and 17.8, respectively.

Table Graphic Jump LocationTable 2.  Observed and Predicted Incidence Rates of Atherosclerotic Cardiovascular Disease in the REGARDS Study by 10-Year Predicted Risk

Calibration for the overall population was poor (Hosmer-Lemeshow χ2 = 84.2, P<.001) (Table 2, the Figure, and eTable 3 in Supplement). The Pooled Cohort risk equations overestimated risk for men and women and whites and blacks. The C index for the overall population was 0.71 (95% CI, 0.69-0.72) and was higher in women compared with men and whites compared with blacks.

Place holder to copy figure label and caption
Figure.
Observed and Predicted Atherosclerotic Cardiovascular Disease Risk Among REGARDS Participants

Predicted risk determined using the Pooled Cohort equations. LDL-C indicates low-density lipoprotein cholesterol; REGARDS, Reasons for Geographic and Racial Differences in Stroke.aThe range of predicted risk for each decile is provided in eTable 3 in Supplement.bThe range of predicted risk for each decile in the REGARDS population with Medicare insurance coverage is provided in eTable 6 in Supplement.cMedicare data are not presented due to a small sample size.

Graphic Jump Location
Pooled Cohort Risk Equations in the Population Considered for Statin Initiation Based on Estimated Atherosclerotic Cardiovascular Disease Risk

Among the subgroup for whom statin treatment should be considered based on atherosclerotic cardiovascular disease risk, there were 338 adjudicated events (192 CHD events and 146 strokes) over 47 481 person-years. Calibration was better in this clinically relevant population, with less overestimation of risk (Hosmer-Lemeshow χ2 = 19.9, P = .01) (Table 3, the Figure, and eTable 3). Most of the overestimation of risk occurred in deciles 7 through 10 of risk, in which participants had a 10-year predicted atherosclerotic cardiovascular disease risk of 10% or greater. Additionally, the Hosmer-Lemeshow χ2 indicated good calibration among women (χ2 = 8.3, P = .41), blacks (χ2 = 11.8, P = .16), and whites (χ2 = 14.0, P = .08). The C index was 0.72 (95% CI, 0.70-0.75) for all participants and, similar to the overall population, was higher in women and whites compared with men and blacks. The Pooled Cohort risk equations performed similarly in the stroke belt (North Carolina, South Carolina, Georgia, Alabama, Mississippi, Louisiana, Tennessee, and Arkansas) and in the remainder of the continental United States (eTable 4 in Supplement).

Table Graphic Jump LocationTable 3.  Observed and Predicted Incidence Rates of Atherosclerotic Cardiovascular Disease Among REGARDS Study Participants Without Diabetes, With Low-Density Lipoprotein Cholesterol of 70 to 189 mg/dL, and Who Were Not Taking Statins by 10-Year Predicted Riska
REGARDS Participants With Medicare-Linked Data

Among the subset of 6121 REGARDS participants with Medicare-linked data, 3333 were in the subgroup for which consideration of statin treatment should be based on atherosclerotic cardiovascular disease risk. Due to the limited number of participants with 10-year predicted atherosclerotic cardiovascular disease risk of less than 5%, we pooled participants with less than 7.5% for these analyses. Characteristics of participants by 10-year predicted risk are provided in eTable 5 in Supplement.

There were 457 atherosclerotic cardiovascular disease events (225 CHD events and 232 strokes, which includes 112 [24.5%] events identified in Medicare claims) during 27 524 person-years in the overall REGARDS population with Medicare-linked data and 234 atherosclerotic cardiovascular disease events (120 CHD events and 114 strokes, which includes 57 [24.4%] events identified in Medicare claims) during 15 094 person-years in the REGARDS population with Medicare-linked data considered for statin treatment based on atherosclerotic cardiovascular disease risk. For both of these populations, the Pooled Cohort risk equations were well calibrated (Table 4, the Figure, and eTable 6 in Supplement).

Table Graphic Jump LocationTable 4.  Observed and Predicted Incidence Rates of Atherosclerotic Cardiovascular Disease Among REGARDS Study Participants Linked by Medicare

With more complete ascertainment of events in this subgroup, there tended to be modest underprediction of event rates by the Pooled Cohort equations. For example, among participants in the REGARDS population with Medicare-linked data who could be considered for statin treatment based on atherosclerotic cardiovascular disease risk, the observed and predicted risk for participants with a 10-year predicted risk of less than 7.5% was 5.3 (95% CI, 2.8-10.1) and 4.0, respectively, risk of 7.5% to less than 10% was 7.9 (95% CI, 4.6-13.5) and 6.4, and risk of 10% or greater was 17.4 (95% CI, 15.3-19.8) and 16.4 (χ2 = 5.4, P = .71).

In this large, contemporary, population-based cohort of black and white US adults, the recently published ACC/AHA Pooled Cohort risk equations appeared to overestimate atherosclerotic cardiovascular disease risk. However, differences in the observed and predicted atherosclerotic cardiovascular disease risk were small when limited to participants without diabetes, with a LDL-C level between 70 and 189 mg/dL, and who were not already taking statins. Calibration in this group is particularly important because it represents the population for whom high predicted risk is intended to trigger a discussion about statin initiation. Furthermore, the observed and predicted atherosclerotic cardiovascular disease risks were much more similar when evaluated in participants with Medicare insurance coverage, including the atherosclerotic cardiovascular disease events identified in Medicare claims. In addition to demonstrating good calibration, the Pooled Cohort risk equations had good discrimination. These findings support the validity of the Pooled Cohort risk equations to inform clinical management decisions.

For risk equations to be useful in clinical practice, they should be well calibrated so that predicted risk estimates are similar to observed disease incidence. In the full working group report accompanying the publication of the Pooled Cohort risk equations, some overprediction of atherosclerotic cardiovascular disease risk was noted in short-term follow-up of the REGARDS study and the Multi-Ethnic Study of Atherosclerosis.1 Following the publication of the working group report, predicted atherosclerotic cardiovascular disease risk using the Pooled Cohort risk equations was reported to be systematically higher than the observed risk in the Women’s Health Study, the Physician’s Health Study, and the Women’s Health Initiative Observational Study.25

Lack of active surveillance in these studies may have led to the appearance of overprediction by the Pooled Cohort risk equations because of underascertainment of events. As reported recently, 3.6% (1345/37 397) of women aged 65 years or older with Medicare Part A coverage in the Women’s Health Initiative Clinical Trial had an MI when defined by study adjudication vs 4.8% (1784/37 397) when including events identified by Medicare claims, which is a 33% higher number of events and is similar to the 25% observed in REGARDS study.26 The primary reasons for these events not being adjudicated were participants not reporting an event or the inability to retrieve hospital records for adjudication. As we demonstrated in the current analysis, atherosclerotic cardiovascular disease risk may appear to be overestimated by the Pooled Cohort risk equations if all events are not identified. The predicted and observed risks were remarkably similar when incorporating a surveillance component (ie, Medicare claims) to provide more complete capture of all clinically relevant events.

A second potential reason for the reported overestimation of atherosclerotic cardiovascular disease risk using the Pooled Cohort risk equations is the high prevalence of statin use in contemporary cohorts.27 At baseline, 29.5% of the REGARDS participants with a 10-year atherosclerotic cardiovascular disease risk of 10% or greater, the group in which the overestimation of atherosclerotic cardiovascular disease risk has been most pronounced, were taking statins. However, the Pooled Cohort risk equations were well calibrated in the subgroup for which these equations were designed to be used, providing assurance of their clinical utility.

The Pooled Cohort risk equations performed well in discriminating between low-risk and high-risk participants. The C index of 0.71 for the Pooled Cohort risk equations in the overall REGARDS study population is similar to the C index observed for the external validation of the Framingham 10-year CHD risk score used in the Adult Treatment Panel III guidelines.28 Additionally, the C index in the current analysis was 0.72 for the population for which the 2013 cholesterol treatment guidelines recommend consideration of statin initiation based on atherosclerotic cardiovascular disease risk.

The C index was substantially lower in our analyses of REGARDS participants aged 65 years or older with Medicare coverage. This was not surprising because discrimination is expected to be lower when risk prediction models are applied in narrowly defined populations. Also, as noted in the 2013 ACC/AHA Guideline on the Assessment of Cardiovascular Risk, there are atherosclerotic cardiovascular disease risk factors that might improve the discrimination of the Pooled Cohort risk equations. Adding risk factors (eg, coronary artery calcium score) to the Pooled Cohort risk equations should be examined in future analyses.

A strength of the current analyses includes the large number of REGARDS study participants residing in the continental United States. The REGARDS study enrolled community-dwelling adults and provides high generalizability to white and black US adults. The REGARDS cohort participant data are linked to Medicare claims, thus providing surveillance for a large subcohort.

The results of our study should be interpreted in the context of its limitations. Although follow-up of REGARDS participants is ongoing, data were only available to calculate observed atherosclerotic cardiovascular disease risk at 5 years. Because the Pooled Cohort risk equations were designed to estimate 10-year atherosclerotic cardiovascular disease risk, studies are needed to ensure its accurate calibration over a longer duration. Even though high positive predictive values have been reported for CHD and stroke events identified using claims-based algorithms,1719 these algorithms have not been validated in the REGARDS study.

Therefore, it is possible that the observed atherosclerotic cardiovascular disease risk when including Medicare events may be overestimated, which could affect the good calibration of the Pooled Cohort that we report. We were not able to assess the effect of statin initiation after baseline on the calibration of the risk equations. Although risk prediction is a useful tool for guiding preventive approaches, counseling and treatment decisions should be individualized, as suggested by the new cholesterol guidelines.

In this cohort of US adults for whom statin initiation is considered based on the ACC/AHA pooled risk equations, observed and predicted 5-year atherosclerotic cardiovascular disease risks were similar, indicating that these risk equations were well calibrated in the population for which they were designed to be used, and demonstrated moderate to good discrimination.

Corresponding Author: Paul Muntner, PhD, University of Alabama at Birmingham, 1665 University Blvd, Ste 230J, Birmingham, AL 35294 (pmuntner@uab.edu).

Published Online: March 29, 2014. doi:10.1001/jama.2014.2630.

Author Contributions: Drs Muntner and Colantonio had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Muntner, Cushman, Goff, G. Howard, Levitan, Safford.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Muntner.

Critical revision of the manuscript for important intellectual content: Colantonio, Cushman, Goff, G. Howard, V. Howard, Kissela, Levitan, Lloyd-Jones, Safford.

Statistical analysis: Muntner, Colantonio, Levitan.

Obtained funding: Cushman, V. Howard, Kissela, Safford.

Administrative, technical, or material support: Cushman, G. Howard, V. Howard, Lloyd-Jones, Safford.

Study supervision: Muntner, G. Howard, V. Howard, Safford.

Conflict of Interest Disclosures: The authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr Muntner reported receiving grants and personal fees from Amgen Inc outside the submitted work. Dr Cushman reported receiving grants from diaDexus outside the submitted work. Dr Levitan reported receiving grants from Amgen Inc outside the submitted work. No other disclosures were reported.

Funding/Support: This research project is supported by a cooperative agreement U01 NS041588 from the National Institute of Neurological Disorders and Stroke, National Institutes of Health, Department of Health and Human Services. Additional support was provided by grants R01 HL080477 and K24 HL111154 from the National Heart, Lung, and Blood Institute.

Role of the Sponsor: Representatives of the National Institutes of Health have been involved in the review of the manuscript but not directly involved in the collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Neurological Disorders and Stroke or the National Institutes of Health.

Additional Information: Drs Goff and Lloyd-Jones served as co-chairs of the 2013 American College of Cardiology/American Heart Association Guideline on the Assessment of Cardiovascular Risk.

Additional Contributions: We thank the other investigators, the staff, and the participants of the REGARDS study for their valuable contributions. A full list of participating REGARDS investigators and institutions and further information about the study can be found at http://www.regardsstudy.org.

Goff  DC  Jr, Lloyd-Jones  DM, Bennett  G,  et al.  2013 ACC/AHA Guideline on the Assessment of Cardiovascular Risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines [published November 12, 2013]. J Am Coll Cardiol. doi:10.1016/j.jacc.2013.1011.1005.
PubMed
Stone  NJ, Robinson  J, Lichtenstein  AH,  et al.  2013 ACC/AHA guideline on the treatment of blood cholesterol to reduce atherosclerotic cardiovascular risk in adults: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines [published online November 7, 2013]. J Am Coll Cardiol. doi:10.1016/j.jacc.2013.11.002.
PubMed
Rosamond  WD, Chambless  LE, Heiss  G,  et al.  Twenty-two-year trends in incidence of myocardial infarction, coronary heart disease mortality, and case fatality in 4 US communities, 1987-2008. Circulation. 2012;125(15):1848-1857.
PubMed   |  Link to Article
Kleindorfer  DO, Khoury  J, Moomaw  CJ,  et al.  Stroke incidence is decreasing in whites but not in blacks: a population-based estimate of temporal trends in stroke incidence from the Greater Cincinnati/Northern Kentucky Stroke Study. Stroke. 2010;41(7):1326-1331.
PubMed   |  Link to Article
Howard  VJ, Cushman  M, Pulley  L,  et al.  The Reasons for Geographic and Racial Differences in Stroke study: objectives and design. Neuroepidemiology. 2005;25(3):135-143.
PubMed   |  Link to Article
Safford  MM, Brown  TM, Muntner  PM,  et al; REGARDS Investigators.  Association of race and sex with risk of incident acute coronary heart disease events. JAMA. 2012;308(17):1768-1774.
PubMed   |  Link to Article
Friedewald  WT, Levy  RI, Fredrickson  DS.  Estimation of the concentration of low-density lipoprotein cholesterol in plasma, without use of the preparative ultracentrifuge. Clin Chem. 1972;18(6):499-502.
PubMed
Fonseca  C, Oliveira  AG, Mota  T,  et al; EPICA Investigators.  Evaluation of the performance and concordance of clinical questionnaires for the diagnosis of heart failure in primary care. Eur J Heart Fail. 2004;6(6):813-822.
PubMed   |  Link to Article
Meschia  JF, Brott  TG, Chukwudelunzu  FE,  et al.  Verifying the stroke-free phenotype by structured telephone interview. Stroke. 2000;31(5):1076-1080.
PubMed   |  Link to Article
Howard  VJ, Kleindorfer  DO, Judd  SE,  et al.  Disparities in stroke incidence contributing to disparities in stroke mortality. Ann Neurol. 2011;69(4):619-627.
PubMed   |  Link to Article
 Stroke—1989: recommendations on stroke prevention, diagnosis, and therapy: report of the WHO Task Force on Stroke and other Cerebrovascular Disorders. Stroke. 1989;20(10):1407-1431.
PubMed   |  Link to Article
Soliman  EZ, Howard  G, Cushman  M,  et al.  Prolongation of QTc and risk of stroke: the REGARDS (REasons for Geographic and Racial Differences in Stroke) study. J Am Coll Cardiol. 2012;59(16):1460-1467.
PubMed   |  Link to Article
Thygesen  K, Alpert  JS, White  HD,  et al; Joint ESC/ACCF/AHA/WHF Task Force for the Redefinition of Myocardial Infarction.  Universal definition of myocardial infarction. Circulation. 2007;116(22):2634-2653.
PubMed   |  Link to Article
Luepker  RV, Apple  FS, Christenson  RH,  et al; AHA Council on Epidemiology and Prevention; AHA Statistics Committee; World Heart Federation Council on Epidemiology and Prevention; European Society of Cardiology Working Group on Epidemiology and Prevention; Centers for Disease Control and Prevention; National Heart, Lung, and Blood Institute.  Case definitions for acute coronary heart disease in epidemiology and clinical research studies. Circulation. 2003;108(20):2543-2549.
PubMed   |  Link to Article
Prineas  RJ, Crow  RS, Blackburn  H. The Minnesota Code Manual of Electrocardiographic Findings: Standards and Procedures for Measurement and Classification. Boston, MA: Wright-OSG; 1982.
Prineas  RJ, Crow  RS, Zhang  ZM. Minnesota Code Manual of Electrocardiographic Findings. 2nd ed. London, England: Springer-Verlag; 2010.
Kiyota  Y, Schneeweiss  S, Glynn  RJ, Cannuscio  CC, Avorn  J, Solomon  DH.  Accuracy of Medicare claims-based diagnosis of acute myocardial infarction: estimating positive predictive value on the basis of review of hospital records. Am Heart J. 2004;148(1):99-104.
PubMed   |  Link to Article
Kokotailo  RA, Hill  MD.  Coding of stroke and stroke risk factors using International Classification of Diseases, revisions 9 and 10. Stroke. 2005;36(8):1776-1781.
PubMed   |  Link to Article
Tirschwell  DL, Longstreth  WT  Jr.  Validating administrative data in stroke research. Stroke. 2002;33(10):2465-2470.
PubMed   |  Link to Article
Colette  D. Modeling Survival Data in Medical Research. London, England: Chapman & Hall; 1994.
Lemeshow  S, Hosmer  DW  Jr.  A review of goodness of fit statistics for use in the development of logistic regression models. Am J Epidemiol. 1982;115(1):92-106.
PubMed
Harrell  FE  Jr, Lee  KL, Mark  DB.  Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15(4):361-387.
PubMed   |  Link to Article
Pencina  MJ, D’Agostino  RB.  Overall C as a measure of discrimination in survival analysis: model specific population value and confidence interval estimation. Stat Med. 2004;23(13):2109-2123.
PubMed   |  Link to Article
Hosmer  DW  Jr, Lemeshow  S. Applied Logistic Regression. 2nd ed. New York, NY: John Wiley & Sons; 2000.
Ridker  PM, Cook  NR.  Statins: new American guidelines for prevention of cardiovascular disease. Lancet. 2013;382(9907):1762-1765.
PubMed   |  Link to Article
Hlatky  MA, Ray  RM, Burwen  DR,  et al.  Use of Medicare data to identify coronary heart disease outcomes in the Women’s Health Initiative. Circ Cardiovasc Qual Outcomes. 2014;7(1):157-162.
PubMed   |  Link to Article
Muntner  P, Levitan  EB, Brown  TM,  et al.  Trends in the prevalence, awareness, treatment and control of high low density lipoprotein-cholesterol among United States adults from 1999-2000 through 2009-2010. Am J Cardiol. 2013;112(5):664-670.
PubMed   |  Link to Article
D’Agostino  RB  Sr, Grundy  S, Sullivan  LM, Wilson  P; CHD Risk Prediction Group.  Validation of the Framingham coronary heart disease prediction scores: results of a multiple ethnic groups investigation. JAMA. 2001;286(2):180-187.
PubMed   |  Link to Article

Figures

Place holder to copy figure label and caption
Figure.
Observed and Predicted Atherosclerotic Cardiovascular Disease Risk Among REGARDS Participants

Predicted risk determined using the Pooled Cohort equations. LDL-C indicates low-density lipoprotein cholesterol; REGARDS, Reasons for Geographic and Racial Differences in Stroke.aThe range of predicted risk for each decile is provided in eTable 3 in Supplement.bThe range of predicted risk for each decile in the REGARDS population with Medicare insurance coverage is provided in eTable 6 in Supplement.cMedicare data are not presented due to a small sample size.

Graphic Jump Location

Tables

Table Graphic Jump LocationTable 1.  Baseline Characteristics of REGARDS Study Participants
Table Graphic Jump LocationTable 2.  Observed and Predicted Incidence Rates of Atherosclerotic Cardiovascular Disease in the REGARDS Study by 10-Year Predicted Risk
Table Graphic Jump LocationTable 3.  Observed and Predicted Incidence Rates of Atherosclerotic Cardiovascular Disease Among REGARDS Study Participants Without Diabetes, With Low-Density Lipoprotein Cholesterol of 70 to 189 mg/dL, and Who Were Not Taking Statins by 10-Year Predicted Riska
Table Graphic Jump LocationTable 4.  Observed and Predicted Incidence Rates of Atherosclerotic Cardiovascular Disease Among REGARDS Study Participants Linked by Medicare

References

Goff  DC  Jr, Lloyd-Jones  DM, Bennett  G,  et al.  2013 ACC/AHA Guideline on the Assessment of Cardiovascular Risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines [published November 12, 2013]. J Am Coll Cardiol. doi:10.1016/j.jacc.2013.1011.1005.
PubMed
Stone  NJ, Robinson  J, Lichtenstein  AH,  et al.  2013 ACC/AHA guideline on the treatment of blood cholesterol to reduce atherosclerotic cardiovascular risk in adults: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines [published online November 7, 2013]. J Am Coll Cardiol. doi:10.1016/j.jacc.2013.11.002.
PubMed
Rosamond  WD, Chambless  LE, Heiss  G,  et al.  Twenty-two-year trends in incidence of myocardial infarction, coronary heart disease mortality, and case fatality in 4 US communities, 1987-2008. Circulation. 2012;125(15):1848-1857.
PubMed   |  Link to Article
Kleindorfer  DO, Khoury  J, Moomaw  CJ,  et al.  Stroke incidence is decreasing in whites but not in blacks: a population-based estimate of temporal trends in stroke incidence from the Greater Cincinnati/Northern Kentucky Stroke Study. Stroke. 2010;41(7):1326-1331.
PubMed   |  Link to Article
Howard  VJ, Cushman  M, Pulley  L,  et al.  The Reasons for Geographic and Racial Differences in Stroke study: objectives and design. Neuroepidemiology. 2005;25(3):135-143.
PubMed   |  Link to Article
Safford  MM, Brown  TM, Muntner  PM,  et al; REGARDS Investigators.  Association of race and sex with risk of incident acute coronary heart disease events. JAMA. 2012;308(17):1768-1774.
PubMed   |  Link to Article
Friedewald  WT, Levy  RI, Fredrickson  DS.  Estimation of the concentration of low-density lipoprotein cholesterol in plasma, without use of the preparative ultracentrifuge. Clin Chem. 1972;18(6):499-502.
PubMed
Fonseca  C, Oliveira  AG, Mota  T,  et al; EPICA Investigators.  Evaluation of the performance and concordance of clinical questionnaires for the diagnosis of heart failure in primary care. Eur J Heart Fail. 2004;6(6):813-822.
PubMed   |  Link to Article
Meschia  JF, Brott  TG, Chukwudelunzu  FE,  et al.  Verifying the stroke-free phenotype by structured telephone interview. Stroke. 2000;31(5):1076-1080.
PubMed   |  Link to Article
Howard  VJ, Kleindorfer  DO, Judd  SE,  et al.  Disparities in stroke incidence contributing to disparities in stroke mortality. Ann Neurol. 2011;69(4):619-627.
PubMed   |  Link to Article
 Stroke—1989: recommendations on stroke prevention, diagnosis, and therapy: report of the WHO Task Force on Stroke and other Cerebrovascular Disorders. Stroke. 1989;20(10):1407-1431.
PubMed   |  Link to Article
Soliman  EZ, Howard  G, Cushman  M,  et al.  Prolongation of QTc and risk of stroke: the REGARDS (REasons for Geographic and Racial Differences in Stroke) study. J Am Coll Cardiol. 2012;59(16):1460-1467.
PubMed   |  Link to Article
Thygesen  K, Alpert  JS, White  HD,  et al; Joint ESC/ACCF/AHA/WHF Task Force for the Redefinition of Myocardial Infarction.  Universal definition of myocardial infarction. Circulation. 2007;116(22):2634-2653.
PubMed   |  Link to Article
Luepker  RV, Apple  FS, Christenson  RH,  et al; AHA Council on Epidemiology and Prevention; AHA Statistics Committee; World Heart Federation Council on Epidemiology and Prevention; European Society of Cardiology Working Group on Epidemiology and Prevention; Centers for Disease Control and Prevention; National Heart, Lung, and Blood Institute.  Case definitions for acute coronary heart disease in epidemiology and clinical research studies. Circulation. 2003;108(20):2543-2549.
PubMed   |  Link to Article
Prineas  RJ, Crow  RS, Blackburn  H. The Minnesota Code Manual of Electrocardiographic Findings: Standards and Procedures for Measurement and Classification. Boston, MA: Wright-OSG; 1982.
Prineas  RJ, Crow  RS, Zhang  ZM. Minnesota Code Manual of Electrocardiographic Findings. 2nd ed. London, England: Springer-Verlag; 2010.
Kiyota  Y, Schneeweiss  S, Glynn  RJ, Cannuscio  CC, Avorn  J, Solomon  DH.  Accuracy of Medicare claims-based diagnosis of acute myocardial infarction: estimating positive predictive value on the basis of review of hospital records. Am Heart J. 2004;148(1):99-104.
PubMed   |  Link to Article
Kokotailo  RA, Hill  MD.  Coding of stroke and stroke risk factors using International Classification of Diseases, revisions 9 and 10. Stroke. 2005;36(8):1776-1781.
PubMed   |  Link to Article
Tirschwell  DL, Longstreth  WT  Jr.  Validating administrative data in stroke research. Stroke. 2002;33(10):2465-2470.
PubMed   |  Link to Article
Colette  D. Modeling Survival Data in Medical Research. London, England: Chapman & Hall; 1994.
Lemeshow  S, Hosmer  DW  Jr.  A review of goodness of fit statistics for use in the development of logistic regression models. Am J Epidemiol. 1982;115(1):92-106.
PubMed
Harrell  FE  Jr, Lee  KL, Mark  DB.  Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15(4):361-387.
PubMed   |  Link to Article
Pencina  MJ, D’Agostino  RB.  Overall C as a measure of discrimination in survival analysis: model specific population value and confidence interval estimation. Stat Med. 2004;23(13):2109-2123.
PubMed   |  Link to Article
Hosmer  DW  Jr, Lemeshow  S. Applied Logistic Regression. 2nd ed. New York, NY: John Wiley & Sons; 2000.
Ridker  PM, Cook  NR.  Statins: new American guidelines for prevention of cardiovascular disease. Lancet. 2013;382(9907):1762-1765.
PubMed   |  Link to Article
Hlatky  MA, Ray  RM, Burwen  DR,  et al.  Use of Medicare data to identify coronary heart disease outcomes in the Women’s Health Initiative. Circ Cardiovasc Qual Outcomes. 2014;7(1):157-162.
PubMed   |  Link to Article
Muntner  P, Levitan  EB, Brown  TM,  et al.  Trends in the prevalence, awareness, treatment and control of high low density lipoprotein-cholesterol among United States adults from 1999-2000 through 2009-2010. Am J Cardiol. 2013;112(5):664-670.
PubMed   |  Link to Article
D’Agostino  RB  Sr, Grundy  S, Sullivan  LM, Wilson  P; CHD Risk Prediction Group.  Validation of the Framingham coronary heart disease prediction scores: results of a multiple ethnic groups investigation. JAMA. 2001;286(2):180-187.
PubMed   |  Link to Article

Letters

CME
Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.

Multimedia

Supplement.

eTable 1. Estimation of race- and gender-specific ASCVD risk using the ASCVD Pooled Cohort risk equations

eTable 2. Baseline characteristics of the REGARDS population included in the study

eTable 3. Observed and predicted atherosclerotic cardiovascular disease events in the REasons for Geographic And Racial Differences in Stroke (REGARDS) study by decile of risk, overall (n=18,498, top panel), and among participants without diabetes, with LDL-C 70-189 mg/dL and not taking statins (n=10,997, bottom panel)

eTable 4. Observed and predicted incidence rates of atherosclerotic cardiovascular disease in the REasons for Geographic And Racial Differences in Stroke (REGARDS) study by 10-year predicted risk, overall population (top panel), and among participants without diabetes, with LDL-C 70-189 mg/dL and not taking statins (bottom panel)

eTable 5. Baseline characteristics of REGARDS participants linked to Medicare claims according to 10-year predicted atherosclerotic cardiovascular disease risk, overall (n=6,121, top panel), and among participants without diabetes, with LDL-C 70 to 189 mg/dL and who were not taking statins (n=3,333, bottom panel)

eTable 6. Observed and predicted ASCVD events in the REasons for Geographic And Racial Differences in Stroke (REGARDS) study participants linked to Medicare claims by decile of risk, overall (n=6,121, top panel), and among participants without diabetes, with LDL-C 70-189 mg/dL and not taking statins (n=3,333, bottom panel)

eFigure. Inclusion criteria applied to the REasons for Geographic and Racial Differences in Stroke (REGARDS) study for the validation of the Pooled Cohort risk equations

References

Supplemental Content

Some tools below are only available to our subscribers or users with an online account.

Web of Science® Times Cited: 9

Related Content

Customize your page view by dragging & repositioning the boxes below.

See Also...
Articles Related By Topic
Related Collections
PubMed Articles
JAMAevidence.com

Users' Guides to the Medical Literature
Clinical Scenario

Users' Guides to the Medical Literature
In contrast, the authors of the Clopidogrel Versus Aspirin in Patients at Risk of Ischaemic...