A Validated Prediction Model for All Forms of Acute Coronary Syndrome:  Estimating the Risk of 6-Month Postdischarge Death in an International Registry FREE

Clinical prediction models may be helpful for medical decision making¹ as patients judged to be at higher risk may receive more aggressive surveillance and/or earlier treatment, while patients estimated to be at lower risk may be reassured and managed less aggressively.^2- 3 By using simple yet valid risk calculations, clinicians can accurately advise patients about their likelihood of an event, and how this likelihood translates into treatment decisions.

The acute coronary syndrome (ACS) encompasses a continuum of conditions ranging from ST-segment elevation myocardial infarction (STEMI) to non–ST-segment elevation myocardial infarction (NSTEMI) and unstable angina.^1,4- 8 Numerous risk-prediction models for differing outcomes exist for the different types of ACS.^5,9 Most models have been developed from large randomized clinical trial populations in which the generalizability to risk prediction in the average clinician's experience is questionable.^6- 10

We were interested in developing a simple risk-prediction tool that would be applicable to all types of ACS, would focus on an important clinical end point of all-cause mortality, and be developed in patients who are similar to those encountered in routine clinical practice.

The Global Registry of Acute Coronary Events (GRACE) is a multinational cooperative effort involving 94 hospitals in 14 countries that has been designed to reflect an unbiased, representative population of patients with ACS. Full details of the GRACE methods have been previously published.^11- 12 For this analysis, patients had to be at least 18 years old and alive at the time of discharge, be admitted for ACS as a presumptive diagnosis (ie, have symptoms consistent with acute myocardial ischemia), and have at least 1 of the following: electrocardiographic changes consistent with ACS, serial increases in serum cardiac biomarkers, and/or documentation of coronary artery disease. The qualifying ACS must not have been precipitated by significant noncardiovascular comorbidity, such as acute anemia or hyperthyroidism. At approximately 6 months after hospital discharge, patients were followed up to ascertain vital status. At each enrolling hospital, study investigators worked with their ethics or institutional review board to obtain appropriate approval to participate.

To ensure enrollment of an unbiased population, the first 10 to 20 consecutive eligible patients were recruited from each site per month. Data were collected by trained coordinators using a standardized case report form. Demographic characteristics, medical history, presenting symptoms, biochemical and electrocardiographic findings, treatment practices, and a variety of hospital outcome data were collected. Standardized definitions for all patient-related variables and clinical diagnoses were used.^11- 12 At discharge, all cases were assigned to STEMI, NSTEMI, or unstable angina categories.^11- 12 Standardized definitions were used for selected hospital complications and outcomes.¹¹

The primary end point was all-cause mortality occurring within 6 months of discharge from hospital. The model was developed by using a multivariable Cox proportional hazards regression backward elimination technique.¹³ The following variables were included in the stepwise Cox proportional hazards regression: baseline characteristics, symptoms and signs at presentation, in-hospital treatments and procedures, and in-hospital complications. A backward stepwise technique evaluated all potential univariate correlates (P<.25) to create a multivariable model containing variables with P<.05. No imputation was performed on the final model. Imputation was tested but did not influence the identification of multivariate predictors or the discriminative power of the model for predicting death.¹⁴ All variables in the final model met the assumptions for proportional hazards.

The model accuracy was tested in several ways. First, we developed the model in all patients with ACS enrolled in GRACE between April 1, 1999, and March 31, 2002. The c statistic was extended to evaluate the discrimination of survival analytic techniques.¹⁵ We checked for any nonlinear relationship between death and each variable in the final model and found none. Selected testing was performed for interactions using the significant prognostic variables from the final model based on interactions that have been reported from other published models. Second, we tested the model in a validation cohort of consecutive patients enrolled in GRACE between April 1, 2002, and December 31, 2003. Statistical analyses were performed using SAS version 8.2 (SAS Institute Inc, Cary, NC) and S-Plus version 3.4 (MathSoft Inc, Seattle, Wash).

The overall follow-up rate in our development cohort was 87.5% for death. A comparison of the patients with and those patients without available follow-up data demonstrated no significant clinical differences in terms of baseline characteristics, symptoms at presentation, in-hospital treatments and procedures, in-hospital outcomes, and postdischarge outcomes.

We developed a bedside prediction tool that can be applied to either a pocket card or a handheld device. Clinical prediction variables were given weighted scores based on the model's variable coefficients. The prediction tool considers the variables taken from the final model assigning a point total to each variable, which allows a total point score for each patient to be calculated. This then is applied to a reference plot nomogram, which shows the corresponding risk of death. Alternatively, the risk of death could be calculated using a handheld device. The clinical application can be found at http://www.outcomes-umassmed.org/grace.

Baseline characteristics, in-hospital treatments, and outcomes of the 15 007 patients used to develop the model (development cohort), 7638 patients used to test the model (validation cohort), and 5116 patients for whom 6-month vital status was unknown are shown in Table 1 and Table 2. The mean age was 65 years and 67% of the cohort were men. Prior or current smoking and hypertension were present as risk factors in more than half of the patients. Approximately 45% of patients had a diagnosis of hyperlipidemia and less than 25% had diabetes mellitus.

The 6-month mortality risk was similar in the development (4.8%, n = 717) and validation cohorts (4.7%, n = 331). Nine multivariate predictors of mortality were identified (Table 3). The model calibrations were retained when testing the model in an independent validation cohort (Table 3 and Figure 1). The model performed well in all forms of ACS with a c statistic of at least 0.70 (Table 4).

Figure 2 illustrates a method to estimate a patient's risk depending on the total score obtained by summing the individual scores for each of the 9 variables in the model. A similar nomogram can be programmed into a handheld device to make the risk calculation automatic once the individual variables have been entered.

We have developed and validated a simple bedside prediction tool that can be used to estimate a patient's postdischarge risk of 6-month mortality in all forms of ACS, regardless of their initial electrocardiogram or biomarker results. By using the power of the GRACE registry (>25 000 patients studied) and focusing on the clinically relevant and wholly unbiased end point of all-cause mortality, we believe that clinicians may find this tool usable and reliable as they attempt to make key diagnostic and treatment decisions among patients hospitalized with ACS.

Previous risk tools have been proposed in estimating in-hospital risk after admission for ACS,^1,4,6- 8 including risk models developed from large clinical trials or registry data by the Thrombolysis in Myocardial Infarction (TIMI) clinical trials group.^4,9 These models are robust in predicting risk for specific end points and in the population in which they have been studied. However, comparison between our methods and those of the TIMI group elucidate the issue of using prediction models in practice. For example, many ACS trials, including the TIMI trial, have used a combined end point that includes recurrent coronary ischemia with a secondary end point of ischemia requiring coronary revascularization. This particular end point is potentially troublesome because it is influenced by local practice style in which the availability of a catheterization laboratory may have more to do with a decision to revascularize than with patient characteristics.^16- 17 It may also lead to inaccuracies in overall prediction. For instance, prior coronary artery bypass graft surgery predicts an increased risk of death after percutaneous coronary intervention but a decreased likelihood of surgical coronary revascularization in patients undergoing coronary angioplasty.¹⁸ This is because prior coronary artery bypass graft surgery is associated with a worse complication rate after reoperative coronary artery bypass graft surgery, so surgeons may try to avoid it where possible. In ACS, studies suggest that at the extremes of older age or serum creatinine clearance, interventionalists reduce their use of percutaneous coronary intervention for fear of complications, while at the same time the risk of death steadily increases.¹⁹ Finally, it is not clear that revascularization is an event that should be avoided. Given the benefits of an early invasive strategy, revascularization may be considered a good event, and in our analysis was a predictor of improved survival at 6-months postdischarge.

Another limitation of prior ACS prediction models relates to the arbitrary separation into STEMI and non–ST-segment elevation ACS populations. Because few models incorporate the entire spectrum of patients with ACS, physicians wishing to apply these models in routine practice need to be aware of multiple risk scores for different types of patients with ACS. In the GRACE model, we were able to demonstrate that diagnostic prediction for in-hospital mortality is similar whether the ST-segment deviation is elevation or depression.¹⁴ In this analysis, we extended that observation to show that one can robustly predict 6-month mortality, regardless of whether the patient presents with STEMI, NSTEMI, or unstable angina. The overall c statistic is high (0.81) compared with other risk models for combined end points such as the TIMI risk model (0.65).⁹

How generalizable are prediction tools? They are as generalizable as the population from which they are derived. In this way, we believe that the GRACE model is unique. In so far as we have studied consecutive patients, observed in care at 94 hospitals, representing 14 countries, and have used a population-based enrollment wherever possible, the GRACE registry model may well be closer to real-world practice than previous studies limited to clinical trial populations or single-region registries. As such, we believe that clinicians will find greater confidence in its applicability in their practices. Because it has been validated on patients enrolled in 2002 and 2003, clinicians can also feel confident that it is current.

GRACE is the largest multinational registry study to include the entire spectrum of patients with ACS. It is designed to be representative of regional communities and uses standardized criteria for defining ACS and hospital outcomes and quality control and audit measures. Given the overall number of deaths in the development (n = 717) and validation (n = 331) cohorts, one has increased confidence of the robustness of the model.

Risk-prediction tools are developed in populations, not individuals. Although the risk-prediction tools may inform practitioners regarding the estimated likelihood of complications in a patient, each individual patient is unique and subject to many influences not measured or ascertained in clinical practice. Local practice referral patterns or practice style may influence average risks of patients being cared for by an individual clinician. To the extent that GRACE studies patients cared for all over the world by hundreds of clinicians, it has an excellent chance to adjust for this variation to create a risk-prediction instrument that is robust for most situations. However, like individual patients, each practice has its set of unmeasured variation that may influence risks, such as socioeconomic status, which was not considered in the current model. The model is applicable to patients hospitalized with ACS and discharged alive, not patients being observed in an emergency department.

In summary, our GRACE 6-month postdischarge prediction model is a simple robust tool for predicting death in patients with ACS and has very good discriminative ability. We believe that clinicians will find it simple to use and applicable to clinical practice.