Risk Stratification for In-Hospital Mortality in Acutely Decompensated Heart Failure:  Classification and Regression Tree Analysis FREE

Heart failure causes considerable morbidity and mortality and is responsible for a tremendous burden on the health care system in the United States.¹ It accounted for approximately 1 million hospital discharges in 2001, an increase of 164% since 1979, and is associated with an overall annual cost of nearly $29 billion.¹ Reported in-hospital mortality ranges from as low as 2.3% among patients enrolled in clinical trials to 19% in referral hospital series.^2- 3

Despite this dramatic increase in the public health burden of hospitalization for heart failure, models for the risk stratification of patients during admission for acute decompensated heart failure (ADHF) are not well established. Clinical risk prediction tools may be helpful in guiding medical decision making. Patients estimated to be at a lower risk may be managed with less intensive monitoring and therapies available on a telemetry unit or hospital ward, whereas a patient estimated to be at a higher risk may require more intensive management in an intensive or coronary care unit. Previous studies, generally based on outpatients with chronic heart failure, have identified a number of variables that are associated with increased mortality, including etiology,⁴ patient age,⁵ clinical assessment at the time of hospitalization,⁴ cardiothoracic ratio,^5- 6 peak oxygen consumption,⁶ left ventricular ejection fraction,⁶ serum sodium concentration,⁵ serum creatinine concentration,^4- 5 and B-type natriuretic peptide concentration.⁷

In contrast, several factors have limited the development of similar models in patients with ADHF. Lack of a consistent definition of ADHF, different nomenclature to describe its clinical features, incomplete data available in administrative data sets, and varying statistical methods have hindered the development of risk stratification tools.^8- 10 Consequently, unlike acute coronary syndromes, in which several systems have been developed for risk stratification,^11- 17 no clinically practical method of risk stratification exists for patients hospitalized with ADHF.

The objective of the present analysis was to develop and validate a practical and user-friendly method of risk stratification for in-hospital mortality among patients admitted with ADHF that could be applicable to routine clinical practice. Data used to model risk were taken from the Acute Decompensated Heart Failure National Registry (ADHERE).^18- 19 This registry collects detailed hospitalization data from initial presentation in the hospital or emergency department until discharge, transfer, or in-hospital death.¹⁹ As an observational database, these data reflect current real-world treatment patterns and in-hospital clinical outcomes for patients hospitalized with ADHF.

The ADHERE registry contains data on patients hospitalized with ADHF in 263 community, tertiary, and academic centers from all regions of the United States.²⁰ For the purpose of the registry, ADHF is defined as new-onset decompensated heart failure or decompensation of chronic, established heart failure with symptoms sufficient to warrant hospitalization. The design, methods, and patient characteristics in the ADHERE registry have been described previously.²⁰ Briefly, medical records are reviewed by trained abstractors at participating study sites and data from consecutive eligible male and female patients aged 18 years or older at the time of hospitalization are entered into the registry using an electronic case report form incorporating real-time validity checking.^19- 20 These data include demographic information, medical history, baseline clinical characteristics, initial evaluation, treatment received, procedures performed, hospital course, and patient disposition.^18- 20 Standardized definitions are used for all patient-related variables, clinical diagnoses, and hospital outcomes.²¹

Institutional review board approval is required for all participating centers; however, informed consent of individuals is not required for registry entry.^19- 20 To preserve patient confidentiality, direct patient identifiers are not collected. Data are reported only in aggregate format. Therefore, registry entries reflect individual hospitalization episodes, not individual patients, and multiple hospitalizations of the same patient may be entered into the registry as separate records.

The current analysis is based on the initial 65 275 patient-records entered into the registry. For this analysis, predictors of in-hospital mortality were determined from an initial derivation cohort consisting of data from October 2001 to February 2003 (33 046 hospitalizations). These data were subjected to classification and regression tree (CART) analysis to identify the best predictors of in-hospital mortality and develop the risk stratification model. The validity of this model was then independently assessed using data from the second validation cohort, consisting of the subsequent 32 229 hospitalization episodes (March 2003 to July 2003).

Admission and/or medical staff recorded race/ethnicity, usually as the patient was registered, using hospital-defined race/ethnicity. Patients were assigned to only 1 race/ethnicity category. Prior studies in patients hospitalized with heart failure have suggested different mortality risk based on race/ethnicity. Race/ethnicity was also a significant univariate predictor of in-hospital mortality in the ADHERE derivation cohort. Race/ethnicity was thus included as one of the 39 variables for CART and logistic regression analysis.

The CART method is an empirical, statistical technique based on recursive partitioning analysis.^22- 24 Unlike multivariable logistic regression, it is well suited to the generation of clinical decision rules.^23- 24 Furthermore, because it does not require parametric assumptions, it can handle numerical data that are highly skewed or multimodal and categorical predictors with either an ordinal or nonordinal structure.^23- 24 The CART method involves the segregation of different values of classification variables through a decision tree composed of progressive binary splits. Every value of each predictor variable is considered as a potential split, and the optimal split is selected based on impurity criterion (the reduction in the residual sum of squares due to a binary split of the data at that tree node). When missing values are encountered in considering a split, they are ignored and the probability and impurity measures are calculated from the nonmissing values of that variable. Each parent node in the decision tree produces 2 child nodes, which in turn can become parent nodes producing additional child nodes. This process continues with both tree building and pruning until statistical analysis indicates that the tree fits without overfitting the information contained in the data set.²³ As a result, CART analysis produces decision trees that are simple to interpret and may be applied at the bedside.

An open-source adaptation of the CART algorithm (tree library in S-PLUS, version 6.0, Insightful Corp, Seattle, Wash) was used to analyze 39 potential clinical variables of interest in the derivation cohort (Table 1). These variables were selected from 80 variables collected in the ADHERE registry by virtue of predicting in-hospital mortality on univariate analysis or having been identified in previous published studies as risk factors for mortality. Nodes in the CART tree were constrained to have a minimum size of 800 records in parent nodes and 400 records in final child nodes. A 10-fold cross-validation was used to assess the predictive ability of the tree model. Mortality was calculated for each of the terminal nodes in the CART tree and used to generate the risk stratification model. The predictive value of this model was then assessed by determination of mortality odds ratios (ORs) and 95% confidence intervals (CIs) between risk groups.

The ability of the derived risk tree to identify ADHF patients at low, intermediate, and high risk for in-hospital mortality was tested. The patients from the validation cohort were classified into risk groups based on the CART tree. Mortality for these risk groups and the mortality ORs and 95% CIs between risk groups were determined and these data were compared with those of the derivation cohort.

Finally, a multivariate logistic regression model was constructed from the derivation cohort (logistic procedure in SAS version 8.2, SAS Institute Inc, Cary, NC), tested in the validation cohort, and the accuracy of the CART and logistic regression models was compared using area under receiver operating characteristic curves in the derivation and validation cohorts.

Baseline characteristics and main outcomes of the 33 046 hospitalization episodes used to develop the model (derivation cohort) and the 32 229 hospitalization episodes used to test the model (validation cohort) are shown in Table 2 and Table 3. The derivation and validation cohorts were similar with respect to age at admission; sex distribution; heart failure history; medical history; and clinical symptoms, vital signs, and laboratory values (Table 2). Clinical outcomes were also similar between the 2 cohorts (Table 3).

Of the 39 variables evaluated, the CART method identified blood urea nitrogen (BUN) level of 43 mg/dL or higher (≥15.35 mmol/L) at admission as the best single discriminator between hospital survivors and nonsurvivors. The next best predictor of in-hospital mortality in both the higher and lower BUN nodes was systolic blood pressure (SBP) at a discrimination level of less than 115 mm Hg. For the node with patients having a BUN level of 43 mg/dL or higher (≥15.35 mmol/L) and SBP of less than 115 mm Hg, a serum creatinine level of 2.75 mg/dL or higher (≥243.1 μmol/L) provided additional prognostic value. Figure 1 depicts the final tree generated by the CART analysis along with the mortality data for each child node of this tree. These branch points permit patient stratification into 5 risk groups: high risk (BUN level ≥43 mg/dL [≥15.35 mmol/L], SBP <115 mm Hg, and creatinine level ≥2.75 mg/dL [≥243.1 μmol/L]), intermediate risk 1 (BUN level ≥43 mg/dL [≥15.35 mmol/L], SBP <115 mm Hg, and creatinine level <2.75 mg/dL [<243.1 μmol/L]), intermediate risk 2 (BUN level ≥43 mg/dL [≥15.35 mmol/L] and SBP ≥115 mm Hg), intermediate risk 3 (BUN level <43 mg/dL [<15.35 mmol/L] and SBP <115 mm Hg), and low risk (BUN level <43 mg/dL [<15.35 mmol/L] and SBP ≥115 mm Hg). Table 4 summarizes the clinical characteristics of patients in these 5 risk groups. The mortality OR between the high- and low-risk groups was 12.9 (95% CI, 10.4-15.9), with statistically significant differences in mortality risk detected between all risk groups except intermediate risk groups 2 and 3 (Table 5). Although additional risk nodes involving additional variables could be generated, they offered little incremental risk discrimination.

The decision tree generated by CART analysis of the derivation cohort was tested for its ability to risk stratify patients in the validation cohort. This risk tree was able to stratify patients into high, intermediate, and low risk (Figure 2). The mortality OR between the high- and low-risk groups was 10.4 (95% CI, 8.4-13.0), with statistically significant differences detected between all risk groups except intermediate risk groups 2 and 3 (Table 4). These absolute mortality rates, as well as the clinical characteristics and mortality ORs between risk groups, were similar to those of the derivation cohort (Table 4 and Table 5)and comparable risk stratification occurred when the analysis was limited to the subset of validation patients with new onset heart failure (in-hospital mortality: 23.6% in the high-risk group; 20.0%, intermediate risk group 1; 5.0%, intermediate risk group 2; 5.1%, intermediate risk group 3; and 1.8%, low-risk group).

Multivariate logistic regression identified BUN level, SBP, heart rate, and age as the most significant mortality risk predictors:

log odds of mortality =
0.0212 × BUN − 0.0192 × SBP
+ 0.0131 × heart rate + 0.0288
× age − 4.72.

The addition of 24 predictors did not meaningfully increase the accuracy of this model. Figure 3 compares in-hospital mortality rates in the derivation and validation cohorts based on risk groups determined by logistic regression. Based on the area under the receiver operating characteristic curves, the accuracy of the CART model (derivation cohort: 68.7%; validation cohort: 66.8%) was modestly less than that of the more complicated logistic regression model (derivation cohort: 75.9%; validation cohort: 75.7%).

This analysis of more than 65 000 recent ADHF hospitalizations in patients demographically and clinically similar to those seen in other large community- or Medicare-based evaluations^25- 27 demonstrates that the risk of in-hospital mortality can be reliably estimated using routinely available vital signs and laboratory data obtained on hospital admission. Overall, in-hospital mortality was 4.1%, but this mortality risk varied more than 10-fold (from 2.1% to 21.9%) based on the patient’s initial SBP and levels of BUN and creatinine.

Multiple evaluations of patients hospitalized for heart failure have demonstrated an association between clinical outcomes and indices of renal function and blood pressure.^28- 32 In the Enhanced Feedback for Effective Cardiac Treatment study, increasing BUN levels and decreasing SBP were significant and independent predictors of both 30-day and 1-year mortality.²⁸ In the Outcomes of a Prospective Trial of Intravenous Milrinone for Exacerbations of Chronic Heart Failure study, these same parameters were significant and independent predictors of mortality or rehospitalization.³¹ Similarly, in a retrospective review of 1004 consecutive patients hospitalized for heart failure at 11 geographically diverse hospitals, worsening renal function was associated with a 7.5-fold increase (95% CI, 2.9- to 19.3-fold increase) in the adjusted risk of in-hospital mortality.³² In a prospective analysis of 412 patients hospitalized at a single institution, the 6-month mortality risk increased with decreasing renal function, which was determined by the change in creatinine levels relative to baseline.³⁰ Renal dysfunction causes further congestion and neurohormonal activation, which are factors associated with adverse outcomes in patients with heart failure.³³

In addition to these parameters, other parameters that have been correlated with clinical outcomes in patients hospitalized with heart failure include age^3,28- 29,34 ; sex³⁴ ; heart failure etiology^34- 35 ; history of previous heart failure hospitalizations³¹ ; comorbid conditions such as cerebrovascular disease, dementia, chronic obstructive pulmonary disease, hepatic cirrhosis, and cancer^28,34 ; respiratory rate²⁸ ; anemia^31,36- 37 ; serum sodium concentration²⁸ ; B-type natriuretic peptide levels^38- 39 ; left ventricular ejection fraction²⁹ ; and heart failure therapy.³ Because multiple risk factors can exist in the same patient, risk factor analysis (to be meaningful) must consider factors in combination rather than isolation. Because previous evaluations tended to treat these factors as isolated entities, they have not produced aclinically practical way of integrating various factors to stratify risk in heart failure patients.⁴⁰

Unlike ADHF, several risk stratification schemes already exist for patients with acute coronary syndromes.^11- 17 These schemes are typically based on multivariable analysis using logistic regression or Cox proportional hazards models. Although schemes using as few as 3^13,17 to more than 20 variables are available,¹² an acute coronary syndromes risk scoring scheme generally involves 7 to 10 variables.^11,14- 16 Such clinical prediction models have been interpreted to be helpful for risk stratification and management of acute coronary syndrome patients and have been integrated into national guidelines.¹¹

Although no in-hospital mortality risk stratification scheme is available for patients hospitalized with heart failure, a heart failure survival score has been developed and independently validated for ambulatory outpatients with heart failure.^40- 41 This score, based on 7 variables—heart failure etiology, heart rate, blood pressure, serum sodium concentration, intraventricular conduction time, left ventricular ejection fraction, and peak oxygen consumption—stratifies patients into low (16%), medium (39%), and high (50%) mortality risk categories.⁴⁰ In addition, hospitalization data have been used to develop a risk score for heart failure readmission.⁴² This risk score, which is based on 16 variables, was moderately predictive in a derivation cohort but it has not been independently validated in a second cohort.⁴²

A significant disadvantage of multivariable-generated risk schemes is their complexity. The number of variables and mathematical functions involved frequently require access to a computer or an electronic calculator to generate a score and to determine risk, making them impractical for bedside assessment unless such tools are readily available. Even when converted to point scores, the tools derived from a multivariate model still require a nomogram reference to convert a point score to risk. Similar to multivariate regression techniques, the CART method can detect interactions between variables.²⁴ Moreover, it yields a decision tree that is relatively easy to apply at the bedside, leading to its potential use in a wide variety of clinical conditions, including infections⁴³ and neurological,⁴⁴ oncological,^45- 46 and cardiac⁴⁷ disorders.

In the current evaluation, the CART method identified 3 of 39 potential variables as significant predictors of in-hospital mortality risk. In a simple 2- to 3-step process, these variables permit identification of patients with low, intermediate, or high risk for in-hospital mortality. Furthermore, the accuracy of this model, which can be easily applied at the bedside, is close to that of the more complicated model derived from logistic regression. Alternately, if computer access or a pocket digital assistant is available at the bedside, the model derived from logistic regression may have advantages.

These validated models should aid medical decision making in patients hospitalized with ADHF. Patients judged to be at higher risk may receive higher-level monitoring and earlier, more intensive treatment for ADHF, while patients estimated to be at lower risk may be reassured and managed less intensively. In addition, these models may prove to be valuable in designing clinical trials to evaluate heart failure therapies because they permit risk to be balanced across treatment groups⁴⁵ and allow for selective inclusion criteria to enroll only patients at high risk for in-hospital mortality.

Potential limitations of the current analysis must be acknowledged. Real-world practice information can be both an advantage and a disadvantage of analyses based on registry data. Study results can be influenced by differences in disease assessment, treatment, and documentation patterns at participating institutions. The ADHERE registry reflects patients cared for by thousands of clinicians at hundreds of hospitals across the country and thus has an excellent chance to adjust for this variation and create a risk prediction model that is robust for most situations. However, this model may not apply to patients who are cared for in settings that deviate substantially from those in ADHERE. In addition, each patient’s actual risk may be influenced by many factors not measured or considered in this model. The CART method favors variables available for analysis in the greatest proportion of patients. Some potential risk factors, such as B-type natriuretic peptide, were obtained in less than 25% of patients. Consequently, there may be additional variables that either were not considered or were considered and rejected because of limited data that could ultimately improve the risk discrimination if assessed in a sufficient number of patients.

Therefore, this model enhances, not replaces, physician assessment. Moreover, because the ADHERE registry does not contain specific patient identifiers, information regarding patient status following hospital discharge is not available. Thus the effects of any of the variables evaluated in this analysis on intermediate- and long-term mortality risks cannot be determined. Similarly, because of the lack of patient identifiers, the analyzed cohorts may contain multiple admissions for the same patient. However, this should not have influenced the study results because the outcome parameter, in-hospital mortality, and the identified risk factors (admission SBP and admission levels of BUN and creatinine) are specific to individual hospitalization episodes. Lastly, the derivation and validation cohorts come from periods that differ both temporally and in duration. Nonetheless, these cohorts are similar in size, baseline characteristics, and clinical outcome. Despite these potential limitations, the current CART-based analysis of the ADHERE registry has created a simple robust tool to predict in-hospital mortality that is easy to use and has good discriminative ability.

In patients hospitalized with ADHF, the risk of in-hospital mortality can be quickly and accurately determined using admission clinical and laboratory variables. Of 39 variables, BUN level of 43 mg/dL or higher (≥15.35 mmol/L), serum creatinine level of 2.75 mg/dL or higher (≥243.1 μmol/L), and SBP of less than 115 mm Hg were independent predictors of high risk for in-hospital mortality in the current CART analysis. On the basis of these 3 variables, ADHF patients can be readily stratified into groups at low, intermediate, and high risk for in-hospital mortality, with mortality risks ranging from 2.1% to 21.9%. The finding that indices of renal status are 2 of the 3 predictors providing the best mortality risk discrimination underscores the importance of renal function in ADHF patients. The continued high mortality for patients hospitalized with ADHF provides a compelling indication to apply tools, such as the risk tree derived in this study, to improve the evaluation and, potentially, management and outcomes of these patients.