Association Between Performance Measures and Clinical Outcomes for Patients Hospitalized With Heart Failure FREE

Heart failure continues to be a serious public health concern in the United States: the overall prevalence of heart failure was 5 million individuals in 2003, with 550 000 new cases being reported each year.¹ Heart failure is the leading cause of hospitalization in persons older than 65 years,² with almost 3.6 million hospitalizations attributed to heart failure as the primary or a secondary discharge diagnosis each year.³ The estimated direct and indirect US costs of heart failure in 2006 are expected to be $29.6 billion.¹ Because heart failure is a substantial cause of morbidity, mortality, and health care expenditures, it is especially important to utilize evidence-based therapies that have been demonstrated to improve clinical outcomes.^4- 5

The American College of Cardiology and the American Heart Association (ACC/AHA) have developed clinical practice guidelines outlining diagnostic and therapeutic interventions for patients with heart failure.⁶ The evidence in favor of implementing certain recommendations is particularly strong, and provision of such interventions is thought to be critical in achieving optimal patient outcomes. Adherence to these suggested interventions may therefore serve as a marker of quality of care and form a foundation for quality improvement. Heart failure performance measures have thus been developed to provide a mechanism through which the quality of medical care can be measured and improved. The ACC/AHA and other organizations have developed processes and criteria to identify performance measures with validity, reliability, and feasibility.⁷ These performance measures are based on clinical practice guidelines but are intended to be confined to structural aspects or processes of care for which evidence is so strong that the failure to perform them reduces the likelihood of optimal patient outcomes.⁴

The recently released ACC/AHA Clinical Performance Measures for Adults With Chronic Heart Failure⁴ includes the following heart failure inpatient performance measures: discharge instructions, evaluation of left ventricular systolic function, angiotensin-converting enzyme (ACE) inhibitor or angiotensin receptor blocker (ARB) for left ventricular systolic dysfunction, adult smoking cessation advice/counseling, and anticoagulant at discharge for patients with atrial fibrillation. These performance measures are similar to those advanced by the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) in the 4 JCAHO ORYX core measure sets⁸ and incorporate elements of diagnosis, patient education, treatment, and self-management.

Although it is expected that application of these carefully developed heart failure inpatient performance measures should result in substantial improvement in heart failure patient outcomes,⁴ available data validating these measures are limited.^9- 10 These performance measures are being used and publicly reported by governmental agencies such as the Centers for Medicare & Medicaid Services (CMS) and accreditation organizations such as JCAHO,¹¹ as well as major insurers, and are often the criteria of pay-for-performance demonstration projects.¹² It is therefore critical that each performance measure be currently valid and that there be a link between the processes being measured and relevant clinical outcomes.⁷

The Organized Program to Initiate Lifesaving Treatment in Hospitalized Patients With Heart Failure (OPTIMIZE-HF) registry provides an opportunity to examine the application of heart failure performance measures in a real-world setting. This study assessed the relationship between current ACC/AHA performance measures and relevant patient clinical outcomes, including 60- to 90-day mortality and a composite end point of mortality or rehospitalization rates. In addition, we evaluated the process-outcome link of a potential performance measure for β-blockade at hospital discharge among eligible patients without contraindications.

OPTIMIZE-HF is a registry and performance improvement initiative to enhance guideline adherence in patients hospitalized with heart failure. The overall OPTIMIZE-HF program objectives have been previously described.^13- 14 The OPTIMIZE-HF registry gathered data on patient characteristics along with key aspects of in-hospital and discharge heart failure management through the use of a Web-based case report form. Follow-up data (60-90 days postdischarge) pertaining to survival, readmissions, and medical regimen were prospectively collected for a prespecified subgroup (10%) of patients between March 2003 and December 2004 from 91 hospitals demographically similar to hospitals participating in OPTIMIZE-HF. Patients were reassessed in a time range of 60 to 90 days after hospital discharge to facilitate scheduling and the likelihood of obtaining complete follow-up.

Admission staff, medical staff, or both recorded race/ethnicity, usually as the patient was registered. Prior studies among patients hospitalized with heart failure have suggested differences in characteristics and outcomes based on race/ethnicity.

To be eligible for the OPTIMIZE-HF registry, patients had to be adults hospitalized with an episode of worsening heart failure as the primary reason for admission or with significant heart failure symptoms that developed during hospitalization in which heart failure was the primary discharge diagnosis. Participating hospitals supplied data on eligible admissions according to established JCAHO methods. The protocol was approved by each participating center's institutional review board or through use of a central institutional review board. Written informed consent was obtained prior to enrollment from patients who participated in the follow-up data collection. The registry data coordinating center was Outcome Sciences Inc (Cambridge, Mass).

The ACC/AHA heart failure performance measures and the inpatient measure descriptions include the following:

Performance measures were constructed using the numerator and denominator definitions defined by the ACC/AHA Clinical Performance Measures for Adults With Chronic Heart Failure⁴ and JCAHO ORYX specifications⁸ assessing use rates among eligible patients without documented contraindications, intolerance, or other physician documentation. Performance measures for complete discharge instructions, smoking cessation counseling, and anticoagulation for atrial fibrillation apply to eligible patients irrespective of left ventricular function. Patients with heart failure who had left ventricular function assessed and left ventricular ejection fraction of less than 40% or moderate to severe or severe systolic dysfunction were included for the ACE inhibitor/ARB performance measure. An additional performance measure for β-blockers was developed using a population definition similar to that used for ACE inhibitor/ARB, excluding patients with documented β-blocker contraindications or intolerance.

All statistical analyses were conducted independently by the Duke Clinical Research Institute, Durham, NC, and only eligible patients were included in each of these analyses. Data are reported as means and standard deviations for continuous variables or percentages of nonmissing values for categorical variables. Patient characteristics and treatments were compared using Pearson χ² test for categorical variables and Wilcoxon test for continuous variables. To adjust for significant covariates, multivariable models were developed for postdischarge all-cause mortality and composite mortality/rehospitalization. Cox proportional hazard modeling was applied to all-cause mortality. Logistic regression was used for the composite end point. Appropriate transformations of continuous measures were made when the linearity assumption was violated. Variable selection techniques included backward, forward-stepwise, and bootstrapping of 100 replicated samples. A P value of .05 was used for both entry into and retention in the model during the selection process. The variables that were common in each and chosen in at least 75% of the replicated samples were retained. The final models were also bootstrapped to obtain an estimate of the c statistic after accounting for the overfitting of testing and creating the models on the same population.

Propensity score analysis was used to account for performance measure selection bias when assessing the association. The variables selected for the score were applied to a logistic regression model with the probability of receiving the performance measure generated as the score. The c statistics for these scores ranged from 0.63 to 0.79. Therefore, additional adjustment through inclusion of covariates was applied to ensure that biases were accounted for as thoroughly as possible. Generalized estimating equations were used to account for the correlation of data within the same hospital in the adjusted models. For a treatment difference of 25% or greater, we would have at least 83% to 99% power for 4 of the 5 measures. The measure for smoking cessation exhibited only 62% power. SAS statistical software, version 8.2 (SAS Institute Inc, Cary, NC) was used for all statistical analyses.

OPTIMIZE-HF enrolled patients hospitalized for heart failure at 259 academic and community hospitals of varying size in the United States. The characteristics of patients (N = 5791) and the 91 hospitals in the follow-up cohort were similar to the overall OPTIMIZE-HF population (N = 48 612) (Table 1). The average age of the 60- to 90-day follow-up cohort was 72 years, 51% were male, 78% of the patients were white, and almost a fifth were African American. Forty-two percent of all patients had an ischemic etiology for heart failure and 43% of patients had diabetes mellitus. Of the 5056 patients (87%) with left ventricular function assessed, 53.2% had left ventricular systolic dysfunction with a mean left ventricular ejection fraction of 36.9%.

Both mean and median length of hospital stay were comparable between the overall population and the 60- to 90-day follow-up cohort (Table 1). The rate of in-hospital mortality was greater for the overall population than for the 60- to 90-day follow-up cohort (3.8% vs 1.6%), reflecting the requirement for informed consent before inclusion in the registry at these hospitals. For patients in the follow-up cohort, 29.6% were readmitted during the 60- to 90-day follow-up period, with a mortality rate of 8.6%. The combined rate of readmission or mortality during the follow-up period was 36.2%.

The number of eligible patients and conformity rates for each performance measure in the overall OPTIMIZE-HF population and the 60- to 90-day follow-up cohort are listed in Table 2. Slightly more than half (54%) of all patients in the overall population and 66% in the 60- to 90-day follow-up cohort received complete discharge instructions. Left ventricular systolic function was evaluated in 89% of the follow-up cohort, with 83% of patients in both populations prescribed an ACE inhibitor or ARB at discharge for left ventricular systolic dysfunction. Only 53% of patients were prescribed the anticoagulant warfarin at discharge for atrial fibrillation. The percentage of patients in the overall population receiving smoking cessation counseling was lower than the percentage receiving counseling in the 60- to 90-day follow-up cohort (62% vs 72%). The percentage of eligible patients receiving prescriptions for β-blockers at discharge was high at 83% and 84% in the overall population and follow-up cohort, respectively, and comparable with the percentage of patients prescribed an ACE inhibitor or ARB.

The conformity rates for the performance measures in patients with adverse clinical events during the first 60 to 90 days of follow-up after hospital discharge are shown in Table 3. Patients who survived were significantly more likely to have evaluation of left ventricular systolic function (P = .02) and prescription of an ACE inhibitor/ARB for left ventricular systolic dysfunction (P<.001). Conformity with performance measures for ACE inhibitors/ARBs for left ventricular systolic dysfunction and smoking cessation counseling were significantly more likely in patients who survived or did not require rehospitalization after discharge (P<.001 and P = .01, respectively). Prescription of a β-blocker at discharge was significantly more likely in patients who survived and in patients who both survived and were not rehospitalized (P<.001 and P = .003, respectively).

Unadjusted hazard ratios and odds ratios as a function of performance measures for patients in the 60- to 90-day follow-up cohort are shown in Table 4. Of the 5 ACC/AHA heart failure performance measures, 2 were predictive of both mortality and mortality/rehospitalization in the unadjusted analysis: prescription of an ACE inhibitor/ARB for left ventricular systolic dysfunction and smoking cessation counseling. The 3 other ACC/AHA performance measures were not significant predictors. However, prescription of a β-blocker at discharge was a significant predictor of both mortality and mortality/rehospitalization.

Following risk and propensity adjustment, none of the current ACC/AHA performance measures was a significant independent predictor of mortality in the first 60 to 90 days after hospital discharge (Table 4). Conformity with the performance measure for ACE inhibitors/ARBs for LVSD was found to be a significant predictor of reduced risk for mortality/rehospitalization (odds ratio, 0.51; 95% confidence interval [CI], 0.34-0.78; P = .002). In contrast with the current ACC/AHA performance measures, conformity with a β-blocker performance measure demonstrated a strong process-outcome link. Prescription of a β-blocker in eligible patients was highly predictive of improved postdischarge survival (hazard ratio, 0.48; 95% CI, 0.30-0.79; P = .004). Prescription of a β-blocker in eligible patients was also an independent predictor of a lower risk of mortality/rehospitalization (odds ratio, 0.73; 95% CI, 0.55-0.96; P = .02).

OPTIMIZE-HF provides an important opportunity to evaluate whether and to what degree conformity with the current ACC/AHA performance measures influences early clinical outcomes in a contemporary cohort of patients hospitalized with heart failure. This registry included patients from a wide variety of academic and nonacademic hospitals from all regions of the United States, containing far more detailed information on patient characteristics, presenting symptoms, treatments, and outcomes than has been available in prior administrative data sets or registries. In this detailed analysis, none of the current recommended ACC/AHA heart failure performance measures was strongly associated with 60- to 90-day postdischarge mortality, and only the ACE inhibitor/ARB performance measure strongly influenced postdischarge mortality or rehospitalization. These findings may have significant clinical and public health implications and suggest that additional measures may be required to more effectively quantify the quality of care provided to heart failure patients in the hospital setting.

The hospital discharge period has been a focus of heart failure guidelines because of the ease of access to patients; a favorable patient disposition to adopt health care recommendations; and the opportunity to implement, manage, and measure intervention strategies.¹⁵ Despite this focus, disparities between recommended treatment and actual care received have been reported.^16- 18 To facilitate the measurement of and improvement in quality of care in heart failure, JCAHO and the ACC/AHA developed evidence-based, standardized performance measures designed to quantify adherence to guidelines.^4,8

Explicit criteria exist for the development of performance measures, including quantifying the numerators and denominators and evaluating the interpretability, applicability, and feasibility of the proposed measures so that they accurately reflect the quality of care.⁷ The performance measures are intended to be confined to those structural aspects or processes of care for which the evidence is so strong that the failure to perform them reduces the likelihood of optimal patient outcomes.^4,7 In developing the ACC/AHA heart failure performance measures, a writing group selected potential measures based on preexisting measures and the ACC/AHA 2005 heart failure guideline update, then rated them on 13 dimensions using 5-point Likert scales, ultimately selecting 5 inpatient measures for heart failure.⁴ The final ACC/AHA inpatient measurement set included all 4 JCAHO heart failure measures, which have already been applied to assessment of hospitals nationwide.

Implicit in the stated goals for the development, selection, and implementation of performance measures is that there should be a strong process-outcome link for each measure. Performance measures for acute myocardial infarction have been well validated, with good process-outcome links for most measures in some but not all studies.^19- 22 However, the extent to which conformity with the current ACC/AHA heart failure performance measures can affect quality of care in contemporary patients hospitalized with heart failure has not been well studied. As these exact measures are the criteria on which heart failure quality of care is being publicly reported at the hospital level by the CMS and JCAHO and are the focus of performance improvement programs and pay-for-performance demonstration projects, it is essential that each measure be valid and that there be a link between each process being assessed and relevant clinical outcomes.¹²

Effectiveness of ACE inhibitor or ARB use at discharge has been examined as a process-outcome link in a study involving 17 456 elderly heart failure patients with left ventricular systolic dysfunction.²³ Discharge prescription of either an ACE inhibitor or an ARB was associated with a 17% relative reduction in 1-year postdischarge mortality after adjustment (risk reduction, 0.83; 95% CI, 0.79-0.88), with rehospitalization rates not reported.²³ In the present study, the performance measure for ACE inhibitor/ARB use at discharge was associated with a trend for lower 60- to 90-day postdischarge mortality and was associated with lower combined 60- to 90-day postdischarge mortality or rehospitalization. These results are also in agreement with those from a discharge medication program that led to improved heart failure patient outcomes.¹⁵ This OPTIMIZE-HF study provides further evidence supporting the process-outcome link for ACE inhibitor/ARB use and extends the findings to a cohort not entirely confined to Medicare patients.

The rationale for use of discharge instructions has been derived from observational studies and trials comparing conventional management with heart failure disease management programs that included patient counseling on diet, exercise, medications, and monitoring.²⁴ One randomized controlled trial of 223 systolic heart failure patients compared the effects of a 1-hour, one-on-one teaching session with a trained nurse educator with the standard discharge process. Patients in the education group were also given a copy of the treatment guidelines for heart failure written in patient-oriented language. Patients receiving the education intervention had a 35% lower risk of rehospitalization or death.²⁵ Heart failure disease management programs have shown improved functional capacity and fewer hospitalizations compared with traditional standards of care.²⁶

A recent meta-analysis of 8 randomized trials found a pooled relative risk for hospital readmission rates of 0.79 (95% CI, 0.68-0.91; P<.001). However, there was no apparent effect on mortality (relative risk, 0.98; 95% CI, 0.72-1.34; P = .90; P = .20 for heterogeneity).²⁶ Notably, these programs involved elements well beyond those assessed in the discharge instruction performance measure. It is unclear, both in these studies and within the OPTIMIZE-HF program, whether the discharge instruction performance measure as extracted from hospital medical records truly reflects whether the patients did or did not receive each defined component. Patient education may be documented in the medical record even if it was done in a rushed or superficial manner at discharge, making it unlikely that the patient would retain the information.²⁴ Conversely, physicians or nurses may instruct a patient about medications, diet, symptoms of worsening heart failure, and daily weight monitoring but may not record this in the patient's chart.²⁴ To be counted as conformity to the performance measure, the presentation of discharge instructions in OPTIMIZE-HF required completion of all 6 domains.⁸ It has not been previously examined whether this or any discharge instruction measure construct distinguishes quality of care for heart failure patients. In the present analysis, the discharge instruction performance measure was not predictive of early mortality or mortality/rehospitalization in the 60- to 90-day time frame.

As with use of discharge instructions, smoking cessation counseling, left ventricular systolic function assessment, and anticoagulation use for atrial fibrillation have not been examined as effective performance measures for heart failure patients. There have been no trials to date that specifically assess smoking cessation counseling in hospitalized patients with heart failure; the guideline recommendations that serve as the basis for this measure were derived from expert opinion.⁴ Although smoking cessation counseling has been validated in patients after myocardial infarction,²⁰ in OPTIMIZE-HF this performance measure showed favorable trends in early clinical outcomes that did not reach statistical significance. Within a 60- to 90-day time frame, it may not be possible for a smoking cessation intervention to have a discernible effect on clinical outcomes in heart failure patients. While left ventricular systolic function assessment as a performance measure in OPTIMIZE-HF was not independently associated with 60- to 90-day outcomes, it is a necessary step to determine eligibility for the ACE inhibitor/ARB and β-blocker measures. National guidelines recommend this diagnostic assessment in all patients with suspected heart failure.⁴ One observational study in patients with newly diagnosed heart failure showed an association between obtaining a left ventricular ejection fraction measurement and both increased use of ACE inhibitors and subsequent improvement in outcomes.²⁷ Although the performance measure for left ventricular function assessment could be a surrogate for other processes that may influence outcomes, this OPTIMIZE-HF analysis found no independent association with clinical outcome for this measure. Similar trends were observed with anticoagulation therapy in OPTIMIZE-HF. While anticoagulation for atrial fibrillation has been shown to reduce the risk of fatal and nonfatal stroke,²⁸ use of anticoagulation in patients with heart failure in OPTIMIZE-HF showed a trend toward lower mortality and mortality/rehospitalization that did not reach significance. However, with annual embolic stroke rates of 3% to 6%, it would be unlikely that anticoagulation could influence these outcomes in a 60- to 90-day time frame.

This study also assessed the process-outcome link of a β-blockade heart failure inpatient performance measure based on the wealth of clinical trial data demonstrating the efficacy of β-blockers in reducing mortality and hospitalizations in patients with heart failure, including those with recent decompensation.^29- 33 β-Blocker use at the time of hospital discharge in the present study was associated with significant improvements in both early mortality and the combination of early mortality and rehospitalization. This study demonstrates that the association between process and 60- to 90-day outcome was stronger for β-blocker prescription at discharge than for any of the current ACC/AHA performance measures.

Although clearly stated methods have been used to develop and implement heart failure inpatient performance measures,⁷ our results suggest that the current ACC/AHA heart failure inpatient performance measures, aside from prescription of an ACE inhibitor/ARB at discharge, are not significantly influencing early postdischarge mortality and mortality/rehospitalization. For the ACC/AHA performance measure sets to achieve their stated goal of serving as a vehicle for more rapidly translating the strongest clinical evidence into practice, better methods for identifying and validating new performance measures may be needed. As this limited set of performance measures is being used to publicly report the quality of heart failure care delivery at the hospital level and is beginning to affect financial payments to medical centers and individual physicians, it is essential that measures be prioritized to include those that are proven to be closely associated with patient outcomes. Additional measures with stronger process-outcome links in the first 60 to 90 days after hospital discharge, such as use of β-blockers in eligible patients, should be considered. In addition to process measures, direct reporting of postdischarge clinical outcomes, such as risk-adjusted 30-, 60-, or 90-day mortality or mortality/rehospitalization rates, may be important. Although these findings require confirmation in other studies, they suggest that use of the ACC/AHA heart failure performance measures in their current form in CMS pay-for-performance programs may not be the most efficacious way to assess quality of care, given the lack of a connection between the majority of performance measures and early heart failure patient outcomes.

OPTIMIZE-HF hospitals are self-selected and, thus, may not be entirely representative of national care patterns and clinical outcomes. The follow-up data included in our analyses are derived from only a subset of patients in the overall registry. Because our follow-up data only extend to 60 to 90 days after discharge, the time frame examined may not have been of sufficient duration to realize the impact of some interventions. However, as the first 60 to 90 days after discharge comprise the period of highest risk for mortality and rehospitalization and are most likely to reflect the processes of hospital-based care, assessing early postdischarge outcomes should be a priority. This analysis may not have been sufficiently powered to detect small but clinically important differences in early postdischarge mortality. We did not assess health-related quality of life, functional capacity, patient satisfaction, or other clinical outcomes that may be of interest, and the ACC/AHA heart failure performance measures may or may not be associated with these outcomes. Eligibility for a performance measure is based on documentation in the medical record and is thus dependent on the accuracy of this documentation. Some patients may have had contraindications or intolerances that were not documented, leading to an overestimation of the number of patients eligible for the performance measure. The associations between care processes and outcomes do not necessarily prove causality. The process measure outcome association can be confounded by patient age, sex, race/ethnicity, comorbidities, and socioeconomic factors. The multivariable and propensity models used in this study factored in age, sex, race/ethnicity, and multiple comorbidities; however, we were not able to adjust for socioeconomic factors. There may also be other measured or unmeasured confounding variables that, had they been adjusted for, would have strengthened or weakened the process-outcome link for some of the performance measures.

Of the 5 current ACC/AHA heart failure inpatient performance measures, only ACE inhibitor/ARB use at discharge was associated with 60- to 90-day postdischarge mortality or rehospitalization. β-Blocker use at the time of hospital discharge was associated with the most significant improvements in heart failure patient outcomes. To more accurately identify practitioners and medical care facilities providing the care that is most closely associated with optimal early postdischarge outcomes, additional measures, as well as better methods for identifying and validating heart failure performance measures, may be needed.