Improving Cardiac Surgery Quality—Volume, Outcome, Process?

In this issue of THE JOURNAL, Peterson and colleagues¹ use data from the Society of Thoracic Surgeons National Cardiac Database (STS NCD), the largest clinical cardiac surgery database in the world, to enhance current understanding of the volume-outcome relationship for coronary artery bypass graft (CABG) surgery. This large national study also will contribute to the ongoing debate regarding the relative merits of volume-based referral strategies, outcome reporting, and process-improvement initiatives to promote cardiac surgery quality.^{2 - 5}

Since the pioneering work of Luft and colleagues in 1979,⁶ hundreds of studies have investigated the association between volume (a structural characteristic of health care providers) and outcome for complex surgical procedures and for the management of certain medical conditions.^{2 ,7 - 15} The overwhelming majority of studies have shown a direct relationship between volume and outcome for both hospitals and individual practitioners. There is also important synergy between these 2 provider levels. High-volume hospitals may improve the outcomes of low-volume surgeons, presumably because of better processes, and high-volume surgeons operating at high-volume hospitals generally have the best results. A recent study by Birkmeyer et al¹⁶ using Medicare claims data suggests that individual surgeon volume may be more important than hospital volume for many procedures.

A number of methodological, statistical, and conceptual concerns regarding the relationship between volume and outcome remain unresolved.^{2 ,8 ,17} Many investigations have used inadequately adjusted Medicare administrative data, volume thresholds vary substantially among procedures and studies, and the mechanism for the association between volume and outcome remains uncertain. The overall relationship is also confounded by the fact that many low-volume programs have excellent outcomes.^{2 ,7 - 8 ,14 ,18 - 19}

Based on the preponderance of evidence, the expense and difficulty of other approaches, and the statistical uncertainty of outcome assessment for infrequently performed procedures, many policy makers have advocated volume-based referral strategies as an expedient way of achieving better health care outcomes.^{9 - 13} However, translating these general concepts into specific health policy has proven to be challenging. For example, what is the goal of volume thresholds? Should physicians, payers, and government agencies selectively direct patients only to those hospitals predicted to be the "best," selectively avoid only the "worst" hospitals, or strive for something in between, such as achieving some predetermined benchmark?^{2 ,8 - 13 ,15} Should such strategies primarily focus on high-risk patients, on procedures that will affect the greatest number of patients, or on procedures with the greatest range of mortality between low- and high-volume providers?^{2 ,11 ,15 ,20} Even if consensus could be achieved on these difficult questions, health care planners still would be left grappling with the practical implications and unintended negative consequences of redirecting large numbers of patients to regional centers.^{2 ,7 ,9 - 11 ,15 ,17 ,21 - 22}

Coronary artery bypass grafting is the most frequently performed and arguably the most extensively studied of all complex surgical procedures.^{2 ,12} Numerous investigators have used administrative data to explore the relationship between CABG procedural volume and outcome,^{16 ,23} but only a few have used risk-adjusted clinical data, including 3 studies based on data that are now more than a decade old. Two of these reports, from the STS NCD²⁴ and the Veterans Affairs Administration,²⁵ showed no significant relationship between volume and outcome except at very low program volume (ie, fewer than 100 annual cases).

The 1991 study by Hannan et al¹⁹ is one of the most important and influential in the literature. Based on 12 448 patients undergoing CABG surgery in New York State in 1989, the first year of the New York Cardiac Surgery Reporting System (CSRS), this study demonstrated a significant inverse relationship between CABG procedural volume and risk-adjusted mortality for both surgeons and hospitals. Although no volume threshold was mentioned by the authors, their results have been used to justify the Leapfrog Group's 450-procedure standard for CABG surgery. However, independent interpretations of these data suggest that the most important threshold may occur at a much lower volume.^{2 ,26}

The 2 most recent studies from the New York CSRS¹⁸ and the STS NCD¹ set a new benchmark for analysis of the relationship between CABG procedural volume and outcome. Each uses contemporary data from an excellent clinical database and incorporates both proper risk adjustment and hierarchical models. The New York study reflects more than a decade of aggressive regulatory oversight and mandatory outcome reporting from a state with few low-volume programs, whereas the STS study represents a more broad-based but voluntary national sample. In the former report, Hannan and colleagues¹⁸ analyzed risk-adjusted outcomes of 57 150 patients undergoing CABG surgery in New York between 1997 and 1999. Risk-adjusted mortality for high-volume hospitals was lower than that of low-volume hospitals regardless of the volume threshold, and high-volume surgeons operating in high-volume hospitals had the lowest risk-adjusted mortality.

The current STS NCD study by Peterson et al¹ is based on 267 089 CABG procedures performed at 439 US sites between January 2000 and December 2001. In contrast to the New York study,¹⁸ in which only half of the programs performed fewer than 500 CABG procedures per year (accounting for 25% of patients), 82% of STS programs were in this category. Fifty-two percent of patients in the STS NCD underwent CABG surgery at hospitals with annual volumes of 450 procedures or fewer, the current Leapfrog Group standard. Median CABG procedural volume was 253 cases in the STS NCD study, probably a realistic representation of the current situation in the United States.

Overall risk-adjusted mortality decreased with increasing hospital volume, but the slope of the relationship was flat (0.07% decrease for every additional 100 cases) and volume was a poor discriminator of better or worse outcomes. The absolute range of risk-adjusted mortality between highest-volume (>450 procedures/year) and lowest-volume (≤150 procedures/year) hospitals was only 0.7%, and many low-volume hospitals had better than average risk-adjusted mortality. The volume-outcome relationship was not significant for patients younger than 65 years or for low-risk patients with expected mortality rates of less than 1.5%. Similar to the results reported by Hannan et al¹⁸ and Birkmeyer et al,¹⁶ the best outcomes (2.4% risk-adjusted mortality) were achieved by high-volume surgeons (>139 procedures/year) operating in high-volume hospitals and the worst outcomes (3.3% risk-adjusted mortality) were observed for low-volume surgeons (≤85 procedures/year) operating in low-volume hospitals.

The best available data demonstrate that the absolute mortality spread for CABG surgery between high- and low-volume centers is small (approximately 1%-2%, compared with 10%-15% for esophagectomy and pancreatectomy), and that many lower-volume programs outperform state and national averages.^{1 ,16 ,18 ,23} These observations reflect a mature and standardized procedure, performed relatively frequently compared with other complex procedures. How can these findings translate into realistic health policy that provides public accountability and improves cardiac surgery outcomes, especially at a time when CABG procedural volumes are declining nationally? Based on current evidence, it appears to me that the Leapfrog Group's 450-procedure standard for CABG surgery is unreasonable and impractical. Were it to be implemented, 77% of the 439 STS centers in the study by Peterson et al¹ would close, as would most programs in, for example, my state of Massachusetts.

If there is any role for CABG volume standards, it would seem most logical to focus this health policy debate on programs that perform 150 or fewer procedures a year. These comprise 22% of STS NCD reporting sites, but certain geographic areas such as the Pacific coast have a markedly disproportionate percentage of such programs and would be most affected by volume standards. Within this lowest-volume group are found the highest average mortality, the most extreme outliers, the steepest volume-outcome relationship, and the greatest outcome variability. Arguably, however, many such programs appear to have excellent results, although the statistical certainty of these estimates is compromised by small sample sizes. Furthermore, closure of such programs would affect the delivery of cardiac and other services to patients in some areas while averting fewer than 50 deaths nationwide.¹ Finally, volume standards might be an incentive to inappropriate utilization for some borderline programs.

Keeping in perspective all these considerations regarding the lowest-volume programs, health policy debate should focus on the relative merits of (1) absolute volume standards that might disenfranchise some programs, (2) volume standards supplemented by mandatory outcome reporting, (3) selective application of volume standards only in metropolitan areas having large regional centers, or (4) volume-based referral only for certain high-risk patient subgroups.

Programs performing between 150 and 450 CABG procedures annually represent the majority of US centers. Most have risk-adjusted outcomes that are distributed rather symmetrically around state and national means, with the expected greater dispersion at lower volumes. Although there is a volume-outcome relationship within this procedure range, the slope of the relationship is relatively flat. Formal outcome measurement should be strongly encouraged for such programs in lieu of volume standards. Cardiac surgeons always have been leaders in voluntary outcomes analysis and continuous quality improvement initiatives.^{4 - 5 ,27 - 30} By formally embracing outcome reporting, they would demonstrate their firm commitment to quality, answer the increasing demand for public accountability, and preempt less-desirable alternatives.

What about programs that exceed the 450-procedure threshold? Higher volume is, on average, associated with better outcomes but does not guarantee them.⁷ Halm et al⁸ note that for 1995-1997, 47% of New York hospitals performing more than 500 CABG procedures per year and one third of high-volume surgeons had risk-adjusted mortality rates higher than the state average. All cardiac surgery programs should be encouraged to participate in the STS NCD, including those with high volume.

The issues of individual surgeon volume thresholds and outcome reporting are particularly controversial.^{3 ,31 - 32} Investigations based on risk-adjusted clinical data^{1 ,18 - 19} and the study by Birkmeyer et al¹⁶ using Medicare data suggest that surgeon volume is at least as important a factor in outcome as is hospital volume. However, many unanswered questions remain, including the importance of overall vs procedure-specific volume, practice at more than 1 center, volumes necessary for initial acquisition of skill as opposed to maintenance of proficiency, and transient vs permanent low-volume status.² Controversy also exists with regard to public reporting of surgeon outcomes. Because surgeons, not hospitals, ultimately choose whether or not to accept a patient for surgery, individual report cards might discourage surgeons from operating on high-risk patients, a group that actually may benefit most from surgical intervention.^{3 ,33 - 34} Surgeons also control how patient comorbidities and operative procedures are coded on data forms, and surgeon report cards may thus encourage "gaming" of the reporting system.^{3 ,31} One reasonable compromise that protects public safety and mitigates such unintended negative consequences would be to publicly release only hospital outcomes. A panel of experts would confidentially review individual surgeon outcomes and refer confirmed outliers to an appropriate state agency.

The public and the media must be educated as to the limitations of even the most sophisticated risk models, the futility of attempting to rank hospitals, and the fluctuations in outcome that occur naturally from year to year. Consumers, regulators, and insurers would be best advised to study such outcome data longitudinally to identify consistent and egregious outliers.

Most importantly, the processes that permit some hospitals and surgeons to consistently obtain better results must be identified.^{2 - 3 ,7 - 8 ,14 ,17 - 18 ,22} The fact that many low-volume hospitals achieve excellent outcomes suggests that superior processes, rather than volume per se, may be an important mechanism for high performance. Perhaps some of these processes are more easily and consistently implemented by high-volume centers, but they are certainly not restricted to them. Efforts to disseminate these best practices to all hospitals and surgeons must intensify,^{4 - 5} because this is the best way to reduce interprovider variability and to improve the quality of all cardiac surgery programs. Use of processes known to be associated with superior results will become a more widely adopted quality measure, as this approach does not rely on sophisticated yet still imperfect methods of risk adjustment.