Use of Continuous Quality Improvement to Increase Use of Process Measures in Patients Undergoing Coronary Artery Bypass Graft Surgery:  A Randomized Controlled Trial FREE

In 1917 Ernest Codman, MD, a Massachusetts surgeon, described his view of a quality evaluation system for medicine, in which clinicians assessed outcomes and the processes that led to those outcomes.¹ The modern iteration of Codman's quality system for medicine is continuous quality improvement (CQI).^2- 3 Adapted from industrial manufacturing principles in Japan and the United States,⁴ CQI in medicine is the repetitive cycle of process and outcomes measurement, design and implementation of interventions to improve the processes of care, and remeasurement to determine the effect on quality of care. Successful CQI programs in medicine have been difficult to achieve, in part due to a lack of appropriate information technology⁵ and organizational infrastructure.⁶

Randomized trials testing the effectiveness of CQI as an approach to quality improvement in medicine to date have yielded mixed results. A report by Soumerai et al⁷ documenting the impact of local opinion leaders in 37 centers was mostly positive, but subsequent studies^8- 9 did not show a benefit from CQI efforts. These trials involved small numbers of sites, making difficult the scalability of the finding to the nation as a whole.

The report herein documents the results of a randomized trial of CQI in medicine undertaken on a national scale. We used the infrastructure of the Society of Thoracic Surgeons (STS) National Cardiac Database (NCD) and a partnership between the STS and the Duke Clinical Research Institute (DCRI) to administer the trial under a grant from the Agency for Healthcare Research and Quality to the STS. The setting of the trial was coronary artery bypass graft (CABG) surgery from January 2000 until July 2002, with the first round of intervention taking place in January 2001.

The objective of this trial was to test whether a low-intensity CQI intervention could be used to speed the national adoption of 2 CABG surgery process-of-care measures: β-blockade and internal mammary artery (IMA) grafting.

The voluntary NCD was begun in 1989,¹⁰ and in collaboration with the DCRI outcomes group has developed mortality, morbidity, and length-of-stay risk models for CABG surgery and other major cardiac procedures for adults.¹¹ Changes in the NCD to facilitate quality improvement efforts have been documented elsewhere.^12- 13 As of December 2002 there were more than 2.1 million adult patient records in the NCD collected from over 400 sites nationwide.¹⁴

The quality of this voluntary dataset has been assessed in a regional independent chart abstraction study, which documented a 96.2% correlation between submitted and reabstracted data elements.¹⁵ Completeness of NCD data has also been compared with data from a Centers for Medicare and Medicaid Services diagnosis–related group dataset (1994-1999) for CABG surgery, and no evidence for underreporting of cases or postoperative events was found in the STS data.¹⁵

Preoperative β-Blockade Therapy in CABG. This measure was selected because of the protective effects and underuse of β-blockade reported in patients with cardiovascular disease^16- 18 and in the noncardiac surgical literature.^19- 20 Nonetheless, some surgeons have expressed concerns regarding use of β-blockade in CABG surgery due to its negative inotropic effects.²¹ Additionally, incorporation of this measure requires significant collaboration of the surgical team with cardiologists and anesthesiologists. Thus this β-blockade measure was felt to be a complex process measure that assesses the incorporation of new evidence into practice under circumstances in which cooperation among clinicians is necessary.

IMA Grafting in Elderly Patients. The acute^22- 23 and long-term^24- 25 survival benefit conveyed by IMA grafting to the left anterior descending coronary artery in patients undergoing CABG surgery overall has been well documented. However, use of IMA grafting in elderly (≥75 years) patients was controversial at the trial inception because of age comorbidity (associated with increased procedural risk)²⁶ and to increased technical complexity and potential for morbidity of IMA grafting (prolonged ventilation, increased perioperative bleeding),²⁷ and to absence of a long-term mortality benefit. Because its use was solely at the discretion of the operating surgeon, IMA grafting was considered a simple measure. This measure thus assessed the expansion of existing evidence into a new patient population.

Prior to the trial, all STS participant sites were surveyed regarding CABG care processes. Of 477 (93%) sites responding, 348 (73%) estimated they used preoperative β-blockade therapy in their patients; however, 86 (18%) strongly disagreed with its use. Likewise, 372 (78%) estimated they used IMA grafting in elderly patients, but 105 (22%) strongly disagreed with its use in this setting.

Because sites were the unit of randomization, the power for these analyses was conservatively calculated using sites as the unit of analysis. The mean (SD) percentage of β-blockade use in the 1997 STS database retrieval of data was 55% (17%) across sites. Assuming an increase of 5 percentage points in use over time in the control arm, sample sizes of 110 sites in the control arm and 110 in each intervention group would yield 86% power to detect an improvement of 7 percentage points from baseline to end-of-study use of β-blockade (ie, 67% use of β-blockade in the intervention group vs 60% in the control arm at study end, or a difference of 7 percentage points in improvement between groups), with a 2-sample t test and a conservative estimate of the variance in the difference. The SD across sites for use of IMA grafting was much lower; assuming the same level of difference, fewer sites were needed for the IMA analysis to detect the same magnitude of difference.

A total of 399 sites performing cardiac surgery and geographically distributed within the United States were considered for inclusion in this study based on ongoing participation in the NCD over the prior 3 years. As a component of the study, 40 sites in Alabama, Colorado, Iowa, Minnesota, New Mexico, and Wyoming participated in a regional consortium group and were not randomized or included in this analysis. The remaining 359 sites were randomized into 1 of 3 arms (Figure 1). Sites randomized to the intervention arm received either material on preoperative β-blockade (n = 124) or on use of IMA grafting in elderly patients (n = 120), and those randomized to the control arm (n = 115) received no intervention material.

Some sites (n = 40) did not provide data during the trial because of economic and/or software issues. There were no systematic differences between intervention groups among these sites, so a resulting selection bias was unlikely.

Participant sites were stratified by yearly volume of CABG surgery prior to randomization by the DCRI because CABG processes and outcomes may be significantly influenced by the procedural volume at that site.

All NCD sites were informed that they may periodically receive supplemental educational reports in addition to the standard site-specific semiannual reports. Participants were specifically not told of the current study design, nor that other NCD participants might be receiving interventions different than the one they received. The Duke University institutional review board served as the multiple projects assurance entity for this study for the STS, and determined that informed consent was not required.

From January through December 2000, baseline measurement data were collected from all 359 sites. In January 2001, a sequence of 3 intervention rounds was begun by distribution of material to randomized sites. This low-intensity CQI intervention consisted of (1) identifying a local opinion leader (quality champion) at each site; (2) a call-to-action letter; (3) process-measure data (scientific evidence for use of data; site-specific measure performance data illustrated against regional, national, and national "best practice" benchmarks); (4) action plan for CQI; (5) CQI slide set and reprints of evidence-based data for process measure and CQI; and (6) contact information.

In round 1, primary recipients were the CQI leader and the data manager at the site, and abbreviated electronic versions were sent to secondary surgeons at the sites. Each site documented if the material was used and by whom.

Rounds 2 (July 2001) and 3 (November 2001) were similar in content. Round 2 additionally included a CQI newsletter specific for the appropriate measure. For round 3, a measure-specific CQI Web site was created, with a tracking system for collecting the number of hits with site identification.

Demographic patient-level and hospital-level characteristics between the 3 arms were compared to evaluate the adequacy of the randomization scheme.

When preoperative β-blockade therapy was considered, all patients were included in the analyses; where use of IMA grafting was considered, patients younger than 75 years and reoperative cases were excluded (Figure 1). First, we performed a preintervention/postintervention analysis for which the outcomes of interest were the site differences between preintervention (January 2000-December 2000) and postintervention (July 2001-July 2002) percentage for use of the measures. Mean differences in use of the measures among intervention sites were compared with mean differences in the control arm using 2-sample t tests.

We then performed a risk-adjusted patient-level analysis to account for differences in patient mixes at the different sites. The outcome measure for this analysis was a dichotomous variable indicating whether the patient received the measure of interest. A hierarchical logistic regression analysis included 28 preoperative patient characteristics used in the current STS model for CABG mortality risk along with random site effects to account for within-site clustering.²⁸ To allow for changes in the use of measures over time, a time trend (linear on the logit scale) was included. To determine whether the time trend for intervention groups differed from that of the control arm, the statistical test of interest was the interaction between time and intervention group.

We also performed a subgroup analysis on the β-blockade and IMA change in the percentage of use, examining the interaction between intervention group and site of CABG surgery, academic vs nonacademic center site characteristic, and use of process measure at baseline. For volume and measure performance, sites were categorized as low, medium, or high according to whether they were in the first, second, or third tertile for the variable of interest.

Table 1 shows demographic patient- and hospital-level characteristics for the 3 randomized arms. There were 309 sites that provided data during the intervention: 101 sites each for the control arm and for the IMA grafting group and 107 for the β-blockade group. Mean annual case volumes were between 305 and 348. The proportion of patients older than 75 years and the baseline (mean for year 2000) use of β-blockade and IMA grafting were equivalent across the 3 arms.

Compared with baseline year 2000 data, the percentage of control sites that improved (increased use of measures in last year) was 66% for both measures; in the β-blockade group, 81% had an increase in use of β-blockade and 63% had an increase in use of IMA grafting; in the IMA group, 75% had an increase in use of β-blockade while 71% had an increase in use of IMA grafting.

Table 2 demonstrates the change in measure use over the study interval in the targeted intervention group compared with the control arm (no intervention) and with the other, nontargeted, intervention group. From January 2000 to July 2002, use of both process measures increased nationally (β-blocked, 60.0% 65.6%; IMA grafting, 76.2%82.8%). While all groups showed an increase in the measure being evaluated over time, patients in the β-blockade intervention group were more likely to receive β-blockade (7.3% [SD, 12.8%]) compared with those in the control arm (3.6% [SD, 11.5%]) (P = .04 in the preintervention/postintervention analyses and P<.001 in the hierarchical analyses). Use of β-blockade also increased more in the nontargeted IMA grafting group than in the control arm (5.4% vs 3.6%, P = .33 and P = .02, respectively, for the 2 analyses).

Use of IMA grafting also tended to increase at IMA intervention sites (8.7% [SD, 17.5%]) vs control sites (5.4% [SD, 15.8%]), although the comparisons were not statistically significant (P = .20 or P = .11 for preintervention/postintervention and hierarchical analyses, respectively).

When the targeted intervention group was compared with both other groups combined, both targeted interventions achieved significance (for hierarchical analysis, P = .03 for IMA grafting vs control plus β-blockade; P = .01 for β-blockade vs control plus IMA grafting).

Figure 2 shows the results of the preintervention/postintervention trends for β-blockade and IMA grafting over the 5 report intervals from spring 2000 to spring 2002, comparing the intervention groups with the control arm for purposes of clarity. With the January 2001 round 1, both the β-blockade and IMA grafting groups increased the use of the process measure and continued to increase use throughout the remainder of the trial interval.

Table 3 indicates that for both the β-blockade and IMA grafting groups the difference in percentage measure use compared with the control percentage difference was significant for site case volumes stratified into low (<228 cases/y), medium (229-449 cases/y), and high (≥450 cases/y) strata. Low-volume sites had significantly greater incorporation of both measures compared with the corresponding medium- or high-volume sites (for interaction: P = .04 for β-blockade; P = .02 for IMA grafting). Compared with nonacademic centers, academic centers showed a trend toward a higher incorporation for both measures. Also, compared with sites with a higher use of measures at baseline, sites with a lower use showed a trend toward improvement for both measures.

This study reports the first randomized trial of CQI accomplished on a national scale. We found that a provider-driven, low-intensity CQI intervention could have demonstrable impact on local CABG care practices within a 2-year period. Although the clinical impact of the trial was modest, we believe the results demonstrate the potential for medical specialty societies to have an impact on the national adoption of important care processes into clinical practice.

This trial tested whether a medical specialty society could leverage clinician motivation in a call to action to CQI at the local level. Bradley et al²⁹ identified factors found to predict adoption of care processes, including defined goals and empowerment of local site leadership; the other 2 factors of CQI infrastructure support and high-quality performance feedback were provided by the platform of a nationally representative clinical database in this trial. The NCD assessed baseline care patterns, gaps between actual and recommended care, and variability among site practices.^30- 31 In the intervention phase, the NCD provided a clinically integrated mechanism for ongoing measurement of quality process and outcomes measures and their evaluation in the context of adjusted patient risk. This site-specific feedback, along with national and best-practice benchmarks,³² provided contemporaneous data for motivation, goal setting, and documentation of local success.

We designed a low-intensity CQI intervention because this approach might have greater effects on health care and policy than a more intensive intervention that might be difficult to duplicate in other health care settings. Moreover, a low-intensity intervention matched the scope of leadership and resources that could be provided by national society sponsorship. Specifically, this initiative did not mandate the use of specific CQI tools but rather allowed individual sites to determine how best to implement changes in practice at their own sites.

A successful change in site behavior due to this low-intensity, multi-faceted³³ intervention was suggested by the percentage of sites that improved in use of the measures during the trial and by the percentage increases in measure use (Table 2). In addition, the variance of practice surrounding these measures generally narrowed; this was more apparent in the IMA grafting group ("simple measure") than in the β-blockade group ("complex measure").

While the β-blockade CQI intervention had significant overall effect and the IMA grafting intervention showed positive trends, the overall clinical magnitude of the results was quite modest. In part, this result may have been due to the process measures examined and the trial context. The 2 measures were selected to test the implementation of new information into clinical practice through this CQI platform, and thus had limited scientific support at the start of the trial. Additionally, their link to outcomes was not established, particularly in context with the underlying ongoing decline in CABG mortality observed over the past decade.³⁴ Subsequent observational analyses from the NCD documented the positive independent impact of IMA grafting³⁰ and preoperative β-blockade therapy³¹ on acute (30-day) CABG mortality and morbidity. Both published in May 2002, these reports had little impact on the trial results.

The improvement over time in the control group for both measures (Figure 2) suggests that the context of the quality improvement infrastructure and the goals of the NCD may have influenced the trial results as well. This improvement in the control arm may reflect use by these sites of intervention measure data included in the routine NCD site reports, but also may reflect cross-contamination of educational and targeted interventional materials as sites (blinded to the study design) compared clinical practices. However, despite these process-measure considerations and general quality improvement trends, this low-intensity CQI effort resulted in a significant improvement in the β-blockade intervention (overall difference in the increase in β-blockade, round 3: 8% between the targeted β-blockade group and the control arm), both in the trend analysis (Figure 2A, P = .04) and in the hierarchical analysis (Table 2).

The lack of significance in the IMA grafting intervention may relate to a number of factors. First, the differential between intervention and control sites was less than anticipated. Although the IMA preintervention/postintervention analysis difference was comparable with the β-blockade difference, the variability of the site differences was about 50% greater for IMA grafting than for β-blockade, and greater than anticipated (Table 2). The patient-level analysis was more sensitive (P = .11), again despite results similar to those of the β-blockade intervention. Other factors included more room for improvement nationwide in the β-blockade group (lower percentage use at the start of the trial), the possibility of a ceiling effect at the highest-performing sites in the IMA grafting group, and the possibility that the intervention was too low-intensity for the nonresponding surgeons (22% in the survey) to change their minds.

This trial was able to test the efficacy of the CQI process in a controlled trial format on a national scale. The trial groups (Table 1) each reflected a national representation. This national scalability distinguishes this trial from prior important observational quality improvement efforts in cardiovascular disease in cardiac surgery,^34- 39 and to a more limited extent, in cardiovascular medicine.^40- 42

In addition, the rigorous evaluation of CQI using the randomized controlled trial format distinguishes this trial from prior studies. Many of these prior successes consisted of either high-intensity or site-specific interventions, while in others it was difficult to account for baseline changes in quality of care. For example, the decline in mortality for CABG surgery observed in New York state³⁷ was paralleled in the Northern New England Consortium group,³⁵ in Massachusetts Medicare-aged patients,⁴³ and more recently was documented to have occurred nationally in both the Veterans Affairs³⁹ and STS³⁴ analyses.

Low-volume sites had a higher incorporation of both measures than their high-volume counterparts (Table 3), and this relationship was inversely related to volume. This response in low-volume sites may reflect a limitation of resources for CQI at low-volume sites that were now being addressed through the STS CQI platform. Given the interest in use of volume criteria alone as a marker of clinician quality,^44- 45 this more aggressive adoption of CQI for local improvement in care may well be an important finding.

It is noteworthy that the results of the trial, while modestly successful in terms of improvement in the use of the measures, were accomplished on a national scale and within a time interval that was rapid compared with many other CQI efforts.⁴⁶ The database and the medical specialty society sponsorship provided mechanisms for national organization, provider engagement, and site-specific performance feedback on quality-of-care metrics. Because improvement and adoption of best practices is primarily a local issue,^47- 48 we speculate that the accomplishments of the trial rest with the local sites and the clinician-led, voluntary focus on CABG outcomes and quality manifested by site participation in the NCD. This platform could provide a template for other areas of medicine.

First, these modest results might have been greater with a more intensive intervention, a more static control group, and/or a lesser degree of cross-contamination; however, the positive impact at a national level with this low-level intervention suggests that both the infrastructure and the intervention contributed to the trial outcome. Second, the outcomes assessed were processes of care with unestablished links to end point outcomes. Other factors that impact on dissemination of information in health care, including organizational and contextual factors, will be important in this study as well.⁴⁶ Third, we did not assess the CQI infrastructure costs; however, the trial costs and the cost of NCD participation at the site level compare favorably with effective regional voluntary and mandatory quality improvement efforts in CABG surgery. Finally, while nationally representative, the trial was not comprehensive for all CABG surgical procedures in the United States. While this study documents that this platform can achieve improvements in the quality of care, it remains for government and third-party entities to recognize this type of effort to facilitate expansion of this approach to all cardiac surgical centers and to other areas of medicine.

This trial of CQI in CABG surgery is the first rigorous successful randomized trial of CQI in medicine achieved on a national scale. As such, these results have important implications for medical specialty clinicians, because the CQI infrastructure used here documents a mechanism for active and effective clinician involvement in the CQI process. Further refinements in this infrastructure and the CQI process are suggested by these modest results; however, the overall scope and success of this trial suggest a model that can be adopted by other clinicians for translating research into everyday practice.