Variation in Carotid Endarterectomy Mortality in the Medicare Population: Title and subTitle BreakTrial Hospitals, Volume, and Patient Characteristics FREE

SINCE 1990, several randomized trials comparing carotid endarterectomy (CEA) with medical therapy for atherosclerotic narrowing of the bifurcation of the common carotid artery have been published.^{1 - 6} The 2 largest trials in the United States, the North American Symptomatic Carotid Endarterectomy Trial (NASCET)⁷ and the Asymptomatic Carotid Atherosclerosis Study (ACAS),⁸ were discontinued early when the beneficial outcomes in the surgical arms crossed the threshold required to invoke the respective trial's "stop rule." The National Institutes of Health released the preliminary results via "clinical alerts" to physicians.^{9 - 10} These alerts suggested that CEA is the preferred treatment for both symptomatic and asymptomatic patients with carotid artery disease if the procedures are performed in centers with low risk of perioperative stroke and death.

Institutions and surgeons who participated in the NASCET and ACAS trials were carefully selected.^{11 - 12} The NASCET trial required participating centers to have performed at least 25 CEAs a year with a less than 6% combined perioperative stroke and death rate. The ACAS trial used a 2-step selection process: hospitals had to submit mortality and morbidity data; if the center had acceptable results, individual surgeons within the institutions submitted data showing a combined perioperative event rate of less than 3%. Additionally, ACAS had ongoing evaluations of perioperative events; if more than 1 perioperative event occurred, an institutional audit was undertaken to evaluate eligibility for further participation in the trial.

Patients were also carefully selected. In the NASCET trial, patients were excluded if they were older than 79 years, had organ failure or cancer likely to cause death in 5 years, or had a cardiac valvular or rhythm abnormality. Additionally, patients were "temporarily ineligible" if they had uncontrolled hypertension or diabetes or had experienced unstable angina or myocardial infarction in the preceding 6 months. Only one third of the patients operated on in participating institutions were enrolled in the trial.¹² In the ACAS trial, patients were excluded if they were older than 79 years, had comorbidity that could increase their operative risk, or had a condition that could preclude long-term participation or was likely to cause death or morbidity. To meet these criteria, 25 patients were screened for every 1 randomized.⁸

These precautions appeared to have been worthwhile. Participating institutions achieved very low perioperative (30 day) mortality rates: 0.6% in NASCET and 0.1% in ACAS. Because the benefit of the procedure depends on achieving these low mortality rates, questions have been raised about generalizing the results of these trials to everyday practice.^{13 - 15} To examine the effectiveness of CEA in "routine practice," we used Medicare data to examine results in all hospitals performing the procedure in the United States. Since Medicare covers nearly all patients 65 years of age or older and over 70% of all CEAs performed are in this age group,¹⁶ these data provide an accurate reflection of overall outcomes for patients undergoing CEA. Because perioperative stroke is difficult to assess from administrative data, we focused on mortality. We asked the following questions: What is the perioperative mortality rate for Medicare patients undergoing CEA, and how does the perioperative mortality rate among those who have their CEA at a trial hospital (one of the institutions participating in the NASCET or ACAS trials) compare with that among those undergoing the procedure in a nontrial hospital?

As part of our work on the Dartmouth Atlas of Health Care,¹⁷ we studied patients undergoing CEA using the Health Care Financing Administration's (HCFA) 100% Medicare Provider Analysis and Review (MEDPAR) file. We identified all Medicare participants who underwent CEA (International Classification of Diseases, Ninth Revision [ICD-9] code 38.12)¹⁸ during 1992 and 1993 (N=138521). Patients were excluded if they underwent a concurrent open heart procedure (diagnosis related group codes 103-111), had missing data on race or sex, or were younger than 65 years (total number excluded = 23449). Finally, for patients with multiple procedures during this time period (N=1772), only the first was counted. This left 113300 patients for this analysis. The health insurance claim number from the MEDPAR file was linked to the HCFA Denominator file, which includes date of death for all Medicare enrollees.

The outcome of interest was perioperative (30 day) mortality. Time to death was calculated using the date of death and date of the CEA procedure from the linked file.

From the coordinating centers for the ACAS and NASCET trials, we obtained a list of all institutions that participated in the trials (the "trial" hospitals). Using the trial hospitals' HCFA provider numbers, we identified all Medicare patients undergoing procedures at one of these institutions. Because previous work has identified a volume-outcomes relationship for CEA,^{19 - 20} we categorized the nontrial hospitals into volume terciles based on the average annual number of Medicare CEAs performed by each.

Patient characteristics used as independent variables included age (65-69, 70-74, 75-79, 80-84, ≥85 years), sex, race (white, black, other), urgency of admission as coded on the MEDPAR claim (elective, urgent, emergent), and comorbidity, as measured by the Dartmouth-Manitoba modification of the Charlson comorbidity index (scored as 0, 1, or ≥2).²¹

Chi-square tests were used to assess mortality rates and mortality trends across categories of predictor.²² We also estimated perioperative mortality, adjusted for patient factors, using logistic regression.²³ Indicator variables were used to define patient characteristics and hospital category. Improvement in model fit from the addition of hospital category to the reduced model was assessed by likelihood ratio tests for nested models.²⁴ The test of trend across hospital categories (trial hospitals to lowest-volume nontrial hospitals) was performed by testing a 4-level categorical variable in the model.²⁵ Adjusted mortality rates by hospital type were calculated by evaluating covariates at the sample mean and "turning on" the appropriate indicator variable, thus adjusting the rates to assume that all institutions treated the same patient mix.²³

Table 1 shows the Medicare CEA population characteristics, their relationship to mortality, and their distribution across trial and nontrial institutions. The overall 30-day mortality rate for Medicare CEA patients in 1992 and 1993 was 1.75%. There was a strong relationship between age and perioperative mortality: Patients 85 years or older were 3 times more likely to die than those younger than 70 years. In contrast to the results of the ACAS trial, men were more likely to die than women. Patients with higher comorbidity scores were much more likely to die, as were those patients admitted emergently for their procedure.

Trial hospitals constituted 3.2 % of institutions performing CEAs; these institutions performed just under 6% of all Medicare CEAs. The mortality rate at trial hospitals was about 20% lower than that at nontrial hospitals. While the distribution of patient characteristics was similar in trial and nontrial institutions, trial hospital patients were somewhat younger, more likely to be male, and more likely to undergo an emergent procedure than those who had their CEA at a nontrial facility. The most striking difference between trial and nontrial institutions was that trial institutions had much higher annual procedure volumes. However, the mean number of procedures in trial institutions was lower than that of high-volume nontrial institutions (76 vs 93, P<.005).

Figure 1 shows crude and adjusted perioperative mortality rates at trial hospitals and the 3-volume terciles of nontrial institutions. For comparison, we also include the ACAS and NASCET perioperative mortality rates. There was a strong linear relationship between perioperative mortality (crude and adjusted) and hospital characteristics. Trial institutions had the lowest adjusted mortality rates (1.4%; 95% confidence interval [CI], 1.2%-1.7%), followed by high-volume (1.7%; 95% CI, 1.6%-1.8%), average-volume (1.9%; 95% CI, 1.7%-2.1%), and low-volume (2.5%; 95% CI, 2.0%-2.9%) nontrial institutions (P for trend, <.001 for both crude and adjusted rates). In the multivariate model controlling for age, sex, race, comorbidity, and urgency of admission, patients undergoing their procedures at a trial institution had a mortality risk reduction of 15% (95% CI, 0%-31%) compared with high-volume nontrial institutions, 25% (95% CI, 7%-40%) compared with average-volume institutions, and 43% (95% CI, 25%-56%) compared with low-volume institutions. Importantly, the mortality rate for the average Medicare patient undergoing the procedure at a trial hospital was substantially higher than those reported in either trial.

Large randomized trials such as NASCET and ACAS are difficult to design and complicated to run. Because of the investigators' desire to increase the probability of finding a difference between therapies if one exists, hospitals and patients were carefully selected. Medicare beneficiaries undergoing CEA experience significantly higher perioperative mortality rates than participants in the major randomized trials that provided evidence for the efficacy of the procedure. Processes of selection that operated at both the institution and patient level could explain the lower mortality rates observed in the trials.

Trial institutions underwent an arduous selection process and represent less than 4% of all institutions in the United States providing CEAs. For the average Medicare patient, no institutional evaluation process occurs; they simply go where their physician tells them to. Most physicians have no idea about their own relative performance, let alone that of the institution in which they work. Indeed, most individual institutions' experience will be insufficient to make a valid assessment of performance: 60% of institutions performed fewer than 17 CEAs annually on Medicare patients (which approximates a total CEA volume of 25 per year, the cutoff used in the trials). We found that Medicare patients who undergo their CEAs at a trial hospital have a lower risk of dying than those who have their procedures at other institutions.

Concerns about the generalizability of the trials because of patient selection are also warranted.²⁶ Randomized patients were much younger (the average age in the randomized patients was approximately 65 years) and healthier than those in general practice. Among hospitals participating in the trials, the perioperative risk for the average Medicare patient (1.4%) substantially exceeded that experienced by the patients who underwent surgery in the trials, even if one assumed all of the deaths in the trials occurred in those older than 65 years (of the 1052 patients who were randomized to and received surgery in the trials, only 3 died). Even among Medicare patients who had elective admissions in our cohort, presumably those at lowest risk, the mortality rate at the trial institutions was 1.22% (data not shown), twice as high as that in NASCET, and 10 times the rate in ACAS. Of particular note is the association of increasing risk with advanced age. Over 40% of Medicare patients undergoing CEA during this time period were 75 years of age or older and more than 15% were 80 years or older, the age eligibility cutoff for the trials. For these patients, the perioperative mortality was greater than 2%. It is unclear that future benefits of the procedure could ever outweigh the high risk of perioperative mortality in this group.²⁷

The volume-outcomes relationship we found is consistent with previous studies.²⁰ Interestingly, despite performing fewer annual procedures on average than the high-volume nontrial institutions, trial institutions had a lower perioperative mortality rate. While we were unable to assess indications for the procedure, if differences in indications account for the apparent volume-outcome effect (as has been suggested by others),²⁸ it would appear that even among high-volume centers, indications vary significantly.

Our findings raise questions about the wisdom of generalizing the findings of controlled trials to all settings in which CEAs are performed. However, several limitations of our study should be considered. First, our measures of outcome and case mix were limited. This would bias our findings only in the unlikely event that stroke rates are consistently lower in those institutions with higher perioperative mortality. Further, because the perioperative mortality measured in our study does not account for those deaths that occurred during carotid arteriography prior to surgery, ours is an underestimate of the total mortality risk of the intervention. We may not have accurately controlled for differences in patient case mix. However, the strong relationship between mortality and age, sex, race, urgency of admission, as well as comorbidity score were accounted for in our adjusted rates. As noted above, we could not measure indications for the procedure.

Second, our hospital categorizations capture multiple components in the delivery of care, including physicians. We are unable to say what part or parts of this delivery process made a difference in the risk of perioperative death, nor could we assess the relationship between physician-specific volume and outcomes. Finally, while the Medicare population accounts for approximately 70% of CEA patients, our results may not reflect results in the non-Medicare population. These limitations, however, underscore the importance of providing more accurate data about the risks and benefits of CEA in routine practice.

The randomized controlled trial is considered to be the "gold standard" for assessing medical efficacy. Since the release of the NASCET and ACAS trials, the number of CEAs performed on Medicare enrollees has nearly doubled.¹⁷ Most of the recent increase is likely because of the performance of the procedure on those with asymptomatic stenosis.²⁹ Three of the 4 trials evaluating CEA in the asymptomatic patient have found CEA to be less efficacious than medical treatment.^{1 - 3 ,8} For asymptomatic patients, the contrast between the efficacy reported in ACAS and the effectiveness in routine practice is of particular concern. Even in the ACAS trial, which had extremely low perioperative mortality, the absolute difference between up-front risk and future gain was small. If the chance of perioperative death is 1 in 100 rather than the 1 in 1000 reported in the trials, this procedure is unlikely to be effective in actual practice. The releases of the trials' results have generated a significant amount of debate in the medical literature.^{29 - 31} It seems that the caution called for by those advocating restraint is warranted.