Operator Experience and Carotid Stenting Outcomes in Medicare Beneficiaries FREE

Carotid stenting is increasingly being used to treat severe carotid atherosclerosis, an important cause of ischemic stroke.¹ Since approval of the first carotid stent system by the US Food and Drug Administration (FDA) in 2004, use of carotid stenting has more than doubled in Medicare beneficiaries.² Explanations for the rapid dissemination of carotid stenting into routine clinical practice include its minimally invasive nature, lack of need for general anesthesia, and an expanded pool of physicians capable of performing it.³ Yet despite its overall promise, the increasing use of carotid stenting also raises potential concern. Carotid stenting is a technically demanding procedure and earlier studies have demonstrated a substantial learning curve with it.^4- 5 Although this has prompted consideration of establishing minimum volume requirements and training criteria for potential operators,^6- 8 specific recommendations have varied across professional organizations, and none have been widely enforced.⁹ As such, the total number of operators currently performing carotid stenting in routine clinical practice and their overall experiences and outcomes with the procedure remain largely unknown.

In this context, we used national Medicare data to approximate recent patterns of utilization and outcomes for carotid stenting in the United States among elderly patients—a high-risk group that makes up approximately three-quarters of those undergoing this procedure.¹⁰ As with other complex medical and surgical procedures, we hypothesized that lower-volume operators performing carotid stenting would have worse risk-adjusted outcomes compared with higher-volume operators. In addition, we examined whether a learning curve existed among new operators with evidence of having first performed carotid stenting after an initial national coverage decision by the Centers for Medicare & Medicaid Services (CMS) in March 2005, hypothesizing that worse outcomes would occur in patients treated earlier during these operators' experience.

From the CMS, we obtained data from the Physician Carrier (Part B), Medicare Provider Analysis and Review (MEDPAR), and Denominator files for all patients undergoing carotid stenting from January 1, 2005, to December 31, 2007. Physician Carrier files include data on claims for procedures by noninstitutional providers, such as physicians, while MEDPAR files include data on acute care hospitalizations and Denominator files contain information on eligibility and enrollment. Procedures of carotid stenting were identified from the Physician Carrier files using Healthcare Common Procedure Coding Systems (HCPCS) codes 37215 and 37216, which were assigned starting in 2004.¹¹ We included only Medicare beneficiaries aged 65 years or older who were continuously enrolled in fee-for-service programs for at least 1 year prior to their procedure. We excluded any patients treated by operators with evidence of having first performed carotid stenting in the last 2 quarters of the study period, given limited follow-up.

For our analyses examining experience at the time of the procedure, we further restricted our study cohort to patients who underwent carotid stenting by new operators who first performed carotid stenting after March 17, 2005, the date of its initial national coverage decision by the CMS.¹² Any operators performing carotid stenting prior to this date were considered experienced because the CMS previously had covered carotid stenting in Medicare beneficiaries only when it was performed within an FDA-approved study protocol. As such, these operators may have been performing the procedure for several years. In addition, we searched for and excluded any additional operators listed as investigators in the Global Carotid Artery Stent Registry, SAPPHIRE trial, and CREST trial,^13- 15 3 large multicenter studies of carotid stenting in the United States that had been active prior to the national coverage decision by the CMS.

The institutional review board of the University of Michigan approved this protocol before its initiation. The requirement for informed consent was waived.

The primary outcome of interest for our analysis was 30-day mortality following the procedure, which was determined from the Denominator files. The secondary outcome of interest was failure to receive an embolic protection device during the procedure. Data on failure to receive an embolic protection device were obtained using the HCPCS codes, which identified patients as having undergone carotid stenting either with or without an embolic protection device. We evaluated the use of embolic protection devices because their use has been linked to better outcomes¹⁶ and is currently required by the CMS for reimbursement.¹⁷

For our analysis, we evaluated 2 independent variables of interest that each reflected separate but related aspects of operator experience: annual volume and experience at the time of the procedure. Annual operator volume was assessed by first identifying the total number of procedures performed among Medicare beneficiaries during the study period for each operator. We annualized this number by determining the length of time in days from when that operator's first procedure was performed to the end of the study period, standardizing that relationship to a 12-month period.

To estimate operator experience at the time of the procedure, we restricted our study cohort to patients treated by new operators who first performed carotid stenting after the national coverage decision by the CMS (see aforementioned details). For each patient, we used the carotid stenting case number of that individual after ranking all procedures by their operator sequentially over the study period (ie, the operator's first, second, third . . . or n th procedure). In instances where more than 1 operator was involved (n = 223), we assigned the experience of the operator who had performed the higher number of procedures up to that time. When more than 1 procedure was performed on the same day by the same operator (n = 5373), all patients on that day were assigned a mean rank of experience for that day. Finally, we included only the index procedure in the analysis of outcomes, although repeat procedures (n = 1757) were included in calculations of experience.

Additional information on age, sex, and race were obtained using the Denominator files. From the Physician Carrier and MEDPAR files, we identified Elixhauser comorbidity conditions¹⁸ using International Classification of Diseases, Ninth Revision, Clinical Modification diagnostic codes from all claims submitted during the 12 months preceding the procedure. We then collapsed these conditions into a comorbidity score for each patient using a previously validated point system.¹⁹ We also used these data sources to determine if a patient had been hospitalized or had an outpatient visit during which a diagnosis of acute stroke or transient ischemic attack was recorded in the 180 days prior to carotid stenting. We specifically chose a 180-day period given that this was the length of time used by the CREST investigators to define symptomatic patients.¹⁵ Similarly, we identified if a patient had undergone carotid endarterectomy in the 1 year prior to carotid stenting. Finally, we determined the specialty of the operator performing carotid stenting using Medicare specialty codes and the 2007 Unique Physician Identification Number Directory.²⁰

We used descriptive statistics to evaluate differences in baseline characteristics after stratifying patients into 4 categories of annual operator volume: less than 6, 6 to 11, 12 to 23, and 24 or more procedures per year. We then used multivariable logistic regression models to assess the relationship between the volume categories and 30-day mortality. Models adjusted for age (65-69 years, 70-74 years, 75-79 years, 80-84 years, and ≥85 years), sex, race (black or nonblack), Elixhauser comorbidity score, presence of acute stroke or transient ischemic attack in the 180 days prior to carotid stenting, presence of a carotid endarterectomy in the 1 year prior to carotid stenting, and date of operator's first procedure during the study period. To account for the inherent clustering within these data (ie, the same operator performs multiple procedures over time), models were constructed using the framework of multilevel modeling with random intercepts included for individual operators.²¹

Similar models were constructed after restricting our study cohort to patients who underwent carotid stenting by new operators who first performed carotid stenting after the national coverage decision by the CMS. We compared 30-day mortality between patients treated early vs late during their operator's experience after stratifying them into 2 categories: 1st to 11th procedure and 12th procedure or higher. To understand the extent to which our findings were being driven by operators who performed a limited number of cases during the study period, we repeated these analyses after further restricting our study cohort to patients treated by new operators who had performed at least 12 procedures during the study period. Finally, additional models were constructed to assess the relationship between both measures of operator experience and failure to receive an embolic protection device.

For each outcome we assessed, we report risk-adjusted odds ratios (ORs) and their corresponding 95% confidence intervals. Overall, our final risk-adjusted models for 30-day mortality had reasonable discrimination, with C statistics of 0.75 for those that evaluated annual operator volume and 0.80 for those that assessed operator experience at the time of the procedure. All analyses were performed using SAS version 9.2 (SAS Institute Inc, Cary, North Carolina) and Stata version 11.0 (Stata Corp, College Station, Texas). P <.05 was considered statistically significant and all tests were 2-sided.

We identified 24 701 patients who underwent carotid stenting by 2339 operators. Of these, 11 846 procedures were performed by 1792 operators who first performed carotid stenting after its national coverage decision. Mean age of the study cohort was 76.2 years (SD, 6.4 years), 9917 (40.2%) were women, and 1056 (4.3%) were black. Overall, 461 patients (1.9%) died within 30 days of their procedure and 1173 patients (4.8%) did not receive an embolic protection device. The median annual operator volume in Medicare beneficiaries during the study period was 3.0 per year (interquartile range, 1.4-6.5). A total of 639 operators (27.3%) performed 6 or more procedures per year and 272 operators (11.6%) performed 12 or more procedures per year (Figure 1). Table 1 shows characteristics of patients across different categories of annual operator volume. Compared with patients treated by high-volume operators, patients treated by very low-volume operators were younger, were more often black, had a higher number of Elixhauser comorbidity conditions, and were more likely to have had an acute stroke in the 180 days prior to carotid stenting and to have had carotid endarterectomy in the 1 year prior to carotid stenting.

We found higher 30-day mortality in patients treated by operators with lower annual volumes of carotid stenting (Figure 2A). Observed 30-day mortality was 2.5% (95% CI, 2.1%-2.9%), 1.9% (95% CI, 1.6%-2.3%), 1.6% (95% CI, 1.3%-1.9%), and 1.4% (95% CI, 1.1%-1.7%) across the 4 categories (P < .001). This difference remained statistically significant after multivariable adjustment (Table 2). For example, compared with patients treated by operators performing 24 or more procedures per year, those treated by operators performing less than 6 procedures per year had an adjusted OR of 1.9 (95% CI, 1.4-2.7; P < .001) for 30-day mortality. A similar relationship was noted when we examined rates of failure to receive an embolic protection device. For example, the adjusted OR was 8.1 (95% CI, 4.4-14.9) for failing to receive an embolic protection device for patients treated by operators performing less than 6 procedures per year compared with those treated by operators performing 24 or more procedures per year (P < .001) (Table 2).

We also found higher 30-day mortality in patients treated early vs late during a new operator's experience (2.3% [95% CI, 2.0%-2.7%] vs 1.4% [95% CI, 1.1%-1.9%], respectively; P < .001) (Figure 2B). This difference also remained statistically significant after multivariable adjustment. For example, compared with patients who were their operator's 12th procedure or higher, those who were among their operator's first 11 procedures had an adjusted OR of 1.7 (95% CI, 1.2-2.4) for 30-day mortality (P = .001) (Table 2). These findings remained largely unchanged during sensitivity analyses that restricted the study cohort to patients treated by new operators who performed 12 or more procedures during the study period (adjusted OR, 1.6; 95% CI, 1.1-2.3; P = .02). Failure to receive an embolic protection device also was more common early during a new operator's experience. For example, the adjusted OR for failing to receive an embolic protection device was 4.8 (95% CI, 3.4-6.8) in patients who were among their operator's first 11 procedures compared with those who were their operator's 12th procedure or higher (P < .001) (Table 2).

We found a 30-day mortality of nearly 2% among Medicare beneficiaries undergoing carotid stenting. Mortality rates for elderly patients in contemporary clinical trials and registries are closer to 1%.^14- 15,22 Although the higher mortality rates we identified are likely being driven to a large extent by an older and less selected population of patients, we identified an additional factor that may be contributing: limited operator experience with carotid stenting as the procedure has disseminated into routine clinical practice. Indeed, we found that fewer than 1 in 8 operators had annual operator volumes of 12 procedures or more during the study period. Furthermore, we noted that patients treated by very low-volume operators and those treated early during a new operator's experience had significantly higher 30-day risk-adjusted mortality.

Ensuring that physicians are adequately experienced to perform innovative and technically-complex procedures, like carotid stenting, is not a new challenge.²³ It has been a factor in several health technologies in the past, most notably following the introduction of laparoscopic procedures in the late 1980s.^24- 25 Yet, ensuring expertise with carotid stenting also raises concerns that are particularly unique to this procedure. For example, carotid stenting is performed by specialists from diverse fields including cardiology, radiology, and surgery.²⁶ These physicians have widely varying clinical backgrounds and technical skills that make it hard to standardize educational programs. An expanded pool of physicians capable of performing it also may make carotid stenting more difficult to concentrate expertise among a few operators, especially given strong interest in the procedure by all 3 specialties. Of course, making policy decisions about restricting use of carotid stenting to highly experienced operators is complicated and involves balancing safety concerns with the potential long-term harm of limiting access to an innovative procedure early during its dissemination.

To a certain extent, the challenge of ensuring adequate expertise among operators has been widely recognized by professional organizations and regulators. For example, professional organizations representing the major specialties involved in carotid stenting have listed specific criteria to guide facilities in credentialing individual operators.^6- 8 These include minimum volume requirements suggested for operators, although specific thresholds vary across groups. The FDA has also encouraged educational initiatives, and in particular, the development of dedicated virtual simulation technologies.^27- 28 Recent work indicates that these initiatives may minimize differences in outcomes between operators with different levels of experience.²⁹ Yet the application of volume requirements in routine clinical practice, the quality of various educational initiatives, and the overall impact of both approaches on outcomes remain largely unknown.

In addition to differences in 30-day risk-adjusted mortality, we also found that failure to use embolic protection devices was more common among patients treated by lower-volume operators and earlier during a new operator's experience. Although we did not have sufficient clinical or anatomical information to identify why an embolic protection device may not have been used in a particular patient, a failure to receive these devices is a potentially important process measure that needs to better understood. It could be that as operators are gaining more skill with these devices, they are simultaneously and independently improving other procedural techniques that lead to better outcomes. However, it could also be that operators with more experience are better at selecting patients based on their suitability for embolic protection devices or even deferring carotid stenting when they cannot be used.

Our study should be interpreted in the context of some limitations. First, we examined carotid stenting in elderly Medicare beneficiaries. Although this age group represents approximately three-quarters of patients undergoing the procedure in the United States, our results may not be generalizable to younger patients. This also means that our determination of operator experience underestimates “overall” experience for any individual operator, especially if their case mix of Medicare beneficiaries differs substantially from other operators. As such, inference of a precise number of procedures that will be associated with better outcomes is not possible from this study. However, determining such a number may be less relevant than understanding the overall association between greater operator experience and outcomes with carotid stenting.

A second issue related to Medicare claims data is the potential for residual confounding, particularly given the minimal changes we found between unadjusted and adjusted ORs from our models. Because we were unable to account for several clinical and anatomical factors, it may be that patients who were treated by lower-volume operators or early during their operator's experience are sicker or more complex in unmeasured ways than those treated by high-volume operators. For example, overtime operators could gain experience not only in performing the procedure but also in selecting patients for it. Although this limits the ability to draw causal inferences from our analysis, the association we identified does point toward the need for further studies to understand potential reasons why outcomes were consistently worse among less experienced operators. Related concerns with using Medicare data include their limited ability to assess additional outcomes of importance (eg, stroke) or the procedure's overall appropriateness relative to alternative treatments, such as carotid endarterectomy or even medical therapy.

Third, our analyses examining early vs late experiences with carotid stenting in new operators is likely to have included some operators who performed carotid stenting prior to the date of the initial national coverage decision by the CMS. Carotid stenting has been described as long ago as the mid-1990s, although in the past its use was more limited. However, we suspect that any misclassification of operators, if present, biased our findings toward the null.

Fourth, we examined the association between these 2 measures of operator experience and outcomes across a large number of physicians. Although our findings represent an “average” effect, studies of the volume-outcome relationship and performance improvement in other areas suggest that individual operators develop and maintain their skills at varying rates,³⁰ and it is even possible that this relationship could vary based on their prior experiences with other endovascular procedures.

In conclusion, many physicians have begun performing carotid stenting in Medicare beneficiaries during recent years, although most operators appear to have developed limited experience with the procedure over time. This finding is important since adjusted outcomes following the procedure are worse among very low-volume operators and early during an operator's experience. Given limitations of these data, caution should be exerted when using our findings to set specific targets for operator experience. Nevertheless, collecting more detailed data about operator experience during the early dissemination of new procedures, like carotid stenting, may help optimize outcomes.