Impact of Risk-Adjusting Cesarean Delivery Rates When Reporting Hospital Performance FREE

CESAREAN DELIVERY is currently the most commonly performed major surgical procedure in the United States. Between 1972 and 1992, the proportion of infants delivered via cesarean increased roughly 4-fold, making rates in the United States among the highest in the industrialized world.^1- 2 These data have raised questions regarding the appropriateness of current practices.^3- 4 Moreover, the higher costs associated with cesarean delivery have led to efforts by insurers, managed care organizations, and others to decrease rates.^5- 6 As a result, hospitals and health plans are often ranked on the basis of cesarean delivery rates, with the implicit assumption that lower rates reflect more appropriate, as well as more efficient, clinical practice. For example, the Public Citizen's Health Research Group recently reported cesarean delivery rates for 3159 hospitals in 41 states and the New England HEDIS Coalition reported rates for 15 health plans in 1993.^7- 8

Cesarean delivery is indicated for many clinical scenarios, and numerous studies have identified patient-specific factors, such as maternal age, breech presentation, placental abnormalities, and fetal distress, that are epidemiologically related to increased likelihood of a cesarean delivery.^9- 20 However, most hospital and health plan comparisons of cesarean delivery rates have not accounted for such factors. For example, the New England HEDIS Coalition Report stratified rates only by maternal age, and the Public Citizen's Health Research Group reported rates in all patients and in women with a prior cesarean delivery.⁷

Failure to account for patient-specific risk factors for cesarean delivery may lead to biased comparisons if such risk factors vary among patients treated by different health care providers. This may be particularly problematic for hospital comparisons, given the wide variation in sociodemographic characteristics of patients at different hospitals and the referral bias that may result from differences in the availability of clinical services for women with high-risk pregnancies.

We conducted the current study to determine the impact of adjusting cesarean delivery rates for patient-specific risk factors in 21 hospitals serving a single large metropolitan area.²¹ We specifically sought to determine the correlation between hospital rankings before and after adjusting for clinical factors that increased the likelihood of cesarean delivery in women who had not undergone a prior cesarean delivery. For each patient in the analysis, maternal and neonatal risk factors were identified from standardized reviews of patients' medical records.

The study was conducted in 21 hospitals in the Cleveland, Ohio, metropolitan area that provide obstetrical services. Total number of deliveries during the study period per hospital ranged from 340 to 8933 (median, 2311). Five hospitals were classified as teaching hospitals, including 4 hospitals with accredited residency training programs in obstetrics and gynecology and 1 hospital in which residents from other clinical services rotated on the obstetrical service. The remaining 16 hospitals were classified as nonteaching hospitals. All hospitals were participants in Cleveland Health Quality Choice, a regional coalition of employers, hospitals, and physicians to compare the quality of hospital-based services.²¹

The study sample included 26127 women without prior cesarean deliveries who were admitted for labor and delivery to the study hospitals during a 30-month period from January 1993 through June 1995. Patients who were undergoing therapeutic abortions or who were not delivered of at least 1 neonate weighing 500 g or more were ineligible for data collection.

All study data were obtained from Cleveland Health Quality Choice. Data included information that was abstracted from patients' hospital records on standard forms by medical records technicians at each hospital. Data did not include unique patient identifiers; thus, the study protocol was exempt from institutional review board approval.

In the 11 hospitals in which the annual number of total deliveries (deliveries in women with and without prior cesarean deliveries) was projected to be less than 1200 (based on prior patient volumes), complete study data were collected for all 14155 eligible patients admitted during the study. In the 10 hospitals with 1200 or more projected annual deliveries, complete study data were collected only for a random sample of eligible patients (n=11972), to decrease the data collection burden. These hospitals were instructed to abstract medical records of all patients with deliveries on days randomly selected by Cleveland Health Quality Choice. The number of days for which hospitals abstracted records was dependent on projected volumes and was designed to achieve annual samples of roughly 1000 deliveries per year (including women both with and without prior cesarean deliveries). During the study, the randomly selected sample represented 26% of all eligible deliveries in hospitals in which sampling was performed, and ranged from 16% to 49% of eligible deliveries in the 10 hospitals.

Data elements included maternal demographics, comorbid illnesses existing prior to the current pregnancy (eg, diabetes mellitus), pregnancy-induced medical illnesses (eg, hypertension), obstetrical conditions (eg, abruptio placentae, cord prolapse, breech presentation of neonate), maternal and neonatal clinical findings (eg, maternal admission blood pressure and hemoglobin level, admission fetal heart rate, neonatal weight), and maternal and neonatal outcomes (eg, type of delivery, neonatal disposition). Procedures to ensure data reliability have been reported previously.^21- 22

Bivariate associations between maternal and neonatal risk factors and cesarean delivery were examined using the χ² test for categorical variables and the t test for continuous and ordinal variables. A multiple logistic regression model was fit to the 39 risk factors associated (P<.1) with the cesarean delivery in bivariate analyses (Table 1). For the logistic regression model, continuous variables (eg, maternal age, fetal heart rate) were categorized and examined as both ordinal variables and as sets of indicator (ie, dummy) variables. Variable classifications that maximized model fit (ie, discrimination and calibration) were retained.

In developing the risk-adjustment model, we included only clinical risk factors and did not include demographic factors such as race and insurance. Inclusion of indicator variables for race and insurance subgroups did not improve risk-adjustment model discrimination or calibration, and analyses based on such a model yielded generally similar study findings. Additionally, because our intent was to use the model to adjust hospital cesarean delivery rates in the current population, and not to export the model for use in other patient populations, we chose not to partition the sample into separate development and validation cohorts and fit models to each cohort. However, given the large sample size of the current study, it is likely that such models would have been similar. Moreover, use of a multivariable risk-adjustment model that included only factors that were statistically significant (P<.05) in logistic regression analyses yielded essentially similar findings.

Model discrimination was assessed by the c statistic,²³ which represents, for all possible pairs of patients who did and did not undergo a cesarean delivery, the proportion of times the patient who had a cesarean delivery had a higher predicted risk of cesarean delivery. The c statistic of the logistic regression model was 0.84, indicating excellent discrimination. Model calibration was assessed by the Hosmer-Lemeshow test, which compares observed and predicted cesarean delivery rates in 10 deciles based on the predicted cesarean delivery risk.^23- 24 The observed number of cesarean deliveries differed (P<.05) from the number predicted by the multivariable model in only 2 deciles of predicted risk. However, because the Hosmer-Lemeshow test was significant (χ²=43.4, df=8, P<.001) and because the model included only dichotomous variables, additional models were evaluated that included first-order interaction terms of the 5 strongest variables, as estimated by Wald χ² statistics. Inclusion of the interaction terms did not improve model discrimination or calibration. The final risk-adjustment model included 44 indicator variables representing 39 distinct risk factors (Table 1).

The risk-adjustment model was used to determine a predicted risk of cesarean delivery (0%-100%) for each patient. Predicted risks in individual patients were aggregated by hospitals to determine mean predicted risks for each hospital. Risk-adjusted hospital cesarean delivery rates were determined by dividing the observed (ie, unadjusted) hospital cesarean delivery rate by the mean predicted hospital cesarean delivery rate, and multiplying that quantity by the observed cesarean delivery rate in the entire study sample. Variability in unadjusted and adjusted cesarean delivery rates across individual hospitals was examined by comparing rates in individual hospitals with the rate in the study sample using a 1-sample test of proportions. Hospitals with rates higher or lower (P<.05) than the overall rate were classified as outliers. Additionally, correlations between unadjusted and adjusted hospital rankings (ie, 1-21) were examined using the Spearman correlation coefficient. All analyses were conducted using SAS, Version 6.12 (SAS Institute Inc, Cary, NC).

The mean (SD) age of the 26127 patients in the study sample was 27.3 (6.0) years. Seventy-eight percent of patients were white and 20% were African American; in 2% of patients, race was classified as other or unknown. Sixty-five percent of patients had commercial insurance (including indemnity and managed care plans), 30% had Medicaid or another governmental form of insurance (eg, Medicare, county assistance), and 5% were uninsured. The overall cesarean delivery rate was 15.9%, and 1.5% of patients were delivered of more than 1 neonate. Labor was induced in 12.8% and 3.5% of women undergoing vaginal delivery and cesarean delivery, respectively. Ninety-six percent of patients received some prenatal care; nearly three fourths (73.5%) of these patients received prenatal care during the first trimester. Histories of long-term tobacco, alcohol, or illegal drug use were documented in 24.8%, 7.3%, and 2.9% of patients, respectively. Histories of hypertension and diabetes prior to the current pregnancy were documented in 0.8% and 0.4% of patients, respectively, and pregnancy-induced hypertension and gestational diabetes were documented in 4.0% and 3.5% of patients, respectively. Neonatal mortality was 0.4% and maternal mortality was 0.01%.

Bivariate and multivariate odds ratios (ORs) of variables included in the logistic regression model are shown in Table 1. Multivariate ORs were highest for breech presentation (OR, 58.2) and face or transverse presentation (OR, 21.6). Prevalence of the clinical risk factors varied across hospitals (Table 1). For example, the frequency of breech presentation in the 21 hospitals varied more than 3-fold (2.4%-7.8%, P=.001).

Observed cesarean delivery rates varied (P<.001) across hospitals, and 7 hospitals were statistical outliers with respect to observed (ie, unadjusted) rates. Four hospitals had rates that were higher (P<.05) than the overall rate of 15.9%; rates in these hospitals were 26.5%, 21.1%, 17.9%, and 17.9%. Three hospitals had rates that were lower (P<.05) than the overall rate; rates in these hospitals were 6.3%, 12.2%, and 13.0%.

Predicted rates of cesarean delivery ranged from 12.0% to 20.5% (P<.001), suggesting that patient mix differed across hospitals with respect to factors that may increase the risk of cesarean delivery. Risk-adjusted rates varied (P<.001) from 8.4% to 22.0%. After risk adjustment, 7 hospitals were also classified as outliers, including 4 high and 3 low outliers. However, outlier status changed for 5 (24%) of the 21 hospitals, including 1 that changed from a high outlier to a nonoutlier, 1 that changed from a low outlier to a nonoutlier, 2 that changed from nonoutliers to high outliers, and 1 that changed from a high outlier to a low outlier (Figure 1).

Differences between unadjusted and risk-adjusted hospital cesarean delivery rates ranged from−2.9% to 4.5%. The absolute difference in rates was 2% or greater for 10 hospitals (48%). Although differences tended to be somewhat larger in hospitals with the most extreme unadjusted rates, the impact of risk adjustment could be observed across the spectrum of observed rates. There was no association between obstetrical volume (ie, total number of deliveries) and unadjusted (R=0.10, P=.67) or adjusted (R=−0.09, P=.69) cesarean delivery rates, or between obstetrical volume and the difference between unadjusted and adjusted cesarean delivery rates (R=0.14, P=.56). Finally, when hospitals were rank-ordered (ie, 1-21) on the basis of cesarean delivery rates (Figure 1), the correlation between unadjusted and adjusted rankings was modest and not statistically significant (R=0.35, P=.12).

The current study examined the impact of adjusting hospital cesarean delivery rates for risk factors that may increase the likelihood of a cesarean delivery, and was based on clinical data abstracted in a standardized manner from patients' medical records. Analyzing a community-based sample of 26127 women who had not had a prior cesarean delivery and who were admitted for labor and delivery to any of 21 different hospitals, we found that unadjusted hospital cesarean delivery rates exhibited roughly a 4-fold variation and that a third of the hospitals were classified as statistical outliers. However, after adjusting hospital rates for the presence of 39 clinical factors that may predispose to cesarean delivery, outlier status changed for 5 of the 21 hospitals. Additionally, the prevalence of the 39 risk factors varied across hospitals, and predicted rates of cesarean delivery based on the 39 factors varied by more than 60%. These findings suggest that hospitals within a single metropolitan region care for different patient populations and that obstetrical case mix varies. Finally, when hospitals were rank-ordered on the basis of cesarean delivery rates, the correlation between unadjusted and adjusted rankings was only moderate. Taken together, the results suggest that hospital comparisons that do not account for differences in patient mix may be methodologically biased and may provide consumers and other health care purchasers with misleading information.

The current findings are consistent with prior epidemiological studies of clinical factors associated with cesarean delivery. Several investigators have demonstrated the importance of maternal age, parity, comorbidity (eg, diabetes, hypertension) and infant birth weight.^{9,13- 14,18- 20} Other factors, such as complications of pregnancy (eg, breech presentation, placenta previa, umbilical cord prolapse) and fetal distress, are also important.^5,20,25- 27 Although dystocia is a major risk factor and one of the most common clinical indications for cesarean delivery, we did not consider this factor because of the lack of a standard clinical definition and the likely variability in how the term is applied in different hospitals. Finally, whereas several studies have demonstrated associations with nonclinical factors such as race, type of health insurance, and time of hospital admission,^26,28- 35 we chose not to account for such factors because of the lack of clinical or biological evidence to suggest their importance as risk factors and because such factors may be associated with differences in health care delivery.

Several investigations have proposed multivariable models for estimating the likelihood of cesarean delivery based on a variety of data sources, including administrative (ie, claims) data, birth certificate data, and medical records information.^5,18- 19,25 Some of these analyses have demonstrated persistence of differences in cesarean delivery rates in specific patient populations (ie, uninsured patients) after applying a risk-adjustment model. Although the importance of risk-adjusting other hospital indicators such as mortality rates and length of stay is well established,^36- 42 to our knowledge, only 1 prior study has directly examined the impact of risk adjustment on cesarean delivery rates in individual hospitals.¹⁸ This study examined singleton births of more than 2500 g in 80 hospitals in Washington State in 1989 and 1990, using a combination of administrative and birth certificate data. The authors stratified their sample into 4 groups (prior cesarean delivery, malpresentation without prior cesarean delivery, first birth without malpresentation, and all other deliveries) and developed separate multivariable models for each group, based on clinical variables documented in birth certificates that were similar to the variables in our model. Using these models to adjust cesarean delivery rates in individual hospitals, the authors concluded that differences in unadjusted and adjusted rates were "small," although no data were provided on the absolute magnitude of the differences or on the impact of risk adjustment on statistical categorizations of hospital rates.

In contrast, we found that hospital rankings often changed and that in 12 hospitals, the relative difference in unadjusted and adjusted rates (ie, [unadjusted−adjusted]/unadjusted) was greater than 10%. Moreover, statistical outlier status, which is frequently used in hospital "report cards," was also affected by risk adjustment. Using a criterion of P<.05, which is often used to denote higher- or lower-than-expected outcomes,⁷ outlier status changed in nearly a fourth of the hospitals, with 1 hospital changing from a high outlier to a low outlier.

In interpreting our findings, it is important to consider several potential methodological limitations. First, although a strength of our study is its use of clinical data that were reliably abstracted from patients' medical records,²² hospitals may systematically vary in the thoroughness with which clinical findings are noted in the medical record. Thus, some of the variation in prevalence of clinical findings across hospitals may reflect differences in documentation. Second, we did not independently determine the appropriateness or medical necessity of cesarean delivery. In identifying risk factors, we chose factors that were used in prior studies, and on the basis of their empirical associations with cesarean delivery in the current population. However, the indications for cesarean delivery surrounding many of the risk factors studied remain controversial. For example, studies have demonstrated that external cephalic version can allow for vaginal delivery in many cases of malpresentation.⁴³ Moreover, because of external pressures, cesarean delivery rates have progressively fallen in recent years.² Such changes in obstetrical practice may mitigate the importance of individual risk factors examined in the current study in future years. Our study did not account for the possible influence of practitioner characteristics (eg, physician vs nurse/midwife and obstetrician vs family practitioner). In addition, although we examined a broad spectrum of teaching and nonteaching hospitals in a large metropolitan region, the generalizability of our findings to other regions should be established, particularly to those regions with different penetration of managed care and prevalence of capitation.

In summary, provider performance is increasingly being scrutinized by consumers and purchasers, and comparative report cards are often publicized. Although the clinical appropriateness of cesarean delivery is rarely measured, cesarean delivery rates remain a commonly used yardstick for comparing hospitals and health plans. Whereas many professional societies⁴⁴ readily acknowledge that cesarean delivery rates in the United States can be safely reduced, our findings suggest that hospital comparisons or profiles of cesarean delivery rates that fail to account for patient characteristics that increase the likelihood of cesarean delivery may be methodologically biased and may mislead health care purchasers.