Hospital-Level Compliance With Asthma Care Quality Measures at Children's Hospitals and Subsequent Asthma-Related Outcomes FREE

Joint Commission–accredited hospitals submit process measure compliance data, many of which are publicly reported, for a variety of common diagnoses, such as acute myocardial infarction, congestive heart failure, and pneumonia.¹ Until recently, none of the more than 50 Joint Commission core measures evaluated care provided to hospitalized children.

To address this shortcoming, the Joint Commission, in collaboration with the Child Health Corporation of America (CHCA), the National Association of Children's Hospitals and Related Institutions (NACHRI), and Medical Management Planning Inc, developed the Children's Asthma Care (CAC) measure set.² This set of process measures evaluates at the hospital level whether patients aged 2 to 17 years admitted with an asthma exacerbation received relievers (CAC-1) and systemic corticosteroids (CAC-2) during the admission and whether they were discharged with a complete home management plan of care (HMPC) (CAC-3). The CAC measures were endorsed by the National Quality Forum and became the first core measure set specifically designed to evaluate the quality of care provided to hospitalized children.^3- 4

Process measures are designed to evaluate compliance with care that is expected and, when provided, should “maximize health benefits to patients” and lead to improved clinical outcomes.^5- 7 Compliance of health care organizations with many hospital-level process measures has improved over time.^5,8- 9 However, their association with improved outcomes has been variable. For example, acute myocardial infarction mortality rates decreased but congestive heart failure mortality rates did not during periods when compliance with their respective process measures increased.^9- 12

The objectives of this multicenter study were to evaluate longitudinal trends in CAC measure compliance at children's hospitals and to determine the association between CAC measure compliance and improvements in clinical outcomes.

Data for this cross-sectional study were obtained from 2 sources, NACHRI's Pediatric Quality Measurement System and the Pediatric Health Information System (PHIS). Quarterly hospital-specific CAC measure compliance data were provided by NACHRI. Participating hospitals collect and transmit CAC measure data to NACHRI from a sample of randomly selected pediatric inpatients aged 2 to 17 years with a principal discharge diagnosis of asthma (International Classification of Diseases, Ninth Revision [ICD-9 ] code 493.XX). Children's Asthma Care measure inclusion criteria are discharge to home, including patients who had spent time in an intensive care unit; length of stay not greater than 120 days; and no enrollment in a clinical trial.¹³ Data quality is monitored and ensured by the Joint Commission through its quality measurement vendor evaluation process.¹⁴

Patients who met CAC measure inclusion criteria and were admitted between January 1, 2008, and September 30, 2010, were included. Patients were followed up through December 31, 2010. Patient-level data to evaluate postdischarge asthma-related emergency department (ED) utilization and hospital readmission were obtained from the PHIS, which contains data from 42 freestanding, tertiary care children's hospitals. Participating hospitals are located in 27 states plus the District of Columbia. These hospitals provide inpatient discharge data, including patient demographics, diagnoses, and procedures.

In the PHIS database, race is categorized as white, black, Asian, American Indian, or other; ethnicity is categorized as Hispanic, non-Hispanic, or unknown. Race and ethnicity were self-reported by patients or parents and included to describe the cohort. Data in PHIS are deidentified, but consistently encrypted medical record numbers allow for tracking individual patients across admissions. The CHCA and participating hospitals jointly ensure data quality as described previously.^15- 16

This research, using deidentified data sets, was not considered human subjects research in accordance with the Common Rule (45 CFR §46.102[f]) and the policies of Phoenix Children's Hospital and the Children's Hospital of Philadelphia institutional review boards.

Information on hospital-specific compliance with CAC-1, CAC-2, CAC-3, and CAC-3 subcomponent measures was obtained for the first quarter of 2008 through the third quarter of 2010. Quarterly CAC measure compliance data are determined retrospectively by medical record review of a random sample of records. A standardized data collection tool, provided by NACHRI, is used to determine if patients received (1) relievers during their hospital stay (CAC-1); (2) systemic corticosteroids during their hospitalization (CAC-2); and (3) an individualized HMPC (CAC-3). To achieve CAC-3 compliance, a copy of the HMPC needs to be present in the medical record, along with documentation that the plan was given to the family. In addition, the HMPC must address all of the following subcomponents: (1) arrangements for follow-up care (HMPC-1) (compliance requires either an appointment to be scheduled or information provided to the family to facilitate making arrangements for follow-up care); (2) environmental control and control of other triggers (HMPC-2); (3) method and timing of rescue actions (HMPC-3); (4) use of controllers (HMPC-4); and (5) use of relievers (HMPC-5).¹³

Our outcome measures were asthma-related (principal diagnosis ICD-9 code 493.XX) postdischarge ED utilization rates and hospital readmission rates at 7, 30, and 90 days. For each index admission, only the most immediate postdischarge outcome related to readmission was counted. For example, if a patient had multiple admissions during the study period, the second admission was counted as an outcome measure for the initial admission while a third admission was counted as an outcome measure for the second admission. The same method was used to determine postdischarge ED utilization rates. In addition, all ED visits were included in the ED utilization rates whether the patient was discharged to home or admitted. To determine whether patients with frequent readmissions substantively affected our results, we performed a sensitivity analysis by restricting the cohort to 1 randomly selected admission per patient.

Hospital- and patient-level covariates were summarized using medians with interquartile ranges (IQRs) for continuous variables and frequencies with percentages for categorical variables. We displayed temporal trends for CAC compliance and readmission rates quarterly using the mean hospital data with 95% confidence intervals and tested for trends with generalized linear mixed models clustered by hospital.

In our primary analyses, we tested whether CAC-3 compliance was related to any of the outcome measures using generalized estimating equations with a logit link function while clustering by hospital. We were not able to associate CAC-3 compliance with individual patients, so we used CAC-3 compliance as a hospital-level exposure in the model. We incorporated a previously published pediatric-specific readmission prediction model for risk adjustment.¹⁷ This model includes factors from a patient's index encounter (eg, demographics, all-patient refined diagnosis related group severity levels, and complex chronic conditions) as well as factors related to the year prior to the patient's index encounter (eg, lag time from prior to index encounter, prior discharge disposition). The model exhibits a C statistic of 0.81.

From this risk-adjusted model, we estimated the odds ratio and 95% confidence interval of experiencing the outcome for a 5% increase in CAC-3 compliance. A 5% increase in CAC-3 compliance was determined a priori to be a meaningful increase in compliance. We used a conservative Bonferroni correction (.05/6 comparisons) whereby P < .008 was considered statistically significant. We based the power for our study on the relationship between CAC-3 compliance and 30-day readmission. For a likelihood ratio χ² test of CAC-3 compliance, which is distributed as normal (0.6, 0.3), in logistic regression with a significance level of .05, a sample size of 24 246 is required to obtain a power of at least 0.9 to detect a response odds ratio of 1.02 for a change of 5% in CAC-3 compliance.

Besides our primary analysis, 3 additional analyses were performed to further explore potential associations between CAC-3 compliance and outcomes. First, we evaluated the subcomponents of CAC-3 compliance using similar models. For this analysis, since we were testing 6 outcomes with 5 different CAC-3 subcomponents, we adjusted the significance level to P < .002 (.05/30 comparisons).

Second, to determine if hospitals with the highest CAC-3 compliance had superior outcomes compared with other hospitals, we aggregated the most recent 3 quarters of data and compared outcomes between hospitals in the highest and lowest quartiles of CAC-3 compliance. We calculated aggregate return rates with 95% exact binomial confidence intervals for hospitals in each group. Comparisons were then made between the rates using a generalized linear mixed-effects model controlling for hospital clustering. Applying the Bonferroni correction (.05/6 comparisons), P < .008 was considered statistically significant.

Finally, to determine whether individual hospitals experienced an improvement in outcomes associated with their change in CAC-3 compliance, we built hospital-specific models using logistic regression. Once again, we used a Bonferroni correction (.05/6 comparisons), and P < .008 was considered statistically significant within each hospital.

All statistical analyses were performed using the statistical software SAS version 9.2 (SAS Institute Inc, Cary, North Carolina).

NACHRI has 195 member organizations, including all 50 freestanding children's hospitals located in the United States. Forty-two of the 50 freestanding children's hospitals (84%) contribute data to the PHIS, the outcome data source for this study. Of the 33 PHIS hospitals that also contribute ED data to PHIS, 30 (60% of all freestanding children's hospitals) also submitted CAC data to NACHRI and were included after consenting to inclusion in this study. The 30 freestanding children's hospitals included in this study had a median of 275 (IQR, 219-326) hospital beds and were located throughout the US census regions: Northeast (n = 3), South (n = 10), North Central (n = 9), and West (n = 8).

There were a median of 1219 (IQR, 801-1698) asthma-related admissions per hospital during the study period. The median number of annual ED visits across hospitals was 48 532 (IQR, 36 798-78 587). The overall study population (Table 1) consisted of 37 267 children with 45 499 asthma hospital admissions and had a median age of 6.1 years (IQR, 3.7-9.6 years). The median length of stay was 2.0 days (IQR, 1.0-3.0 days).

The minimum quarterly compliance rates that a hospital reported for CAC-1 and CAC-2 measures were 97.1% and 89.5%, respectively. Individual hospital-level CAC-2 compliance exceeded 95% during 97.9% of the quarters. While overall CAC-3 compliance improved during the study period, there was interhospital and temporal variation (Figure 1). Mean CAC-3 compliance was 40.6% (95% CI, 34.1%-47.1%) during the initial 3 quarters and improved to 72.9% (95% CI, 68.8%-76.9%) during the final 3 quarters of the study.

Because the overall CAC-3 measure is an “all-or-nothing” measure, compliance with each of the individual CAC-3 subcomponents was higher than overall CAC-3 performance. Aggregate hospital compliance with the CAC-3 subcomponents followed a temporal pattern that was similar to the overall CAC-3 compliance (eFigure). Compliance was lowest in addressing arrangements for follow-up care, with mean compliance ranging from 55.5% (95% CI, 48.9%-62.1%) in the first quarter of 2008 to 80.0% (95% CI, 76.6%-84.6%) in the third quarter of 2010. Compliance was highest in addressing use of relievers, with mean compliance ranging from 64.2% (95% CI, 58.3%-70.0%) in the first quarter of 2008 to 87.4% (95% CI, 85.6%-89.2%) in the third quarter of 2010.

Figure 2 shows the hospital aggregate postdischarge ED utilization and asthma-related readmission rates over time. The mean postdischarge ED utilization rates were 1.5% (95% CI, 1.3%-1.6%) at 7 days, 4.3% (95% CI, 4.0%-4.5%) at 30 days, and 11.1% (95% CI, 10.5%-11.7) at 90 days, while the mean quarterly readmission rates were 1.4% (95% CI, 1.2%-1.6%) at 7 days, 3.1% (95% CI, 2.8%-3.3%) at 30 days, and 7.6% (95% CI, 7.2%-8.1%) at 90 days. Quarterly postdischarge ED utilization and readmission rates at 7, 30, and 90 days did not change during the study period.

Compliance with CAC-1 and CAC-2 was high, with little variability across hospitals, which precluded examination of their association with the specified outcomes. Comparisons of the overall CAC-3 compliance with 7-, 30-, and 90-day postdischarge ED utilization and asthma-related readmission rates are shown in Table 2. We performed a sensitivity analysis by restricting the cohort to 1 randomly selected admission per patient and obtained similar results (eTable). In addition, the association of CAC-3 subcomponent compliance and the outcomes was also examined (Table 3). There was no significant association between CAC-3 compliance, overall or independently, for any of the subcomponents and either of the outcome measures at any of the 3 time intervals.

To evaluate the possibility that hospitals with the highest CAC-3 compliance may have had superior outcomes compared with hospitals with lower compliance rates, we compared hospitals in the highest- and lowest-performing quartiles of CAC-3 compliance. The quartile cut points in overall CAC-3 compliance were 84.1% for the highest quartile and 54.7% for the lowest. No significant differences in postdischarge asthma-related ED utilization or readmission rates were noted at 7, 30, or 90 days (Table 4).

Finally, we also performed individual analyses for each of the 30 hospitals to determine whether individual hospitals experienced improved outcomes associated with their CAC-3 compliance. One hospital was noted to have an associated lower 30-day postdischarge ED utilization rate with improved CAC-3 compliance (odds ratio, 0.43; 95% CI, 0.24-0.79; P = .006). None of the other hospital-specific models demonstrated a significant association between CAC-3 compliance and postdischarge ED utilization or readmission rates at 7, 30, or 90 days.

Compliance with the CAC measure set (CAC-1, CAC-2, and CAC-3) over a 33-month period was evaluated for 30 children's hospitals individually and in aggregate. Compliance with CAC-1 and CAC-2 measures remained high over time with little variation across hospitals, so no analysis could be performed to determine whether improved compliance led to improved outcomes. In contrast, aggregate CAC-3 compliance was initially modest but improved during the study period, with substantial variation in compliance and improvement among the hospitals. We did not find a statistically significant association between aggregate CAC-3 compliance and postdischarge ED utilization or asthma-related readmission rates at 7, 30, or 90 days. Nor did we find differences in outcomes when comparing hospitals in the highest and lowest quartiles of CAC-3 compliance or differences in outcomes with CAC-3 compliance at the individual hospital level. Since CAC-3 compliance at the hospital level is not associated with improved outcomes at the hospital level, our findings suggest that within children's hospitals, CAC measure set compliance alone cannot serve as a means to evaluate and compare the quality of care provided for patients admitted with asthma exacerbations.

In our study, aggregate CAC-1 and CAC-2 compliance was very high from the outset, with little room for improvement. As expected, these findings support the notion that within children's hospitals, use of bronchodilators and systemic corticosteroids has become standard practice for treatment of patients admitted with asthma exacerbations. Importantly, though, not every hospital providing pediatric care submits CAC measure data. Consideration should be given to ensuring that all hospitals providing pediatric care demonstrate, at least for a period of time, that the use of relievers and systemic corticosteroids is routine practice.

The inability to identify an association between CAC-3 measure compliance and outcomes, analyzed in a number of different ways, is an important finding given that the Joint Commission considers the CAC measure set an “accountability measure.” The Joint Commission embraces accountability measures as appropriate for use for public reporting and pay for performance. To be considered an accountability measure, compliance should lead to improved outcomes with little risk of unintended adverse consequences.⁵ In addition, the measure needs to accurately capture that care was delivered as intended and that a tight linkage with few intervening steps exists between the process being monitored and the improved outcome.⁵ While the health of children hospitalized with asthma clearly improves when they receive relievers (CAC-1) and systemic corticosteroids (CAC-2), the CAC-3 measure, in its current form, may not meet the criteria set out for accountability measures.

Our findings must be interpreted in the context of pediatric inpatient care and its follow-up. During a hospitalization, patients and families need to be educated about how to care for their asthma. In addition, appropriate outpatient follow-up care needs to be secured prior to discharge. The concept of implementing an HMPC prior to discharge makes intuitive sense. However, there are several possible reasons for the lack of an association between CAC-3 compliance and outcomes in this study. First, the CAC-3 measure does not adequately assess the quality and means of posthospital implementation of the HMPC. Instead, the CAC-3 measure demonstrates only that an HMPC document was completed with the CAC-defined components and provided to a patient. From this standpoint, the CAC-3 measure is similar to the congestive heart failure measure intended to evaluate whether a patient received education and discharge planning without considering the quality of the interventions.^5,18 Second, there may be too many intervening steps between the process being measured and the outcome. Even the best HMPCs do not guarantee adherence to prescribed medications after discharge, trigger avoidance, early recognition of symptoms of exacerbations, or primary care follow-up, all of which likely affect subsequent ED utilization and readmission rates. Finally, it is possible that HMPCs completed and provided in the inpatient setting do not lead to improved outcomes to the same degree as they have been demonstrated to do in other settings. Prior studies have not attempted to demonstrate an association between HMPCs provided in the inpatient setting and subsequent outcomes.^19- 21

There are several limitations worth considering. First, it is possible that we have not identified the appropriate outcome measures expected to be associated with improvements in CAC-3 compliance. The assumption that high-quality inpatient care leads to lower subsequent ED utilization and readmission rates in this patient population may be incorrect. Second, there was limited hospital-level variability in the individual outcomes. This, along with the fact that the readmission risk adjustment model we used was developed for all hospitalized children and not validated specifically for patients with asthma exacerbations, may have affected our ability to detect associations between CAC-3 compliance and outcomes. Third, because of the manner in which CAC data are collected, we used aggregate compliance rates rather than compliance at the individual patient level. Individual patients receiving an HMPC may have experienced improved outcomes, including the ones we measured or others that could not be evaluated using administrative data, such as greater medication adherence or increased symptom-free days. Similarly, some patients may have received a complete HMPC but the document was not present in the chart for retrospective review. This scenario would lead to underestimation of the percentage of patients who received an appropriate HMPC. Opportunities exist for further studies to attempt to associate use of an HMPC at the patient level in the inpatient setting with improved outcomes. Fourth, ED visits and hospital readmission could be measured only at the hospital where the index admission occurred. Follow-up care occurring at hospitals not included in the PHIS database was not counted. This limitation would underestimate the proportion of children requiring subsequent ED or hospital care. Finally, our study was performed using data from large, freestanding children's hospitals that contribute data to the PHIS. By submitting relevant data to the PHIS and by consenting to participate in the study, we may have selected hospitals that are inherently providing better quality of care than nonparticipating hospitals. It is possible that an association between CAC measure compliance and outcomes exists at other hospitals that provide care to children. Thus, our results may not be generalizable to all inpatient health care settings caring for children.

In conclusion, within children's hospitals, compliance with CAC-1 and CAC-2 measures has remained high with little variability over time. Continuing to collect data on CAC-1 and CAC-2 compliance within children's hospitals may not be valuable. Improvement over time has been demonstrated in CAC-3 compliance, but improved CAC-3 measure compliance was not associated with lower postdischarge ED utilization or asthma-related readmission rates. Consideration should be given to refining the CAC-3 measure set to ensure that high-quality HMPCs are being developed using evidence-based resources and that they are conveyed to families in an effective manner. Finally, the CAC-3 measure in its current form may not meet the criteria outlined by the Joint Commission for accountability measures. Until CAC-3 compliance can be linked to improved outcomes, the Joint Commission should reconsider whether the CAC-3 component of the measure set is appropriately classified as an “accountability measure” suitable for public reporting, accreditation, or pay for performance.