Enhancement of Claims Data to Improve Risk Adjustment of Hospital Mortality FREE

Risk-adjusted hospital mortality rates for specified conditions and procedures frequently are used in public reports and pay-for-performance programs as indicators of the quality of hospital care.^1- 3 Risk adjustment often is based solely on administrative claims data from uniform bills that hospitals submit to payers. These data lack clinically important pathophysiological information and do not distinguish between conditions that were present on admission (POA; ie, potential risk factors) and complications that occurred during hospitalization. The validity of risk-adjustment systems that use only administrative data has been challenged repeatedly,^4- 9 and there is general agreement^10- 23 that additional data are required to predict accurately an individual patient's risk of dying.

Physicians are particularly concerned that inadequate risk adjustment penalizes the practitioners and facilities that care for the sickest patients and may result in the denial of needed care to high-risk patients.^24- 26 Consumer advocates and payers, through initiatives such as the Consumer-Purchaser Disclosure Project,²⁷ are attempting to expand administrative data sets to include clinical information to ensure that incentives reward high-quality clinical care. On the other hand, many hospital administrators have complained that the cost of retrieving supplementary clinical data from medical records is prohibitive, and some researchers have argued that risk adjustment using only administrative data can be made sufficiently accurate to support valid comparisons among hospitals.^28- 29

The addition of a POA modifier for secondary diagnosis codes first was proposed in 1991³⁰ and was successfully adopted in New York State in 1994 and in California in 1996.³¹ The planned implementation in March 2007 of new standards for hospital claims data includes nationwide adoption of this modifier, which distinguishes conditions that develop during hospital stays (potential complications of care) from conditions that were present at admission (potential treatment-independent risk factors). Inclusion of POA codes in administrative data sets should permit analysts to incorporate important predictors of inpatient mortality into administrative risk-adjustment equations without improperly designating patients as having high intrinsic risks at admission when their increased vulnerability resulted from hospital-acquired complications.³¹

The adoption of Logical Observation Identifiers Names and Codes³² for laboratory data and advances in electronic health data technology have lowered the cost of retrieving numerical laboratory data at many hospitals.³³ Because of substantial differences in the cost of obtaining various types of clinical data, limited enhancement of administrative data sets appears to be both practical and desirable. Ideally, clinical data elements selected for this purpose will be relatively inexpensive to obtain and will be useful predictors of mortality for multiple conditions and procedures.

This study was designed to test the hypothesis that the combination of POA modifiers for secondary diagnoses and a limited set of numerical laboratory data would improve risk adjustment of inpatient mortality for a diverse set of clinical conditions and procedures. We also hypothesized that further additions of highly specific, difficult-to-obtain clinical data sometimes considered important predictors of inpatient mortality by clinicians would add little to the accuracy of predictive models.

Risk-adjustment models were created and analyzed using data from July 2000 through June 2003 from 188 Pennsylvania hospitals supplied by the Pennsylvania Health Care Cost Containment Council.³⁴ Case-level claims data were supplemented with clinical data abstracted from medical records by specially trained personnel using MediQual's proprietary Atlas clinical information system.³⁵ This system defines a broad array of clinical data elements, including historical information, laboratory results, vital signs, clinical symptoms and signs, pathophysiological abnormalities, and composite pathophysiological scores, which are collected and stored along with the hospital day on which each clinical finding was observed.

Risk-adjusted mortality rates were analyzed for 5 health conditions (acute myocardial infarction, congestive heart failure, acute cerebrovascular accident, gastrointestinal tract hemorrhage, or pneumonia) and 3 surgical procedures (abdominal aortic aneurysm repair, coronary artery bypass graft surgery, or craniotomy). The Agency for Healthcare Research and Quality's Inpatient Quality Indicator software version 2.1 was used to identify cases that met criteria for inclusion in each group.³⁶

Four models were constructed for each condition and procedure (1 set of data for each of the 5 conditions and 1 set of data for each of the 3 procedures). The first model termed administrative used standard claims data. The second model termed POA used data abstracted from medical records to determine whether coded secondary diagnoses had been present at admission. The third model termed laboratory used POA codes and numerical laboratory data (often available in electronic form; eg, creatinine, hematocrit level) documented on the first day of hospitalization prior to a procedure requiring general or regional anesthesia. The fourth model termed clinical used the criteria in the third model plus vital signs, other laboratory data not included in the third model (eg, bacterial culture results), Atlas key clinical findings abstracted from medical records (eg, immunocompromised, lethargy), and composite clinical scores (ie, American Society of Anesthesiologists classification, Glasgow Coma Score) documented on the first day of hospitalization prior to a procedure requiring general or regional anesthesia.

The administrative model was based solely on data from hospital bills (ie, age, sex, and principal diagnoses, secondary diagnoses, and procedures coded according to the International Classification of Diseases, Ninth Revision, Clinical Modification [ICD-9-CM]). To avoid using hospital-acquired complications as risk factors, hospital bills from New York and California (secondary diagnoses were modified by POA codes in these states) were used to help identify which secondary diagnoses were generally present at admission. Secondary diagnoses were eligible for inclusion as risk factors in the administrative model only when they were coded as hospital-acquired complications in fewer than 20% of cases in which they occurred.

The POA model included additional secondary diagnoses excluded from the administrative model because of their association with unacceptably high rates of complications. Because Pennsylvania claims data do not include POA codes, clinical data in the Atlas database were used to determine whether coded secondary diagnoses were present at admission. In the creation of surrogate POA codes, the Atlas database served as a substitute for the complete medical record available to coders in New York and California. For example, posthemorrhagic anemia was excluded from the administrative model for congestive heart failure because hospitals in New York and California coded it as acquired during hospitalization in more than 30% of the cases in which it occurred. However, posthemorrhagic anemia was eligible for inclusion as a risk factor in the POA model for congestive heart failure when the Atlas database documented that anemia was present on the day of admission.

For each condition or procedure, candidate risk factors were constructed from principal diagnosis codes, up to 8 secondary diagnosis codes, up to 6 procedure codes, and clinical data elements associated with higher than average mortality rates. Infrequently occurring codes were combined with codes for clinically similar conditions or procedures that had similar mortality rates. Continuous measures (eg, age, creatinine level) were transformed into 1 or more categorical variables based on clinical judgment and empirical evaluation of associated mortality rates.

For each condition or procedure, stratified random samples of live discharges and fatalities were combined to create 3 mutually exclusive data sets: a training set (50%), a validation set (25%), and a test set (25%). Partitioning the data in this way facilitated the construction of more robust models.

A preliminary predictive equation was developed on the training set for each condition or procedure using only age categories and individual hospital identifiers. (Including hospitals as risk factors during model development is a standard technique for reducing possible bias caused by the associations between the prevalence of potential risk factors at individual hospitals and the quality of care provided by those hospitals.) For the administrative, POA, and laboratory models, additional potential risk factors were added in a sequence determined by forward stepwise logistic regression.³⁷ To avoid overfitting, variables added after the minimum value of the Schwarz criterion³⁸ was attained were removed from models. (This criterion weighs the trade-off between the fit of a model and its complexity.) The remaining predictive variables and their coefficients were evaluated for clinical plausibility. On rare occasions, clinically problematic variables were eliminated or modified. To avoid substituting more expensive clinical variables for less costly ones with almost equivalent predictive power, predictive variables selected for the laboratory model were retained and additional clinical risk factors were added in sequence as described above.

For each of the 4 models (administrative, POA, laboratory, and clinical) for each condition or procedure, a nested sequence of models was created first with 1 variable selected using the training data set, then with 2 variables, and lastly with all the variables. Variables were added to successive models in the order in which they were entered in the minimum Schwarz criterion model. From each nested sequence of models, the validation set was used to select the model with the smallest average prediction error³⁹ as the final validated model. Finally, the coefficients of the variables in the validated models were retained, the hospital variables were removed, and the intercepts were recalculated to equate observed and predicted mortality rates.

Case-level discriminatory power (ie, the ability of a model to distinguish cases that died from those that survived) was computed on the test set using c statistics.⁴⁰ All data management and statistical analyses were performed using SAS software versions 8 and 9.1 (SAS Institute Inc, Cary, NC). The study design was approved by the Abt Associates' institutional review board.

The numbers of cases ranged from 5309 (abdominal aortic aneurysm repair) to 200 506 (congestive heart failure) (Table). Mortality rates ranged from 3.2% for coronary artery bypass graft surgery to 10.8% for acute cerebrovascular accident.

Designating secondary diagnoses as present at admission increased the average number of secondary diagnosis variables included from 8.6 in the administrative model to 15.4 in the POA model. This increase occurred because secondary diagnoses such as acute renal failure in patients admitted to the hospital with pneumonia, who also had elevated creatinine levels on the day of admission, were eligible for inclusion as risk factors in the POA model. Comparison of the POA model and the laboratory model revealed that the addition of an average of 11.1 numerical laboratory values present on the first hospital day was accompanied by an average reduction of 4.5 secondary diagnosis variables. This reduction reflected the substitution of more specific laboratory values for less specific secondary diagnosis variables (eg, pH ≤7.25 or pH >7.25 but ≤7.35 replaced the ICD-9-CM secondary diagnosis code for acidosis in acute myocardial infarction). Compared with the laboratory model, an average of 9 additional clinical findings present on the first hospital day were incorporated into the clinical model.

The final models included a total of 20 numerical laboratory determinations, 3 other laboratory determinations (eg, blood cultures), 5 vital signs, 22 key clinical findings, and 2 composite scores. Many individual numerical laboratory results and vital signs appeared in the clinical models for 4 or more conditions or procedures (eg, pH and prothrombin time were risk factors in the clinical models for all 5 conditions and 3 procedures). On the other hand, few key clinical findings appeared in the models for more than 2 of the conditions and procedures. The Glasgow Coma Score was a risk factor for 3 of the 5 conditions and 1 of the 3 procedures and the American Society of Anesthesiologists classification was a risk factor for the 2 other procedures.

The receiver operating characteristic curves reflecting the average c statistics of alternative models are shown in the Figure. The average c statistic increased from 0.50 for no risk adjustment to a mean (SD) of 0.79 (0.02) for the administrative model, to 0.84 (0.01) for the POA model, to 0.86 (0.01) for the laboratory model, to 0.88 (0.01) for the clinical model.

This study was designed to guide the selection of a cost-effective set of clinical data elements to improve the validity of comparisons of risk-adjusted hospital mortality rates. Because these comparisons often are important components of public reports, pay-for-performance programs, and quality improvement initiatives, it is essential that they accurately reflect the quality of care provided by each facility.⁴¹ Unlike most previous studies that attempted to derive the most parsimonious or most sophisticated risk-adjustment model for a single condition or procedure or to compare models based on administrative data to corresponding models based on clinical data, the principal goal of this study was to evaluate the relative performances of alternative equations based on progressively more detailed data sets and identify one that could meet the sometimes conflicting needs of physicians, hospital administrators, and payers. Therefore, a diverse sample of conditions and procedures was evaluated, and methodological uniformity was emphasized to minimize the confounding effects of differences in analytic technique and to obtain precise estimates of improvements in the risk adjustment directly attributable to changing the type of data available for use in the predictive equations.

In deriving the administrative and POA models, care was taken to avoid using risk factors based on conditions or procedures that reflected potentially avoidable hospital-acquired complications rather than intrinsic patient risks at the time of admission. For both the administrative and POA models, the use of procedure codes as risk factors was limited to situations in which they were found to be irreplaceable surrogates for intrinsic patient risk. In the administrative model, secondary diagnoses were eligible for use as risk factors only when a separate analysis documented that they only rarely reflected hospital-acquired complications. In the POA model, secondary diagnoses ineligible for the administrative model were considered as potential risk factors only if the Atlas clinical data substantiated their presence on the first day of hospitalization.

A national standard for adding a POA code to administrative claims data in the UB-04 (the uniform bill used to submit all hospital claims to payers) is planned as part of the revised ICD-9-CM coding modifications for 2007.^31,42 In this study, the use of a surrogate for this code resulted in noteworthy improvements in the performance of the risk-adjustment models, confirming the value of this new coding convention. Substantial additional improvements in the performance of risk-adjustment models occurred when numerical laboratory values were added to the POA codes. The sum of all further improvements from adding other clinical data elements was substantially less than improvements achieved by adding surrogate POA coding and numerical laboratory values to the standard administrative data.

Data collected by the Pennsylvania Health Care Cost Containment Council⁴³ demonstrated that when hospitals routinely collect a specified set of clinical data elements on a large number of discharges, they can reduce the cost of retrieving these data by investing in standardized record formats and electronic aids to data collection and by training less expensive personnel to obtain required data quickly and accurately. In addition, a recent study⁴⁴ by HIMSS Analytics found that 80.8% of hospitals had the computerized laboratory systems required to support the laboratory models. Therefore, many hospitals currently should be capable of electronically merging administrative and numerical laboratory data, thereby reducing their costs of acquiring laboratory data by eliminating the need for manual abstraction. In contrast, only 10.6% of hospitals currently have computerized nursing documentation required to support models that include vital signs and other clinical data, although rapid improvements in health information technology are anticipated over the next few years.³³

The present study was limited by its use of only 1 indicator of hospital quality (ie, mortality) and by its failure to evaluate directly the effects of variations in coding practices and in the number of secondary diagnosis codes included in the centralized databases. Recommendations about the inclusion of specific data elements within each level of clinical data may not apply in all circumstances because some data elements not identified in this study might prove to be important for outcomes and conditions outside the scope of this investigation. In addition, measures of function were not available but have been shown to have value in predicting outcomes.⁴⁵

In summary, this analysis strongly supports the value of enhancing administrative claims data with POA codes and a limited set of numerical laboratory values obtained at admission. These data provide information required to avoid errors in the designation of hospitals and their medical staffs as delivering better than average or worse than average care. On the other hand, secondary abstraction of difficult-to-obtain key clinical findings appears to add little to the risk adjustment of inpatient mortality rates.