0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Original Contribution |

Enhancement of Claims Data to Improve Risk Adjustment of Hospital Mortality FREE

Michael Pine, MD, MBA; Harmon S. Jordan, ScD; Anne Elixhauser, PhD; Donald E. Fry, MD; David C. Hoaglin, PhD; Barbara Jones, MA; Roger Meimban, PhD; David Warner, MS; Junius Gonzales, MD, MBA
[+] Author Affiliations

Author Affiliations: Michael Pine and Associates Inc, Chicago, Ill (Drs Pine, Fry, and Meimban, and Ms Jones); Department of Medicine, Pritzker School of Medicine, University of Chicago, Chicago, Ill (Dr Pine); Abt Associates Inc, Cambridge, Mass (Drs Jordan, Hoaglin, Gonzales, and Mr Warner); School of Medicine, Tufts University, Boston, Mass (Dr Jordan); and Agency for Healthcare Research and Quality, Rockville, Md (Dr Elixhauser).

More Author Information
JAMA. 2007;297(1):71-76. doi:10.1001/jama.297.1.71.
Text Size: A A A
Published online

Context Comparisons of risk-adjusted hospital performance often are important components of public reports, pay-for-performance programs, and quality improvement initiatives. Risk-adjustment equations used in these analyses must contain sufficient clinical detail to ensure accurate measurements of hospital quality.

Objective To assess the effect on risk-adjusted hospital mortality rates of adding present on admission codes and numerical laboratory data to administrative claims data.

Design, Setting, and Patients Comparison of risk-adjustment equations for inpatient mortality from July 2000 through June 2003 derived by sequentially adding increasingly difficult-to-obtain clinical data to an administrative database of 188 Pennsylvania hospitals. Patients were hospitalized for acute myocardial infarction, congestive heart failure, cerebrovascular accident, gastrointestinal tract hemorrhage, or pneumonia or underwent an abdominal aortic aneurysm repair, coronary artery bypass graft surgery, or craniotomy.

Main Outcome Measures C statistics as a measure of the discriminatory power of alternative risk-adjustment models (administrative, present on admission, laboratory, and clinical for each of the 5 conditions and 3 procedures).

Results The mean (SD) c statistic for the administrative model was 0.79 (0.02). Adding present on admission codes and numerical laboratory data collected at the time of admission resulted in substantially improved risk-adjustment equations (mean [SD] c statistic of 0.84 [0.01] and 0.86 [0.01], respectively). Modest additional improvements were obtained by adding more complex and expensive to collect clinical data such as vital signs, blood culture results, key clinical findings, and composite scores abstracted from patients' medical records (mean [SD] c statistic of 0.88 [0.01]).

Conclusions This study supports the value of adding present on admission codes and numerical laboratory values to administrative databases. Secondary abstraction of difficult-to-obtain key clinical findings adds little to the predictive power of risk-adjustment equations.

Figures in this Article

Risk-adjusted hospital mortality rates for specified conditions and procedures frequently are used in public reports and pay-for-performance programs as indicators of the quality of hospital care.13 Risk adjustment often is based solely on administrative claims data from uniform bills that hospitals submit to payers. These data lack clinically important pathophysiological information and do not distinguish between conditions that were present on admission (POA; ie, potential risk factors) and complications that occurred during hospitalization. The validity of risk-adjustment systems that use only administrative data has been challenged repeatedly,49 and there is general agreement1023 that additional data are required to predict accurately an individual patient's risk of dying.

Physicians are particularly concerned that inadequate risk adjustment penalizes the practitioners and facilities that care for the sickest patients and may result in the denial of needed care to high-risk patients.2426 Consumer advocates and payers, through initiatives such as the Consumer-Purchaser Disclosure Project,27 are attempting to expand administrative data sets to include clinical information to ensure that incentives reward high-quality clinical care. On the other hand, many hospital administrators have complained that the cost of retrieving supplementary clinical data from medical records is prohibitive, and some researchers have argued that risk adjustment using only administrative data can be made sufficiently accurate to support valid comparisons among hospitals.28,29

The addition of a POA modifier for secondary diagnosis codes first was proposed in 199130 and was successfully adopted in New York State in 1994 and in California in 1996.31 The planned implementation in March 2007 of new standards for hospital claims data includes nationwide adoption of this modifier, which distinguishes conditions that develop during hospital stays (potential complications of care) from conditions that were present at admission (potential treatment-independent risk factors). Inclusion of POA codes in administrative data sets should permit analysts to incorporate important predictors of inpatient mortality into administrative risk-adjustment equations without improperly designating patients as having high intrinsic risks at admission when their increased vulnerability resulted from hospital-acquired complications.31

The adoption of Logical Observation Identifiers Names and Codes32 for laboratory data and advances in electronic health data technology have lowered the cost of retrieving numerical laboratory data at many hospitals.33 Because of substantial differences in the cost of obtaining various types of clinical data, limited enhancement of administrative data sets appears to be both practical and desirable. Ideally, clinical data elements selected for this purpose will be relatively inexpensive to obtain and will be useful predictors of mortality for multiple conditions and procedures.

This study was designed to test the hypothesis that the combination of POA modifiers for secondary diagnoses and a limited set of numerical laboratory data would improve risk adjustment of inpatient mortality for a diverse set of clinical conditions and procedures. We also hypothesized that further additions of highly specific, difficult-to-obtain clinical data sometimes considered important predictors of inpatient mortality by clinicians would add little to the accuracy of predictive models.

Risk-adjustment models were created and analyzed using data from July 2000 through June 2003 from 188 Pennsylvania hospitals supplied by the Pennsylvania Health Care Cost Containment Council.34 Case-level claims data were supplemented with clinical data abstracted from medical records by specially trained personnel using MediQual's proprietary Atlas clinical information system.35 This system defines a broad array of clinical data elements, including historical information, laboratory results, vital signs, clinical symptoms and signs, pathophysiological abnormalities, and composite pathophysiological scores, which are collected and stored along with the hospital day on which each clinical finding was observed.

Risk-adjusted mortality rates were analyzed for 5 health conditions (acute myocardial infarction, congestive heart failure, acute cerebrovascular accident, gastrointestinal tract hemorrhage, or pneumonia) and 3 surgical procedures (abdominal aortic aneurysm repair, coronary artery bypass graft surgery, or craniotomy). The Agency for Healthcare Research and Quality's Inpatient Quality Indicator software version 2.1 was used to identify cases that met criteria for inclusion in each group.36

Four models were constructed for each condition and procedure (1 set of data for each of the 5 conditions and 1 set of data for each of the 3 procedures). The first model termed administrative used standard claims data. The second model termed POA used data abstracted from medical records to determine whether coded secondary diagnoses had been present at admission. The third model termed laboratory used POA codes and numerical laboratory data (often available in electronic form; eg, creatinine, hematocrit level) documented on the first day of hospitalization prior to a procedure requiring general or regional anesthesia. The fourth model termed clinical used the criteria in the third model plus vital signs, other laboratory data not included in the third model (eg, bacterial culture results), Atlas key clinical findings abstracted from medical records (eg, immunocompromised, lethargy), and composite clinical scores (ie, American Society of Anesthesiologists classification, Glasgow Coma Score) documented on the first day of hospitalization prior to a procedure requiring general or regional anesthesia.

The administrative model was based solely on data from hospital bills (ie, age, sex, and principal diagnoses, secondary diagnoses, and procedures coded according to the International Classification of Diseases, Ninth Revision, Clinical Modification [ICD-9-CM]). To avoid using hospital-acquired complications as risk factors, hospital bills from New York and California (secondary diagnoses were modified by POA codes in these states) were used to help identify which secondary diagnoses were generally present at admission. Secondary diagnoses were eligible for inclusion as risk factors in the administrative model only when they were coded as hospital-acquired complications in fewer than 20% of cases in which they occurred.

The POA model included additional secondary diagnoses excluded from the administrative model because of their association with unacceptably high rates of complications. Because Pennsylvania claims data do not include POA codes, clinical data in the Atlas database were used to determine whether coded secondary diagnoses were present at admission. In the creation of surrogate POA codes, the Atlas database served as a substitute for the complete medical record available to coders in New York and California. For example, posthemorrhagic anemia was excluded from the administrative model for congestive heart failure because hospitals in New York and California coded it as acquired during hospitalization in more than 30% of the cases in which it occurred. However, posthemorrhagic anemia was eligible for inclusion as a risk factor in the POA model for congestive heart failure when the Atlas database documented that anemia was present on the day of admission.

For each condition or procedure, candidate risk factors were constructed from principal diagnosis codes, up to 8 secondary diagnosis codes, up to 6 procedure codes, and clinical data elements associated with higher than average mortality rates. Infrequently occurring codes were combined with codes for clinically similar conditions or procedures that had similar mortality rates. Continuous measures (eg, age, creatinine level) were transformed into 1 or more categorical variables based on clinical judgment and empirical evaluation of associated mortality rates.

For each condition or procedure, stratified random samples of live discharges and fatalities were combined to create 3 mutually exclusive data sets: a training set (50%), a validation set (25%), and a test set (25%). Partitioning the data in this way facilitated the construction of more robust models.

A preliminary predictive equation was developed on the training set for each condition or procedure using only age categories and individual hospital identifiers. (Including hospitals as risk factors during model development is a standard technique for reducing possible bias caused by the associations between the prevalence of potential risk factors at individual hospitals and the quality of care provided by those hospitals.) For the administrative, POA, and laboratory models, additional potential risk factors were added in a sequence determined by forward stepwise logistic regression.37 To avoid overfitting, variables added after the minimum value of the Schwarz criterion38 was attained were removed from models. (This criterion weighs the trade-off between the fit of a model and its complexity.) The remaining predictive variables and their coefficients were evaluated for clinical plausibility. On rare occasions, clinically problematic variables were eliminated or modified. To avoid substituting more expensive clinical variables for less costly ones with almost equivalent predictive power, predictive variables selected for the laboratory model were retained and additional clinical risk factors were added in sequence as described above.

For each of the 4 models (administrative, POA, laboratory, and clinical) for each condition or procedure, a nested sequence of models was created first with 1 variable selected using the training data set, then with 2 variables, and lastly with all the variables. Variables were added to successive models in the order in which they were entered in the minimum Schwarz criterion model. From each nested sequence of models, the validation set was used to select the model with the smallest average prediction error39 as the final validated model. Finally, the coefficients of the variables in the validated models were retained, the hospital variables were removed, and the intercepts were recalculated to equate observed and predicted mortality rates.

Case-level discriminatory power (ie, the ability of a model to distinguish cases that died from those that survived) was computed on the test set using c statistics.40 All data management and statistical analyses were performed using SAS software versions 8 and 9.1 (SAS Institute Inc, Cary, NC). The study design was approved by the Abt Associates' institutional review board.

The numbers of cases ranged from 5309 (abdominal aortic aneurysm repair) to 200 506 (congestive heart failure) (Table). Mortality rates ranged from 3.2% for coronary artery bypass graft surgery to 10.8% for acute cerebrovascular accident.

Table Graphic Jump LocationTable. Number of Hospitals, Cases, and Fatalities and Mortality Rate for Each Condition and Procedure

Designating secondary diagnoses as present at admission increased the average number of secondary diagnosis variables included from 8.6 in the administrative model to 15.4 in the POA model. This increase occurred because secondary diagnoses such as acute renal failure in patients admitted to the hospital with pneumonia, who also had elevated creatinine levels on the day of admission, were eligible for inclusion as risk factors in the POA model. Comparison of the POA model and the laboratory model revealed that the addition of an average of 11.1 numerical laboratory values present on the first hospital day was accompanied by an average reduction of 4.5 secondary diagnosis variables. This reduction reflected the substitution of more specific laboratory values for less specific secondary diagnosis variables (eg, pH ≤7.25 or pH >7.25 but ≤7.35 replaced the ICD-9-CM secondary diagnosis code for acidosis in acute myocardial infarction). Compared with the laboratory model, an average of 9 additional clinical findings present on the first hospital day were incorporated into the clinical model.

The final models included a total of 20 numerical laboratory determinations, 3 other laboratory determinations (eg, blood cultures), 5 vital signs, 22 key clinical findings, and 2 composite scores. Many individual numerical laboratory results and vital signs appeared in the clinical models for 4 or more conditions or procedures (eg, pH and prothrombin time were risk factors in the clinical models for all 5 conditions and 3 procedures). On the other hand, few key clinical findings appeared in the models for more than 2 of the conditions and procedures. The Glasgow Coma Score was a risk factor for 3 of the 5 conditions and 1 of the 3 procedures and the American Society of Anesthesiologists classification was a risk factor for the 2 other procedures.

The receiver operating characteristic curves reflecting the average c statistics of alternative models are shown in the Figure. The average c statistic increased from 0.50 for no risk adjustment to a mean (SD) of 0.79 (0.02) for the administrative model, to 0.84 (0.01) for the POA model, to 0.86 (0.01) for the laboratory model, to 0.88 (0.01) for the clinical model.

Figure. Receiver Operating Characteristic Curves for the Models
Graphic Jump Location

The 4 receiver operating characteristic curves represent data from each of the 4 risk-adjustment models: standard administrative model; present at admission (POA) model, standard administrative model with POA modifiers; laboratory model, POA plus numerical laboratory data; and clinical model, laboratory model plus all available clinical data. The diagonal dotted line indicates no risk adjustment. The data markers represent cut points of mortality risk predicted by each model in increments of 2%. Mortality rate cut points shown in the plot include 0% (upper right), 2% (gray), 4% (blue), 6% (pink), 8% (black), and 100% (bottom left). Each model is based on 8 data sets (1 set of data for each of the 5 conditions and 1 set of data for each of the 3 procedures). Predicted mortality rates from the 8 data sets in each model were averaged and compared with the observed values to calculate the true positive rate (sensitivity, where positive equals dead) and false-positive rate (100 minus specificity) at each mortality risk cut point.

This study was designed to guide the selection of a cost-effective set of clinical data elements to improve the validity of comparisons of risk-adjusted hospital mortality rates. Because these comparisons often are important components of public reports, pay-for-performance programs, and quality improvement initiatives, it is essential that they accurately reflect the quality of care provided by each facility.41 Unlike most previous studies that attempted to derive the most parsimonious or most sophisticated risk-adjustment model for a single condition or procedure or to compare models based on administrative data to corresponding models based on clinical data, the principal goal of this study was to evaluate the relative performances of alternative equations based on progressively more detailed data sets and identify one that could meet the sometimes conflicting needs of physicians, hospital administrators, and payers. Therefore, a diverse sample of conditions and procedures was evaluated, and methodological uniformity was emphasized to minimize the confounding effects of differences in analytic technique and to obtain precise estimates of improvements in the risk adjustment directly attributable to changing the type of data available for use in the predictive equations.

In deriving the administrative and POA models, care was taken to avoid using risk factors based on conditions or procedures that reflected potentially avoidable hospital-acquired complications rather than intrinsic patient risks at the time of admission. For both the administrative and POA models, the use of procedure codes as risk factors was limited to situations in which they were found to be irreplaceable surrogates for intrinsic patient risk. In the administrative model, secondary diagnoses were eligible for use as risk factors only when a separate analysis documented that they only rarely reflected hospital-acquired complications. In the POA model, secondary diagnoses ineligible for the administrative model were considered as potential risk factors only if the Atlas clinical data substantiated their presence on the first day of hospitalization.

A national standard for adding a POA code to administrative claims data in the UB-04 (the uniform bill used to submit all hospital claims to payers) is planned as part of the revised ICD-9-CM coding modifications for 2007.31,42 In this study, the use of a surrogate for this code resulted in noteworthy improvements in the performance of the risk-adjustment models, confirming the value of this new coding convention. Substantial additional improvements in the performance of risk-adjustment models occurred when numerical laboratory values were added to the POA codes. The sum of all further improvements from adding other clinical data elements was substantially less than improvements achieved by adding surrogate POA coding and numerical laboratory values to the standard administrative data.

Data collected by the Pennsylvania Health Care Cost Containment Council43 demonstrated that when hospitals routinely collect a specified set of clinical data elements on a large number of discharges, they can reduce the cost of retrieving these data by investing in standardized record formats and electronic aids to data collection and by training less expensive personnel to obtain required data quickly and accurately. In addition, a recent study44 by HIMSS Analytics found that 80.8% of hospitals had the computerized laboratory systems required to support the laboratory models. Therefore, many hospitals currently should be capable of electronically merging administrative and numerical laboratory data, thereby reducing their costs of acquiring laboratory data by eliminating the need for manual abstraction. In contrast, only 10.6% of hospitals currently have computerized nursing documentation required to support models that include vital signs and other clinical data, although rapid improvements in health information technology are anticipated over the next few years.33

The present study was limited by its use of only 1 indicator of hospital quality (ie, mortality) and by its failure to evaluate directly the effects of variations in coding practices and in the number of secondary diagnosis codes included in the centralized databases. Recommendations about the inclusion of specific data elements within each level of clinical data may not apply in all circumstances because some data elements not identified in this study might prove to be important for outcomes and conditions outside the scope of this investigation. In addition, measures of function were not available but have been shown to have value in predicting outcomes.45

In summary, this analysis strongly supports the value of enhancing administrative claims data with POA codes and a limited set of numerical laboratory values obtained at admission. These data provide information required to avoid errors in the designation of hospitals and their medical staffs as delivering better than average or worse than average care. On the other hand, secondary abstraction of difficult-to-obtain key clinical findings appears to add little to the risk adjustment of inpatient mortality rates.

Corresponding Author: Michael Pine, MD, MBA, 1210 Chicago Ave, Suite 503, Evanston, IL 60202 (mpine@aol.com).

Author Contributions: Dr Pine had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Pine, Jordan, Elixhauser, Hoaglin.

Acquisition of data: Jordan, Elixhauser, Jones.

Analysis and interpretation of data: Pine, Jordan, Fry, Hoaglin, Jones, Meimban, Warner, Gonzales.

Drafting of the manuscript: Pine, Jordan, Fry, Jones, Meimban, Warner.

Critical revision of the manuscript for important intellectual content: Jordan, Elixhauser, Hoaglin, Gonzales.

Statistical analysis: Pine, Jordan, Hoaglin, Jones, Meimban.

Obtained funding: Pine, Jordan, Elixhauser.

Administrative, technical, or material support: Pine, Jordan, Elixhauser, Warner.

Study supervision: Pine, Jordan, Elixhauser.

Financial Disclosures: Dr Hoaglin reported owning shares of stock in Cardinal Health Inc. None of the other authors reported any disclosures.

Funding/Support: This study was supported by the Agency for Healthcare Research and Quality contract 233-02-0088 (task order 13).

Role of the Sponsor: The Agency for Healthcare Research and Quality (AHRQ) specified the overarching study design in a request for proposal. Data were collected by Pennsylvania hospitals, which were required by law to submit these data to the Pennsylvania Health Care Cost Containment Council. Data were transmitted to the AHRQ by the Council and the AHRQ transmitted them to the authors. Further data management, analyses, interpretation, and preparation of the manuscript were the independent work of the authors. The manuscript was reviewed by the Council and by the director of the Center for Delivery, Organizations, and Markets at AHRQ prior to the initial submission. Complete specifications of data elements, potential risk factors, and risk-adjustment equations are available in the final report submitted to the AHRQ. Administrative and Atlas clinical data were provided by the Council, an independent state agency responsible for addressing the problem of escalating health costs, ensuring the quality of health care, and increasing access to health care for all citizens regardless of ability to pay. The Council provided data in an effort to further its mission of educating the public and containing health care costs in Pennsylvania. The Council, its agents and staff, have made no representation, guarantee, or warranty, express or implied, that the data are error-free, or that the use of the data will avoid differences of opinion or interpretation. Analyses reported in this article were not prepared by the Council.

Disclaimer: The Pennsylvania Health Care Cost Containment Council, its agents and staff, bear no responsibility or liability for the results of these analyses, which are solely the opinion of the authors.

Agency for Healthcare Research and Quality.  Inpatient quality indicators, technical specifications [February 20, 2006]. http://www.qualityindicators.ahrq.gov/downloads/iqi/iqi_technical_specs_v30.pdf. Accessed November 6, 2006
 HealthGrades Inc Web site. http://www.healthgrades.com. Accessed November 6, 2006
McKay NL, Deily ME. Comparing high- and low-performing hospitals using risk-adjusted excess mortality and cost inefficiency.  Health Care Manage Rev. 2005;30:347-360
PubMed   |  Link to Article
Hannan EL, Kilburn H Jr, Lindsey ML.  et al.  Clinical versus administrative data bases for CABG surgery: does it matter?  Med Care. 1992;30:892-907
PubMed   |  Link to Article
Jollis JG, Ancukiewicz M, DeLong ER.  et al.  Discordance of databases designed for claims payment versus clinical information systems: implications for outcomes research.  Ann Intern Med. 1993;119:844-850
PubMed   |  Link to Article
Iezzoni LI, Ash AS, Coffman GA.  et al.  Predicting in-hospital mortality: a comparison of severity measurement approaches.  Med Care. 1992;30:347-359
PubMed   |  Link to Article
Iezzoni LI, Ash AS, Shwartz M.  et al.  Predicting who dies depends on how severity is measured: implications for evaluating patient outcomes.  Ann Intern Med. 1995;123:763-770
PubMed   |  Link to Article
Hannan EL, Racz MJ, Jollis JG.  et al.  Using Medicare claims data to assess provider quality for CABG surgery: does it work well enough?  Health Serv Res. 1997;31:659-678
PubMed
Pine M, Norusis M, Jones B.  et al.  Predictions of hospital mortality rates: a comparison of data sources.  Ann Intern Med. 1997;126:347-354
PubMed   |  Link to Article
Knaus WA, Zimmerman JE, Wagner DP.  et al.  APACHE-acute physiology and chronic health evaluation: a physiologically based classification system.  Crit Care Med. 1981;9:591-597
PubMed   |  Link to Article
Brewster AC, Karlin BG, Hyde LA.  et al.  MEDISGRPS: a clinically based approach to classifying hospital patients at admission.  Inquiry. 1985;22:377-387
PubMed
Daley J, Jencks S, Draper D.  et al.  Predicting hospital-associated mortality for Medicare patients: a method for patients with stroke, pneumonia, acute myocardial infarction, and congestive heart failure.  JAMA. 1988;260:3617-3624
PubMed   |  Link to Article
Hannan EL, Kilburn H Jr, O'Donnell JF.  et al.  Adult open heart surgery in New York State: an analysis of risk factors and hospital mortality rates.  JAMA. 1990;264:2768-2774
PubMed   |  Link to Article
Smith DW, Pine M, Bailey RC.  et al.  Using clinical variables to estimate the risk of patient mortality.  Med Care. 1991;29:1108-1129
PubMed   |  Link to Article
Higgins TL, Estafanous FG, Loop FD.  et al.  Stratification of morbidity and mortality outcome by preoperative risk factors in coronary artery bypass patients: a clinical severity score.  JAMA. 1992;267:2344-2348
PubMed   |  Link to Article
Lemeshow S, Teres D, Klar J.  et al.  Mortality probability models (MPM II) based on an international cohort of intensive care unit patients.  JAMA. 1993;270:2478-2486
PubMed   |  Link to Article
Van Ruiswyk J, Hartz A, Kuhn E.  et al.  A measure of mortality risk for elderly patients with acute myocardial infarction.  Med Decis Making. 1993;13:152-160
PubMed   |  Link to Article
Khuri SF, Daley J, Henderson W.  et al.  The Department of Veterans Affairs' NSQIP [National VA Surgical Quality Improvement Program]: the first national, validated, outcome-based, risk-adjusted, and peer-controlled program for the measurement and enhancement of the quality of surgical care.  Ann Surg. 1998;228:491-507
PubMed   |  Link to Article
Pine M, Jones B, Lou YB. Laboratory values improve predictions of hospital mortality.  Int J Qual Health Care. 1998;10:491-501
PubMed   |  Link to Article
Gibbs J, Cull W, Henderson W.  et al.  Preoperative serum albumin level as a predictor of operative mortality and morbidity.  Arch Surg. 1999;134:36-42
PubMed   |  Link to Article
Ghali WA, Faris PD, Galbraith PD.  et al.  Sex differences in access to coronary revascularization after cardiac catheterization: importance of detailed clinical data.  Ann Intern Med. 2002;136:723-732
PubMed   |  Link to Article
Grover FL, Shroyer AL, Hammermeister K.  et al.  A decade's experience with quality improvement in cardiac surgery using the Veterans Affairs and Society of Thoracic Surgeons national databases.  Ann Surg. 2001;234:464-474
PubMed   |  Link to Article
Lee DS, Austin PC, Rouleau JL.  et al.  Predicting mortality among patients hospitalized for heart failure: derivation and validation of a clinical model.  JAMA. 2003;290:2581-2587
PubMed   |  Link to Article
Schneider EC, Epstein AM. Influence of cardiac surgery performance reports on referral practices and access to care: a survey of cardiovascular specialists.  N Engl J Med. 1996;335:251-256
PubMed   |  Link to Article
Moscucci M, Eagle KA, Share D.  et al.  Public reporting and case selection for percutaneous coronary interventions: an analysis from two large multicenter percutaneous coronary intervention databases.  J Am Coll Cardiol. 2005;45:1759-1765
PubMed   |  Link to Article
Narins CR, Dozier AM, Ling FS.  et al.  The influence of public reporting of outcome data on medical decision making by physicians.  Arch Intern Med. 2005;165:83-87
PubMed   |  Link to Article
National Health Care Purchasing Institute.  Status report: consumer and purchaser disclosure project [October 2002]. http://www.pbgh.org/programs/standardization/documents/DisclosureProjectNewsletter-October2002.pdf. Accessed November 6, 2006
Krumholz HM, Wang Y, Mattera JA.  et al.  An administrative claims model suitable for profiling hospital performance based on 30-day mortality rates among patients with an acute myocardial infarction.  Circulation. 2006;113:1683-1692
PubMed   |  Link to Article
Krumholz HM, Wang Y, Mattera JA.  et al.  An administrative claims model suitable for profiling hospital performance based on 30-day mortality rates among patients with heart failure.  Circulation. 2006;113:1693-1701
PubMed   |  Link to Article
Naessens JM, Brennan MD, Boberg CJ.  et al.  Acquired conditions: an improvement to hospital discharge abstracts.  Qual Assur Health Care. 1991;3:257-262
PubMed   |  Link to Article
Coffey R, Milenkovic M, Andrews RM.US Agency for Healthcare Research and Quality.  The case for the present on admission (POA) indicator: HCUP methods series report 2006-01 [June 26, 2006]. http://www.hcup-us.ahrq.gov/reports/2006_1.pdf. Accessed November 6, 2006
Regenstreif Institute.  Logical observation identifiers names and codes (LOINC). http://www.regenstrief.org/loinc. Accessed November 6, 2006
Berner ES, Detmer DE, Simborg D. Will the wave finally break? a brief view of the adoption of electronic medical records in the United States.  J Am Med Inform Assoc. 2005;12:3-7
PubMed   |  Link to Article
 Pennsylvania Health Care Cost Containment Council Web site. http://www.phc4.org. Accessed November 6, 2006
Cardinal Health.  Atlas outcomes system. http://www.mediqual.com/products/atlas.asp. Accessed November 6, 2006
Agency for Healthcare Research and Quality.  Quality indicators software for SAS, zipped. http://www.qualityindicators.ahrq.gov/iqi_download.htm. Accessed November 6, 2006
Hosmer DW, Lemeshow S. Logistic regression. In: Applied Logistic Regression. 2nd ed. New York, NY: John Wiley and Sons; 2000:91-142
Schwarz G. Estimating the dimension of a model.  Ann Stat. 1978;6:461-464
Link to Article
Hastie T, Tibshirani R, Friedman J. The Elements of Statistical Learning. New York, NY: Springer-Verlag; 2001
Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve.  Radiology. 1982;143:29-36
PubMed
Lee TH, Meyer GS, Brennan TA. A middle ground on public accountability.  N Engl J Med. 2004;350:2409-2412
PubMed   |  Link to Article
National Uniform Billing Committee.  UB-04 Data specifications manual (Beta3): present on admission indicator. http://www.nubc.org/public/whatsnew/POA.pdf. Accessed November 6, 2006
Pennsylvania Healthcare Cost Containment Council.  Independent accountant's [Parente Randolph] report on applying agreed-upon procedures [August 5, 2004]. http://www.phc4.org. Accessed December 1, 2006
Garets D, Davis M. Electronic medical records vs electronic health records: yes, there is a difference: a HIMSS analytics white paper [2006]. http://www.himssanalytics.org/docs/WP_EMR_EHR.pdf. Accessed November 6, 2006
Lee SJ, Lindquist K, Segal MR, Covinsky KE. Development and validation of a prognostic index for 4-year mortality in older adults.  JAMA. 2006;295:801-808
PubMed   |  Link to Article

Figures

Figure. Receiver Operating Characteristic Curves for the Models
Graphic Jump Location

The 4 receiver operating characteristic curves represent data from each of the 4 risk-adjustment models: standard administrative model; present at admission (POA) model, standard administrative model with POA modifiers; laboratory model, POA plus numerical laboratory data; and clinical model, laboratory model plus all available clinical data. The diagonal dotted line indicates no risk adjustment. The data markers represent cut points of mortality risk predicted by each model in increments of 2%. Mortality rate cut points shown in the plot include 0% (upper right), 2% (gray), 4% (blue), 6% (pink), 8% (black), and 100% (bottom left). Each model is based on 8 data sets (1 set of data for each of the 5 conditions and 1 set of data for each of the 3 procedures). Predicted mortality rates from the 8 data sets in each model were averaged and compared with the observed values to calculate the true positive rate (sensitivity, where positive equals dead) and false-positive rate (100 minus specificity) at each mortality risk cut point.

Tables

Table Graphic Jump LocationTable. Number of Hospitals, Cases, and Fatalities and Mortality Rate for Each Condition and Procedure

References

Agency for Healthcare Research and Quality.  Inpatient quality indicators, technical specifications [February 20, 2006]. http://www.qualityindicators.ahrq.gov/downloads/iqi/iqi_technical_specs_v30.pdf. Accessed November 6, 2006
 HealthGrades Inc Web site. http://www.healthgrades.com. Accessed November 6, 2006
McKay NL, Deily ME. Comparing high- and low-performing hospitals using risk-adjusted excess mortality and cost inefficiency.  Health Care Manage Rev. 2005;30:347-360
PubMed   |  Link to Article
Hannan EL, Kilburn H Jr, Lindsey ML.  et al.  Clinical versus administrative data bases for CABG surgery: does it matter?  Med Care. 1992;30:892-907
PubMed   |  Link to Article
Jollis JG, Ancukiewicz M, DeLong ER.  et al.  Discordance of databases designed for claims payment versus clinical information systems: implications for outcomes research.  Ann Intern Med. 1993;119:844-850
PubMed   |  Link to Article
Iezzoni LI, Ash AS, Coffman GA.  et al.  Predicting in-hospital mortality: a comparison of severity measurement approaches.  Med Care. 1992;30:347-359
PubMed   |  Link to Article
Iezzoni LI, Ash AS, Shwartz M.  et al.  Predicting who dies depends on how severity is measured: implications for evaluating patient outcomes.  Ann Intern Med. 1995;123:763-770
PubMed   |  Link to Article
Hannan EL, Racz MJ, Jollis JG.  et al.  Using Medicare claims data to assess provider quality for CABG surgery: does it work well enough?  Health Serv Res. 1997;31:659-678
PubMed
Pine M, Norusis M, Jones B.  et al.  Predictions of hospital mortality rates: a comparison of data sources.  Ann Intern Med. 1997;126:347-354
PubMed   |  Link to Article
Knaus WA, Zimmerman JE, Wagner DP.  et al.  APACHE-acute physiology and chronic health evaluation: a physiologically based classification system.  Crit Care Med. 1981;9:591-597
PubMed   |  Link to Article
Brewster AC, Karlin BG, Hyde LA.  et al.  MEDISGRPS: a clinically based approach to classifying hospital patients at admission.  Inquiry. 1985;22:377-387
PubMed
Daley J, Jencks S, Draper D.  et al.  Predicting hospital-associated mortality for Medicare patients: a method for patients with stroke, pneumonia, acute myocardial infarction, and congestive heart failure.  JAMA. 1988;260:3617-3624
PubMed   |  Link to Article
Hannan EL, Kilburn H Jr, O'Donnell JF.  et al.  Adult open heart surgery in New York State: an analysis of risk factors and hospital mortality rates.  JAMA. 1990;264:2768-2774
PubMed   |  Link to Article
Smith DW, Pine M, Bailey RC.  et al.  Using clinical variables to estimate the risk of patient mortality.  Med Care. 1991;29:1108-1129
PubMed   |  Link to Article
Higgins TL, Estafanous FG, Loop FD.  et al.  Stratification of morbidity and mortality outcome by preoperative risk factors in coronary artery bypass patients: a clinical severity score.  JAMA. 1992;267:2344-2348
PubMed   |  Link to Article
Lemeshow S, Teres D, Klar J.  et al.  Mortality probability models (MPM II) based on an international cohort of intensive care unit patients.  JAMA. 1993;270:2478-2486
PubMed   |  Link to Article
Van Ruiswyk J, Hartz A, Kuhn E.  et al.  A measure of mortality risk for elderly patients with acute myocardial infarction.  Med Decis Making. 1993;13:152-160
PubMed   |  Link to Article
Khuri SF, Daley J, Henderson W.  et al.  The Department of Veterans Affairs' NSQIP [National VA Surgical Quality Improvement Program]: the first national, validated, outcome-based, risk-adjusted, and peer-controlled program for the measurement and enhancement of the quality of surgical care.  Ann Surg. 1998;228:491-507
PubMed   |  Link to Article
Pine M, Jones B, Lou YB. Laboratory values improve predictions of hospital mortality.  Int J Qual Health Care. 1998;10:491-501
PubMed   |  Link to Article
Gibbs J, Cull W, Henderson W.  et al.  Preoperative serum albumin level as a predictor of operative mortality and morbidity.  Arch Surg. 1999;134:36-42
PubMed   |  Link to Article
Ghali WA, Faris PD, Galbraith PD.  et al.  Sex differences in access to coronary revascularization after cardiac catheterization: importance of detailed clinical data.  Ann Intern Med. 2002;136:723-732
PubMed   |  Link to Article
Grover FL, Shroyer AL, Hammermeister K.  et al.  A decade's experience with quality improvement in cardiac surgery using the Veterans Affairs and Society of Thoracic Surgeons national databases.  Ann Surg. 2001;234:464-474
PubMed   |  Link to Article
Lee DS, Austin PC, Rouleau JL.  et al.  Predicting mortality among patients hospitalized for heart failure: derivation and validation of a clinical model.  JAMA. 2003;290:2581-2587
PubMed   |  Link to Article
Schneider EC, Epstein AM. Influence of cardiac surgery performance reports on referral practices and access to care: a survey of cardiovascular specialists.  N Engl J Med. 1996;335:251-256
PubMed   |  Link to Article
Moscucci M, Eagle KA, Share D.  et al.  Public reporting and case selection for percutaneous coronary interventions: an analysis from two large multicenter percutaneous coronary intervention databases.  J Am Coll Cardiol. 2005;45:1759-1765
PubMed   |  Link to Article
Narins CR, Dozier AM, Ling FS.  et al.  The influence of public reporting of outcome data on medical decision making by physicians.  Arch Intern Med. 2005;165:83-87
PubMed   |  Link to Article
National Health Care Purchasing Institute.  Status report: consumer and purchaser disclosure project [October 2002]. http://www.pbgh.org/programs/standardization/documents/DisclosureProjectNewsletter-October2002.pdf. Accessed November 6, 2006
Krumholz HM, Wang Y, Mattera JA.  et al.  An administrative claims model suitable for profiling hospital performance based on 30-day mortality rates among patients with an acute myocardial infarction.  Circulation. 2006;113:1683-1692
PubMed   |  Link to Article
Krumholz HM, Wang Y, Mattera JA.  et al.  An administrative claims model suitable for profiling hospital performance based on 30-day mortality rates among patients with heart failure.  Circulation. 2006;113:1693-1701
PubMed   |  Link to Article
Naessens JM, Brennan MD, Boberg CJ.  et al.  Acquired conditions: an improvement to hospital discharge abstracts.  Qual Assur Health Care. 1991;3:257-262
PubMed   |  Link to Article
Coffey R, Milenkovic M, Andrews RM.US Agency for Healthcare Research and Quality.  The case for the present on admission (POA) indicator: HCUP methods series report 2006-01 [June 26, 2006]. http://www.hcup-us.ahrq.gov/reports/2006_1.pdf. Accessed November 6, 2006
Regenstreif Institute.  Logical observation identifiers names and codes (LOINC). http://www.regenstrief.org/loinc. Accessed November 6, 2006
Berner ES, Detmer DE, Simborg D. Will the wave finally break? a brief view of the adoption of electronic medical records in the United States.  J Am Med Inform Assoc. 2005;12:3-7
PubMed   |  Link to Article
 Pennsylvania Health Care Cost Containment Council Web site. http://www.phc4.org. Accessed November 6, 2006
Cardinal Health.  Atlas outcomes system. http://www.mediqual.com/products/atlas.asp. Accessed November 6, 2006
Agency for Healthcare Research and Quality.  Quality indicators software for SAS, zipped. http://www.qualityindicators.ahrq.gov/iqi_download.htm. Accessed November 6, 2006
Hosmer DW, Lemeshow S. Logistic regression. In: Applied Logistic Regression. 2nd ed. New York, NY: John Wiley and Sons; 2000:91-142
Schwarz G. Estimating the dimension of a model.  Ann Stat. 1978;6:461-464
Link to Article
Hastie T, Tibshirani R, Friedman J. The Elements of Statistical Learning. New York, NY: Springer-Verlag; 2001
Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve.  Radiology. 1982;143:29-36
PubMed
Lee TH, Meyer GS, Brennan TA. A middle ground on public accountability.  N Engl J Med. 2004;350:2409-2412
PubMed   |  Link to Article
National Uniform Billing Committee.  UB-04 Data specifications manual (Beta3): present on admission indicator. http://www.nubc.org/public/whatsnew/POA.pdf. Accessed November 6, 2006
Pennsylvania Healthcare Cost Containment Council.  Independent accountant's [Parente Randolph] report on applying agreed-upon procedures [August 5, 2004]. http://www.phc4.org. Accessed December 1, 2006
Garets D, Davis M. Electronic medical records vs electronic health records: yes, there is a difference: a HIMSS analytics white paper [2006]. http://www.himssanalytics.org/docs/WP_EMR_EHR.pdf. Accessed November 6, 2006
Lee SJ, Lindquist K, Segal MR, Covinsky KE. Development and validation of a prognostic index for 4-year mortality in older adults.  JAMA. 2006;295:801-808
PubMed   |  Link to Article

Letters

CME
Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.

Multimedia

Some tools below are only available to our subscribers or users with an online account.

Web of Science® Times Cited: 138

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Collections
PubMed Articles