0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Review |

The Public Release of Performance Data:  What Do We Expect to Gain? A Review of the Evidence FREE

Martin N. Marshall, MSc, MD, FRCGP; Paul G. Shekelle, MD, PhD; Sheila Leatherman, MSW; Robert H. Brook, MD, ScD
[+] Author Affiliations

Author Affiliations: School of Postgraduate Medicine and Health Sciences, University of Exeter, Exeter, England (Dr Marshall); RAND Health Program, Santa Monica (Drs Shekelle and Brook), Greater Los Angeles Veterans Affairs Healthcare System, Los Angeles (Dr Shekelle), Calif; Center for Health Care Policy and Evaluation, United Health Group, Minnetonka, Minn (Ms Leatherman); Judge Institute of Management, University of Cambridge, Cambridge, England (Ms Leatherman); and University of California, Los Angeles, Center for Health Services, Los Angeles (Dr Brook). Dr Marshall is now with the National Primary Care Research and Development Centre, University of Manchester, Manchester, England.


JAMA. 2000;283(14):1866-1874. doi:10.1001/jama.283.14.1866.
Text Size: A A A
Published online

Context Information about the performance of hospitals, health professionals, and health care organizations has been made public in the United States for more than a decade. The expected gains of public disclosure have not been made clear, and both the benefits and potential risks have received minimal empirical investigation.

Objective To summarize the empirical evidence concerning public disclosure of performance data, relate the results to the potential gains, and identify areas requiring further research.

Data Sources A literature search was conducted on MEDLINE and EMBASE databases for articles published between January 1986 and October 1999 in peer-reviewed journals. Review of citations, public documents, and expert advice was conducted to identify studies not found in the electronic databases.

Study Selection Descriptive, observational, or experimental evaluations of US reporting systems were selected for inclusion.

Data Extraction Included studies were organized based on use of public data by consumers, purchasers, physicians, and hospitals; impact on quality of care outcomes; and costs.

Data Synthesis Seven US reporting systems have been the subject of published empirical evaluations. Descriptive and observational methods predominate. Consumers and purchasers rarely search out the information and do not understand or trust it; it has a small, although increasing, impact on their decision making. Physicians are skeptical about such data and only a small proportion makes use of it. Hospitals appear to be most responsive to the data. In a limited number of studies, the publication of performance data has been associated with an improvement in health outcomes.

Conclusions There are several potential gains from the public disclosure of performance data, but use of the information by provider organizations for quality improvement may be the most productive area for further research.

Figures in this Article

Information about the performance of hospitals, health professionals, and health care organizations is increasingly being released into the public domain.1 The data, often produced in the form of "report cards," "provider profiles," or "consumer reports," necessitates the development and dissemination of standardized reports on quality of care and facilitates comparisons of performance over time, among providers, and against defined standards of good practice. Health care performance data have been made public in the United States for more than a decade,2 and the production and dissemination of report cards is now a multimillion-dollar industry. However, evaluation of the impact of report cards has not kept pace with the development of reporting systems.37 In addition, there has been minimal agreement among the various stakeholders about expected gains from the release of comparative performance data. It has therefore proved difficult to judge whether the benefits of public disclosure of performance data outweigh the disadvantages.

The failure to articulate a clear rationale for the public release of performance data is illustrated by the wide variety of ways in which commentators believe the data might be used. The most common expectation is that such data will promote an efficient market economy in health care,8,9 usually in the belief that information about performance will encourage consumers to choose to access high-quality providers.1013 Some authorities suggest that the information could be used by providers as a marketing tool.14 Others suggest that it will help to control costs15,16 or at least counter the influence of cost as the principal determinant of purchaser decision making.1719 In addition, public information about performance has been proposed as a tool to regulate the health system,10,20 a method of ensuring accountability of provider organizations,14,21 or of making judgments about the performance of individual professionals.4 Finally, some perceive public disclosure to be a mechanism to promote quality improvement by informing purchasers18 or encouraging providers to focus on quality problems.10,14,22,23

Public disclosure therefore has been advocated as a mechanism to regulate providers of care, ensure accountability, inform, promote quality improvement, and encourage cost control. These diverse aims have made it difficult to identify evaluation criteria that can be used to assess its impact. In this article, we examine the research evidence to determine the extent to which the expectations of public disclosure have been realized and identify a future research agenda.

Data Sources, Study Selection, and Data Extraction

An extensive review of published articles in peer-reviewed journals was conducted between October 1998 and October 1999 using MEDLINE and EMBASE electronic databases. We limited our review to evaluations of US reporting systems published between January 1986 and October 1999. While organizations may carry out evaluations without publishing the results, we considered peer-review publication an important marker of the rigor of such evaluations and therefore restricted our search to published reports. Searches were conducted independently by 1 of the authors (M.N.M.) and a professional librarian using the following Medical Subject Headings: report cards, public performance reports, provider profiling, public/consumer/patient information, and consumer reports. Only original research articles were reviewed. The citations in all articles were searched. Authors of the principal published studies and other experts in the field were asked to recommend relevant published studies. In addition, documents and Web sites prepared by a range of public and private organizations were searched for references to original evaluative studies relating to public disclosure.

Data Synthesis

There are many reporting systems in the United States but only 7 of these systems appear to have been subjected to rigorous empirical evaluation. A total of 21 peer-reviewed publications were found relating to these 7 systems (Table 1). Descriptive, observational, and quasi-experimental study designs were most commonly used; there was only 1 randomized controlled trial. For the purpose of this review, the literature has been categorized into use of public reports by the key stakeholders (consumers, purchasers, physicians, and provider organizations), the impact on quality of care outcomes, and the financial costs (Table 2). The term "provider" refers principally to hospitals in the studies. The studies cited in Table 1 that are relevant to the data extraction categories are described in greater detail in Table 2. In addition to these 21 studies, 5 other articles are cited that describe general issues relating to public disclosure but are not specific to any 1 reporting system.

Table Graphic Jump LocationTable 1. Reporting Systems Evaluated in Peer-Reviewed Publications*
Do Consumers Use Public Reports?

There is a growing body of evidence that consumers want more information about health care provider performance and are able and willing to identify the content and format of the information of greatest use to them.9,36 However, evidence suggests that the information has only a limited impact on consumer decision making.

Consumer use of the Pennsylvania report card, which publishes hospital- and surgeon-specific data on coronary artery bypass graft (CABG) mortality, was assessed using a telephone survey of patients who had undergone CABG surgery in the previous year in 4 Pennsylvania hospitals.12 Two of the hospitals had lower and 1 higher than expected mortality rates. Of 474 patients surveyed, 56 were aware of the performance report at the time of their surgery and less than one quarter of these stated that it had any significant impact on their choice of surgeon. Most of these patients were unable to correctly specify the data on which they based their choice of surgeon. When the report was described, more than half of the respondents said that the information would have influenced their choice of surgeon, but there was a low level of willingness to pay for it.

Hibbard and Weeks11 studied the impact of comparative data about physician fees on consumer use of services. The authors randomly allocated Medicare enrollees and state employees in Oregon to receive information about physician fees. No statistically significant difference in physician visit rate, expenditures on ambulatory care, or costs per visit was observed between the intervention and control groups. Other studies have shown that consumers continue to use hospitals with high mortality rates27,28 and that willingness to use hospitals is more likely to be influenced by anecdotal press reports of untoward deaths in hospitals than by risk-adjusted mortality data.28

The only study suggesting that public disclosure had a significant effect on consumer decision making evaluated the impact of the New York State CABG mortality report on hospital and physician market share and price changes before and after publication of the data.19 This quasi-experimental study indicated that hospitals and physicians with better outcomes experienced higher rates of growth in market share and that physicians with better outcomes had higher rates of growth of charges for the procedure. The magnitude of the association declined over time, suggesting that the market responded primarily to new information.

The reasons for consumers' lack of interest in and use of performance data include difficulty in understanding the information,35,37 disinterest in the nature of the information available,39 lack of trust in the data,36 problems with timely access to the information,12 and lack of choice.40 There is evidence from descriptive studies that consumers rate anecdotal evidence from family and friends more highly than empirical evidence.28,36

Do Purchasers Use Public Reports?

Employers are the main purchasers of health care in the United States, both as individual corporations and in coalitions that have considerable aggregate purchasing power. Policy analysts have expected these coalitions to be more responsive to public performance data than individual consumers, but current evidence suggests that public disclosure has only a small, although possibly increasing, effect on purchasing behavior.

The National Committee for Quality Assurance (NCQA) is responsible for 1 of the highest-profile reporting systems in the United States, the Health Employer Data Information Set (HEDIS) and the NCQA accreditation program. Its impact on employer choice of plan was examined by surveying large US employers.29 The results were compared with a similar survey conducted by the same research team the previous year. The percentage of employers familiar with accreditation increased from 29% to 35%. Of employers, 11% considered NCQA accreditation to be very important, and 5% considered HEDIS data to be very important. Less than 10% required NCQA accreditation for plan selection and 1% provided the HEDIS data to their employees to help them choose a plan. Only marginally greater use of performance data was found in a similar study by Hibbard et al.18

Do Physicians Use Public Reports?

The acceptability to and use of public performance data by physicians may be an important factor in the successful implementation of a public disclosure policy. The evidence suggests that physicians are interested in but skeptical about such data and that they consider it to be of minimal use.

A survey of cardiologists' attitudes to and use of the New York State CABG mortality data revealed that most cardiologists found the reports easy to read and considered the data to be very or somewhat accurate.32 Twenty-two percent routinely discussed the data with their patients, and 38% considered it to have influenced their referral pattern. The most common objection to the reports was a perception that it discouraged cardiac surgeons from operating on high-risk patients.

A survey of cardiologists' and cardiac surgeons' responses to the Pennsylvania CABG reporting system produced similar results23—a minority considered the data to be important when assessing the performance of a particular surgeon or discussed the data with their patients. Almost two thirds of cardiologists reported increasing problems finding surgeons to operate on high-risk patients, and the same proportion of cardiac surgeons reported that they were less willing to operate on such patients.

Do Hospitals and Other Provider Organizations Use Public Reports?

There is some evidence that provider organizations are more responsive to performance data than consumers, purchasers, or individual physicians.

A quasi-experimental design was used to examine the impact of an obstetrics consumer report on hospital behavior in Missouri.14 Half of the hospitals that did not have an infant car seat program, formal transfer arrangements, or breastfeeding nurse educators prior to publication of the report instituted or planned these services after the report was published. Hospitals in competitive markets were twice as likely to implement changes than those with monopolies. Several clinical outcome indicators improved after publication, including patient satisfaction and cesarean delivery rates.

Studies conducted in organizations exposed to publication of data about their performance have produced mixed assessments of the use of the information. Some organizations, particularly those whose performance is shown in a positive light by the reports, use the information for benchmarking, to promote collaboration across departments, and for internal monitoring of performance.8,20,33 In contrast, other organizations respond more negatively, particularly in the early stages of release of the data.15,22,24,25 Providers that have been identified as poor performers are more likely to criticize the validity of the data and the resource implications of acting on it.24

Impact on Quality of Care Outcomes

The most fundamental question relating to public disclosure is the extent to which it influences outcomes of care. Three studies specifically attempted to address this issue, all of which used quasi-experimental designs.

Hannan et al10 studied all patients undergoing isolated CABG surgery who were discharged between 1989 and 1992 from New York hospitals where the procedure was performed. A clinical database was used to identify significant independent risk factors. Actual mortality decreased by 21% during the study period. Illness severity of the patients being operated on increased during this time so that risk-adjusted mortality decreased even further, by 41%, which was considerably higher than the national average. The study occasioned considerable debate among academics, and several further studies attempted to support or refute the authors' conclusions that publication made a significant contribution to the observed improvement.31,34,4144

Potential explanations for the New York data have been suggested31 and relate largely to changes in hospital policy resulting from public disclosure.30,43 First, an exodus of low-volume and high-mortality surgeons followed the publication of the data, probably as a result of hospitals restricting operating privileges. Second, a marked improvement in the performance of non–low-volume surgeons was observed. Finally, there was an improvement in the performance of surgeons new to the system.

The main criticism of the release of data in New York was that the data could have reduced access to CABG surgery by forcing sicker patients to seek surgery outside the state or by surgeons refusing to operate on high-risk patients. Peterson et al34 used national Medicare data to examine trends in the percentage of New York residents receiving out-of-state surgery before and after initiation of the provider profiling program. They also examined procedure use by elderly patients with myocardial infarctions within the state to determine whether high-risk patients were being refused treatment. Contrary to a previous single-center study,41 Peterson and colleagues found that the percentage of New York residents receiving out-of-state bypass operations decreased between 1987 and 1992 and the likelihood of bypass surgery following myocardial infarctions increased. They confirmed a reduction in 30-day mortality in New York well above the national average during the period studied. The only other area with similar figures was northern New England, where provider profile reports were also provided, although for internal use rather than for publication.45

In an evaluation of the Cleveland Health Quality Choice project, Rosenthal and colleagues26 reported improved outcomes associated with publication of mortality data for 6 common medical conditions and 2 surgical operations. The authors examined 101,060 discharges from 30 northeastern Ohio hospitals and found significant and sustained reductions in risk-adjusted mortality for most conditions following publication.

Impact on Costs

There has been little evaluation of the costs of publishing performance data, in part because some of the data are collected anyway for internal evaluation and administrative purposes and because of the multiple contributors to data collection, analysis, and dissemination. The only published estimate that could be found described the costs of producing data for the Pennsylvania reporting system.38 Average costs were estimated to be $16 per patient discharge, at 1991 prices, to collect and report the data. Costs depend on the data system used, but given the initial costs of the development of measures, analytical methods, and data management systems and ongoing costs of data collection, analysis, auditing, dissemination, and management of the responses, this figure is likely to be an underestimate of costs at 2000 prices.

Public disclosure of performance data is a major health policy initiative that represents a potential challenge to health professionals' traditional concept of autonomy.46 It is therefore surprising that after more than a decade of experience of public disclosure in the United States, so few current reporting systems have been the subject of so little published empirical evaluation.

There are several possible reasons for this lack of examination. One is an inevitable time lag between the introduction and evaluation of a new initiative. In addition, some people may regard public disclosure as so obviously the right thing to do that formal evaluation of its impact is not necessary, or the "political correctness" of challenging a tool of informed consumerism may discourage candid appraisal. Also, an evaluation of current reporting systems may be being delayed until more sophisticated data are available, and vested business interests may feel threatened by formal evaluation of an area in which there is considerable business potential. Moreover, funding agencies may not see evaluation as a priority, and the methodological problems of researching such complex interventions may be a deterrent.7

Whether the limited evidence summarized in this article is regarded as supportive of, or counter to, a policy promoting greater public disclosure is open to debate. Those who are in favor will focus on the small but significant gains and will question the sensitivity of conventional research methods to measure subtle improvements in quality of care. They may also point out that the main impact of the information will not be seen for several years, particularly if the quality of the data continues to improve. Antagonists will focus on the negative findings and will emphasize the small gains in relation to the considerable public and private investment of time and resources.

Figure 1 summarizes the issues pertaining to public disclosure in a conceptual framework. The evidence does not provide strong support for those who believe that public disclosure should focus on the role of the consumer or purchaser. Neither individual consumers nor group purchasers appear to search out, understand, or use the currently available information to any significant extent. If consumers are regarded by some as a key audience, then performance data should perhaps be aimed more at consumer advocates and interest groups than at individual consumers. The skepticism of physicians suggests that they too are unlikely to be key players in promoting or encouraging more effective use of publicly available data. The key stakeholders seem to be the provider organizations because the available evidence suggests that publication of performance data results in provider behavior change that contributes to observed improvement in both the processes and outcomes of care. Provider organizations are sensitive to their public image and potential legal risk and often have the authority to act on suboptimal levels of performance and promote good standards of practice. They therefore respond to performance data as a competitive opportunity or risk management imperative even though the purchasing side of the market is less sensitive to it. An intrinsic professional competitiveness on the part of the provider organizations, based on a desire to be seen to be performing well, might therefore be just as important a motivator as economic gain. If this is the case, the reasons for organizational providers being more sensitive to the data than individual physicians warrants further investigation.

Figure. Conceptual Model of Public Disclosure
Graphic Jump Location

This review of the published research reveals many unanswered questions. First, while few would question the importance of performance data as one component of quality improvement, there is considerable uncertainty about the relative merits and risks of using such data internally or making it publicly available. Second, the unintended and potentially adverse consequences of public disclosure have received minimal attention. There is anecdotal evidence that making performance data public can result in gaming and an inappropriate focus on what is being measured.47,48 The negative impact of publication of poor performance on public trust and professional morale6 has also been highlighted, but these issues require empirical investigation. Third, there is considerable doubt about the most effective and appropriate level for the reporting of performance data and the degree of risk adjustment of health outcomes required to achieve a balance between cost, effectiveness, and fairness to providers.49,50

The philosophical stance of those who evaluate the impact of public disclosure should be explicit. The narrowest perspective, and the one that has been most commonly adopted, would view public disclosure as a specific intervention and consider only those benefits and costs that relate directly to release of the data. A broader perspective, analogous to other examples relating to freedom of information, would consider implications for society, such as benefits arising from public discussion, damage to professional reputations, and resource implications resulting from demands to see the "best" provider. The broadest perspective would consider disclosure as a right in a democratic society, irrespective of the costs, and focus on release of data as an intended outcome, rather than a controllable intervention.

An increasing level of public disclosure of health care performance data is inevitable and reflects greater demands for information and accountability in all walks of life. We believe that the use of public performance data by consumers and purchasers or for regulation purposes, will remain relatively less important for the foreseeable future than use of the data as a catalyst to stimulate and promote internal quality improvement mechanisms at the level of the organizational provider. Articulation of such a clear purpose for public disclosure and the development of an evidence base to guide its implementation will ensure that disclosure is conducted in a way that will maximize the potential benefits and reduce the associated risks.

Davies HT, Marshall MN. Public disclosure of performance data.  Lancet.1999;353:1639-1640.
Epstein A. Rolling down the runway: the challenges ahead for quality report cards.  JAMA.1998;279:1691-1696.
Green J, Wintfield N. Report cards on cardiac surgeons.  N Engl J Med.1995;332:1229-1232.
Kassirer JP. The use and abuse of practice profiles.  N Engl J Med.1994;330:634-635.
Lansky D. Overview: performance measures.  Jt Comm J Qual Improv.1996;22:439-442.
Davies HT, Lampel J. Trust in performance indicators.  Qual Health Care.1998;7:159-162.
Thomson R, Lally J. Clinical indicators.  Qual Health Care.1998;7:122.
Bentley JM, Nash DB. How Pennsylvania hospitals have responded to publicly released reports on coronary artery bypass graft surgery.  Jt Comm J Qual Improv.1998;24:40-49.
Edgman-Levitan S, Cleary PD. What information do consumers want and need?  Health Aff (Millwood).1996;15:42-56.
Hannan EL, Kilburn H, Racz M, Shields E, Chassin MR. Improving the outcomes of coronary artery bypass surgery in New York State.  JAMA.1994;271:761-766.
Hibbard JH, Weeks EC. Does the dissemination of comparative data on physician fees affect consumer use of services?  Med Care.1989;27:1167-1174.
Schneider EC, Epstein AM. Use of public performance reports.  JAMA.1998;279:1638-1642.
Lansky D. Measuring what matters to the public.  Health Aff (Millwood).1998;17:40-41.
Longo DR, Land G, Schramm W, Fraas J, Hoskins B, Howell V. Consumer reports in health care.  JAMA.1997;278:1579-1584.
Berwick DM, Wald DL. Hospital leaders' opinions of the HCFA mortality data.  JAMA.1990;263:247-249.
Sirio CA, McGee JL. Public reporting of clinical outcomes.  Am J Med Qual.1996;11:S78-S81.
Brook RH. Health reform is on its way.  Ann Intern Med.1994;120:84-86.
Hibbard JH, Jewett JJ, Legnini MW, Tusler M. Choosing a health plan.  Health Aff (Millwood).1997;16:172-180.
Mukamel DB, Mushlin AI. Quality of care information makes a difference.  Med Care.1998;36:945-954.
Rosenthal GE, Hammar PJ, Way LE.  et al.  Using hospital performance data in quality improvement: the Cleveland Health Quality Choice experience.  Jt Comm J Qual Improv.1998;24:347-360.
Topol EJ, Califf RM. Scorecard cardiovascular medicine: its impact and future directions.  Ann Intern Med.1994;120:65-70.
Rainwater JA, Romano PS, Antonius DM. The California Hospital Outcomes Project.  Jt Comm J Qual Improv.1998;24:31-39.
Schneider EC, Epstein AM. Influence of cardiac-surgery performance reports on referral practices and access to care.  N Engl J Med.1996;335:251-256.
Luce JM, Thiel GD, Holland MR.  et al.  Use of risk-adjusted outcome data for quality improvement by public hospitals.  West J Med.1996;164:410-414.
Romano PS, Rainwater JA, Antonius D. Grading the graders: how hospitals in California and New York perceive and interpret their report cards.  Med Care.1999;37:295-305.
Rosenthal GE, Quinn L, Harper DL. Declines in hospital mortality associated with a regional initiative to measure hospital performance.  Am J Med Qual.1997;12:103-112.
Vladeck BC, Goodwin EJ, Myers LP, Sinisi M. Consumers and hospital use: the HCFA "death list."  Health Aff (Millwood).1988;7:122-125.
Mennemeyer ST, Morrisey MA, Howard LZ. Death and reputation: how consumers acted upon HCFA mortality information.  Inquiry.1997;34:117-128.
Gabel JR, Hunt KA, Hurst K, Peat Marwick KP. When Employers Choose Health Plans. New York, NY: Commonwealth Fund; 1998. Report 293.
Hannan EL, Kumar D, Racz M.  et al.  New York State's Cardiac Surgery Reporting System.  Ann Thorac Surg.1994;58:1852-1857.
Hannan EL, Siu AL, Kumar D, Kilburn H, Chassin MR. The decline in coronary artery bypass graft mortality in New York State.  JAMA.1995;273:209-213.
Hannan EL, Stone CC, Biddle TL, DeBuono BA. Public release of cardiac surgery outcomes data in New York.  Am Heart J.1997;134:55-61.
Dziuban SW, McIlduff JB, Miller SJ, Dal Col RH. How a New York cardiac surgery program uses outcomes data.  Ann Thorac Surg.1994;58:1871-1876.
Peterson ED, Delong ER, Jollis JG, Muhlbaier LH, Mark DB. The effects of New York's bypass surgery provider profiling on access to care and patient outcomes in the elderly.  J Am Coll Cardiol.1998;32:993-999.
Jewett JJ, Hibbard JH. Comprehension of quality of care indicators.  Health Care Financ Rev.1996;18:75-94.
Robinson S, Brodie M. Understanding the quality challenge for health consumers: the Kaiser/AHCPR survey.  Jt Comm J Qual Improv.1997;23:239-244.
Hibbard JH, Jewett JJ, Engelmann S, Tusler M. Can Medicare beneficiaries make informed choices?  Health Aff (Millwood).1998;17:181-193.
US General Accounting Office.  Health Care Reform: "Report Cards" Are Useful but Significant Issues Need to Be AddressedWashington, DC: US General Accounting Office; 1994. Publication HEHS-94-219.
Goldfield N, Larson C, Roblin D.  et al.  The content of report cards.  Jt Comm J Qual Improv.1999;25:423-433.
Schoen C, Davis K. Erosion of Employer-Sponsored Health Insurance Coverage and Quality [periodic brief]. New York, NY: Commonwealth Fund; 1998.
Omoigui NA, Miller DP, Brown KJ.  et al.  Outmigration for coronary bypass surgery in an era of public dissemination of clinical outcomes.  Circulation.1996;93:27-33.
Hannan EL, Siu AL, Kumar D.  et al.  Assessment of coronary artery bypass graft surgery performance in New York.  Med Care.1997;35:49-56.
Chassin MR, Hannan EL, DeBuono BA. Benefits and hazards of reporting medical outcomes publicly.  N Engl J Med.1996;334:394-398.
Ghali WA, Ash AS, Hall RE, Moskowitz MA. Statewide quality improvement initiatives and mortality after cardiac surgery.  JAMA.1997;277:379-382.
O'Connor GT, Plume SK, Olmstead EM, Morton JR, Maloney CT. A regional intervention to improve the hospital mortality associated with coronary artery bypass graft surgery.  JAMA.1996;275:841-846.
Marshall MN, Shekelle PG, Leatherman S, Brook RH. The Publication of Performance Data in Health CareLondon, England: Nuffield Trust. In press.
Goddard M, Mannion R, Smith PC. The NHS Performance FrameworkYork, England: Centre for Health Economics; 1998. The York series on the National Health Service White Paper, No. 158.
Smith P. On the unintended consequences of publishing performance data in the public sector.  Int J Public Adm.1995;18:277-310.
Iezzoni LI. The risks of risk adjustment.  JAMA.1997;278:1600-1607.
Iezzoni LI. Risk Adjustment for Measuring Health Care Outcomes2nd ed. Chicago, Ill: Health Administration Press; 1997.

Figures

Figure. Conceptual Model of Public Disclosure
Graphic Jump Location

References

Davies HT, Marshall MN. Public disclosure of performance data.  Lancet.1999;353:1639-1640.
Epstein A. Rolling down the runway: the challenges ahead for quality report cards.  JAMA.1998;279:1691-1696.
Green J, Wintfield N. Report cards on cardiac surgeons.  N Engl J Med.1995;332:1229-1232.
Kassirer JP. The use and abuse of practice profiles.  N Engl J Med.1994;330:634-635.
Lansky D. Overview: performance measures.  Jt Comm J Qual Improv.1996;22:439-442.
Davies HT, Lampel J. Trust in performance indicators.  Qual Health Care.1998;7:159-162.
Thomson R, Lally J. Clinical indicators.  Qual Health Care.1998;7:122.
Bentley JM, Nash DB. How Pennsylvania hospitals have responded to publicly released reports on coronary artery bypass graft surgery.  Jt Comm J Qual Improv.1998;24:40-49.
Edgman-Levitan S, Cleary PD. What information do consumers want and need?  Health Aff (Millwood).1996;15:42-56.
Hannan EL, Kilburn H, Racz M, Shields E, Chassin MR. Improving the outcomes of coronary artery bypass surgery in New York State.  JAMA.1994;271:761-766.
Hibbard JH, Weeks EC. Does the dissemination of comparative data on physician fees affect consumer use of services?  Med Care.1989;27:1167-1174.
Schneider EC, Epstein AM. Use of public performance reports.  JAMA.1998;279:1638-1642.
Lansky D. Measuring what matters to the public.  Health Aff (Millwood).1998;17:40-41.
Longo DR, Land G, Schramm W, Fraas J, Hoskins B, Howell V. Consumer reports in health care.  JAMA.1997;278:1579-1584.
Berwick DM, Wald DL. Hospital leaders' opinions of the HCFA mortality data.  JAMA.1990;263:247-249.
Sirio CA, McGee JL. Public reporting of clinical outcomes.  Am J Med Qual.1996;11:S78-S81.
Brook RH. Health reform is on its way.  Ann Intern Med.1994;120:84-86.
Hibbard JH, Jewett JJ, Legnini MW, Tusler M. Choosing a health plan.  Health Aff (Millwood).1997;16:172-180.
Mukamel DB, Mushlin AI. Quality of care information makes a difference.  Med Care.1998;36:945-954.
Rosenthal GE, Hammar PJ, Way LE.  et al.  Using hospital performance data in quality improvement: the Cleveland Health Quality Choice experience.  Jt Comm J Qual Improv.1998;24:347-360.
Topol EJ, Califf RM. Scorecard cardiovascular medicine: its impact and future directions.  Ann Intern Med.1994;120:65-70.
Rainwater JA, Romano PS, Antonius DM. The California Hospital Outcomes Project.  Jt Comm J Qual Improv.1998;24:31-39.
Schneider EC, Epstein AM. Influence of cardiac-surgery performance reports on referral practices and access to care.  N Engl J Med.1996;335:251-256.
Luce JM, Thiel GD, Holland MR.  et al.  Use of risk-adjusted outcome data for quality improvement by public hospitals.  West J Med.1996;164:410-414.
Romano PS, Rainwater JA, Antonius D. Grading the graders: how hospitals in California and New York perceive and interpret their report cards.  Med Care.1999;37:295-305.
Rosenthal GE, Quinn L, Harper DL. Declines in hospital mortality associated with a regional initiative to measure hospital performance.  Am J Med Qual.1997;12:103-112.
Vladeck BC, Goodwin EJ, Myers LP, Sinisi M. Consumers and hospital use: the HCFA "death list."  Health Aff (Millwood).1988;7:122-125.
Mennemeyer ST, Morrisey MA, Howard LZ. Death and reputation: how consumers acted upon HCFA mortality information.  Inquiry.1997;34:117-128.
Gabel JR, Hunt KA, Hurst K, Peat Marwick KP. When Employers Choose Health Plans. New York, NY: Commonwealth Fund; 1998. Report 293.
Hannan EL, Kumar D, Racz M.  et al.  New York State's Cardiac Surgery Reporting System.  Ann Thorac Surg.1994;58:1852-1857.
Hannan EL, Siu AL, Kumar D, Kilburn H, Chassin MR. The decline in coronary artery bypass graft mortality in New York State.  JAMA.1995;273:209-213.
Hannan EL, Stone CC, Biddle TL, DeBuono BA. Public release of cardiac surgery outcomes data in New York.  Am Heart J.1997;134:55-61.
Dziuban SW, McIlduff JB, Miller SJ, Dal Col RH. How a New York cardiac surgery program uses outcomes data.  Ann Thorac Surg.1994;58:1871-1876.
Peterson ED, Delong ER, Jollis JG, Muhlbaier LH, Mark DB. The effects of New York's bypass surgery provider profiling on access to care and patient outcomes in the elderly.  J Am Coll Cardiol.1998;32:993-999.
Jewett JJ, Hibbard JH. Comprehension of quality of care indicators.  Health Care Financ Rev.1996;18:75-94.
Robinson S, Brodie M. Understanding the quality challenge for health consumers: the Kaiser/AHCPR survey.  Jt Comm J Qual Improv.1997;23:239-244.
Hibbard JH, Jewett JJ, Engelmann S, Tusler M. Can Medicare beneficiaries make informed choices?  Health Aff (Millwood).1998;17:181-193.
US General Accounting Office.  Health Care Reform: "Report Cards" Are Useful but Significant Issues Need to Be AddressedWashington, DC: US General Accounting Office; 1994. Publication HEHS-94-219.
Goldfield N, Larson C, Roblin D.  et al.  The content of report cards.  Jt Comm J Qual Improv.1999;25:423-433.
Schoen C, Davis K. Erosion of Employer-Sponsored Health Insurance Coverage and Quality [periodic brief]. New York, NY: Commonwealth Fund; 1998.
Omoigui NA, Miller DP, Brown KJ.  et al.  Outmigration for coronary bypass surgery in an era of public dissemination of clinical outcomes.  Circulation.1996;93:27-33.
Hannan EL, Siu AL, Kumar D.  et al.  Assessment of coronary artery bypass graft surgery performance in New York.  Med Care.1997;35:49-56.
Chassin MR, Hannan EL, DeBuono BA. Benefits and hazards of reporting medical outcomes publicly.  N Engl J Med.1996;334:394-398.
Ghali WA, Ash AS, Hall RE, Moskowitz MA. Statewide quality improvement initiatives and mortality after cardiac surgery.  JAMA.1997;277:379-382.
O'Connor GT, Plume SK, Olmstead EM, Morton JR, Maloney CT. A regional intervention to improve the hospital mortality associated with coronary artery bypass graft surgery.  JAMA.1996;275:841-846.
Marshall MN, Shekelle PG, Leatherman S, Brook RH. The Publication of Performance Data in Health CareLondon, England: Nuffield Trust. In press.
Goddard M, Mannion R, Smith PC. The NHS Performance FrameworkYork, England: Centre for Health Economics; 1998. The York series on the National Health Service White Paper, No. 158.
Smith P. On the unintended consequences of publishing performance data in the public sector.  Int J Public Adm.1995;18:277-310.
Iezzoni LI. The risks of risk adjustment.  JAMA.1997;278:1600-1607.
Iezzoni LI. Risk Adjustment for Measuring Health Care Outcomes2nd ed. Chicago, Ill: Health Administration Press; 1997.

Letters

CME
Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.

Multimedia

Some tools below are only available to our subscribers or users with an online account.

Web of Science® Times Cited: 418

Related Content

Customize your page view by dragging & repositioning the boxes below.

See Also...
Articles Related By Topic
Related Collections
PubMed Articles
JAMAevidence.com

Users' Guides to the Medical Literature
Chapter e23. How to Use an Article About Quality Improvement