Expected and Unanticipated Consequences of the Quality and Information Technology Revolutions

Nearly 2 decades after the prediction that an age of “assessment and accountability” would soon transform health care,¹ the quality movement has finally arrived. Driven by evidence that US medical practice comports with best evidence approximately half the time,² that large numbers of medical errors continue to occur,³ and that clinically indefensible disparities in care exist across regions⁴ and racial and ethnic groups,⁵ health care payers and the US government have decided that quality should be measured, publicly reported, and perhaps even compensated differentially. The latter trend, known as pay-for-performance, has been strongly endorsed by the Centers for Medicare & Medicaid Services.⁶

The scientific underpinnings of quality measurement are rising to the task. Donabedian's insight⁷ to divide quality into structure (how care is organized), process (what was done), and outcomes (what happened to the patient) is the quality movement's scaffolding. Its planks are thousands of research studies linking certain structures or processes to clinically meaningful outcomes. For example, taking advantage of such studies, the Centers for Medicare & Medicaid Services now measures the quality of care for hospitalized patients with pneumonia by time to administration of the first dose of antibiotics, and whether patients received guideline-concordant antibiotics, influenza vaccination, pneumococcal vaccination, and smoking cessation counseling.⁸

After decades of traversing a snail-like adoption curve, computerization is also on the verge of fundamentally altering medical practice, partly because of this quality revolution. The labor costs of retrieving and reviewing medical records to gather the required quality data and the ineffectiveness of non–systems-based solutions for improving performance are driving institutions to recognize the need to track clinical processes and outcomes and to prompt clinicians via computerized systems.⁹ That, coupled with the omnipresence of computers in virtually all other aspects of modern life, is promoting the rapid adoption of electronic medical records and computerized physician order entry (CPOE) in health care.¹⁰

Against this background, this article aims to help individuals and institutions to understand and better anticipate some of the potential outcomes, both expected and potentially unforeseen, of the quality and information technology revolutions.

The simple act of defining excellence, measuring it, and disseminating the results skews the system. Although diverse measures sometimes catalyze improvements in unmeasured areas through fundamental system redesign,¹¹ this is an unusual outcome. More typically, individuals and institutions begin to focus on improving their performance on the variables measured, in doing so turning away from others. This “playing for the test” is not only expected; in some cases, it is the point of the whole exercise.¹²

A few examples from my own institution (University of California San Francisco [UCSF] Medical Center) may help illustrate this point. Our rate of administration of pneumococcal vaccine to inpatients was historically quite low (10% in 2003), in part because a 2001 literature review of the practice performed by some of our faculty found a relatively limited benefit.¹³ Once public reporting began, the low rate of pneumococcal vaccination performance transitioned from factoid to embarrassment (because these data were available on the Internet), which led to aggressive efforts to improve the institution's rates on this measure, accompanied by similar efforts focused on the other publicly reported quality measures. These efforts, which are being undertaken around the United States, led to some interesting issues.

First, like many hospitals, the UCSF Medical Center's information system often fails to accurately chronicle patients' history of pneumococcal vaccination. In the past, this uncertainty gave clinicians pause before vaccinating inpatients, because physicians wanted to avoid unnecessary vaccinations. Now, unless the physician is certain that the patient has received the vaccine in the past, it is usually administered. Consequently, many patients inappropriately receive multiple doses.

A second example involves the quality measure of time required to administer the initial dose of antibiotics for a patient with pneumonia (the benchmark is <4 hours after hospital arrival). When a patient presents to the emergency department with shortness of breath, cough, and infiltrates on chest radiograph, it may take several hours to establish whether the patient has pneumonia, heart failure, pulmonary embolism, or another diagnosis. However, quality measurement also changes the threshold for administration of the first dose of antibiotics, often persuading clinicians that the downside of delaying antibiotics for the patient who ultimately proves to have pneumonia (poor performance on the public quality report) is greater than the negative consequence for the patient with heart failure who receives antibiotics unnecessarily.

A third example is that nonphysician case managers have been employed in many institutions to help remind clinicians about publicly measured care processes. At the UCSF Medical Center, there have been instances in which case managers, focusing on the measured processes, have queried physicians about a patient's influenza vaccine status or smoking cessation documentation before the patient's septic shock or acute myocardial infarction has been addressed. Even if administering pneumococcal vaccine to inpatients is an appropriate quality marker, it certainly is not the most important problem in a seriously ill patient with pneumonia.

Another problem is that the current science of quality measurement is bewildered by the patient with multiorgan system disease, which is characteristic of most patients who are admitted to hospitals, as well as of many elderly outpatients. The difficulty in applying today's quality measures for these patients was illustrated by Boyd et al¹⁴ in an article that described a hypothetical 79-year-old woman with 5 common diseases: hypertension, osteoporosis, osteoarthritis, type 2 diabetes mellitus, and chronic obstructive pulmonary disease. Had this patient received guideline-concordant therapy, she would have been administered 13 medicines, costing more than $5000 per year, and with more than 20 potential drug-disease, drug-drug, and drug-diet interactions.¹⁴ The health care practitioner prescribing this polypharmacy would receive a high ranking on quality measurement metrics, even if the adherence to clinical guidelines would have harmed or bankrupted this patient.

These problems with complexity and multisystem illness underlie many physicians' objections to some facets of the quality movement. Administering pneumococcal vaccine and appropriate antibiotics to patients with pneumonia might be aspects of quality care. However, there certainly are clinicians who forget these interventions periodically, but provide superb care to patients with multiple clinical, ethical, and psychosocial problems. How can this problem be resolved? First, the science of quality measurement must mature to permit the assessment of quality of care for patients with multiple disorders. Second, clinicians' abilities also should be measured in other key areas, such as treating acutely ill patients with multiorgan illness, working effectively in teams, and safely performing procedures, and by incorporating measures such as maintenance of board certification and performance in simulated scenarios with simulated patients.^{15 - 16}

The early literature on computerizing health care, particularly the medication prescribing process, was uniformly favorable.^{17 - 18} However, these studies generally were conducted in a small number of institutions that built homegrown computer systems over decades and were staffed with physicians, researchers, and administrators who believed strongly in the value of the systems.¹⁹ The true test came when less committed institutions began installing off-the-shelf systems, a change that has been accompanied by reports of many unforeseen consequences. Some of these include clinicians selecting incorrect medications from computerized pick-lists and an example of a patient without diabetes who was nearly injected with a large dose of insulin after his bar-coded wrist-band was inadvertently switched with that of a patient with diabetes (the computer faithfully listed the bedside glucose test results of the patient with diabetes under the record of the patient without diabetes).^{20 - 21} A recent study reported a 3-fold increase in mortality among critically ill children after the installation of a popular commercial computer system, mostly attributed to changed clinician workflow patterns.²² For example, both physicians and nurses were required to spend excessive time at the computer screen when they previously would have been at their patients' bedsides.

Just as the early literature on computerization was misleadingly positive, this new literature may be unduly pessimistic. In implementing systems as complex as CPOE, problems are inevitable and working through them may be the only path to ultimate improvement. Rather than slowing down computerization, it will be important for clinicians, health care organizations, regulators, payers, and vendors to collectively appreciate the negative consequences of poorly designed systems and rapidly improve the systems to minimize any transitional harm. In this regard, computerization is similar to resident duty hours regulations, which may have negative effects on safety in the short term (by necessitating more hand-offs) but will undoubtedly improve patient safety and resident well-being after these transitions are improved.

Among health care's computer implementation disasters, the problems that beset Cedars-Sinai Medical Center, Los Angeles, California, when it adopted an expensive homegrown CPOE system have become legendary in information technology circles. Within a few weeks after the CPOE system was launched in late 2002, physicians all but threatened to strike if the system was not turned off.²³ The problem was attributed to layers of decision support that forced physicians who wanted to order vancomycin, for example, to click through several layers of “Are you sure?” screens before the computer would accept the order. The pushback from physicians led to the abandonment of the $34 million system; 3 years later, a replacement system had not yet been installed. Instead, Cedars-Sinai Medical Center hired extra staff to double check physician orders. “We trap potential misadventures,” lamented the hospital's vice president for medical affairs in 2005. “But that could be done better with technology.”²⁴

The Cedars-Sinai Medical Center experience has been framed as a cautionary tale about the need to gain physician buy-in and ensure that CPOE does not compromise efficiency. Another, more fundamental view is that, operating in an environment in which quality (largely measured as adherence to evidence-based processes) is being assessed, publicly reported, and perhaps compensated, every health care organization faces the challenge of ensuring that all of its physicians adhere to these processes. As the stakes grow (eg, from simple transparency to pay-for-performance), the pressure on institutions to assert central control over physicians' practices will become irresistible, and the unique ability of computers to impose this control may become the nidus of clinician-institutional conflict. Physicians are likely to resist challenges to their clinical autonomy by asserting that the wisdom born of their individual experience and the uniqueness of their patients creates the need for decisions that may diverge from guidelines.

The individuals or organizations responsible for embedding decision support and guidelines into computerized systems will need to navigate these challenging waters, taking advantage of the ability of such systems to ruthlessly enforce rules and standards, while not being so prescriptive that they create “alert fatigue,” disabling inefficiency, or physician backlash. Importantly, until the science of guideline development and quality measurement improves, the systems must preserve physicians' abilities to apply the art of medicine when patients do not fit the templates, such as in those patients with multisystem illness or rapidly changing disease courses.

The Cedars-Sinai Medical Center fiasco taught hospitals that pushing decision support too fast might generate a backlash that can disintegrate an entire project. However, even if this lesson drives administrators and decision-makers to create kinder, gentler CPOE (at least during the implementation phase), can there be any doubt that central control of physicians' practice will need to be exercised, especially when there is evidence of substandard performance on publicly reported measures? Although it would be an error to make systems overly intrusive, it also would be a mistake to allow physicians to practice low-quality medicine when higher-quality care could be electronically ensured. The search for the proper balance between these extremes will be one of the most interesting challenges in the next decade of the quality and information technology movements.

The vision (held by both patients and clinicians) of a great physician was imprinted decades ago: the virtuoso diagnostician, practicing the art of medicine, supported by others who followed “orders.” As chronicled by Wolfe in The Right Stuff,²⁵ the early test pilots conceived of themselves in a similarly independent way, as “Top Gun” cowboys who broke rules, lived on the edge, and were lauded for their remarkable skills and fearless swagger. Wolfe also points out that there was one small problem with flying in this era: the pilots' mortality rate was 25% because many of their planes ended as smoking heaps.²⁵

The Right Stuff traces the evolution of the ideal pilot from the gunslinger type, personified by Chuck Yeager, to a very different kind of persona, exemplified by John Glenn (later an Ohio senator). Glenn was a churchgoing engineer and Eagle Scout who believed in regulations, checklists, read-backs, and safety systems. As aviation made this transition, it became far less exciting to be a pilot, just as Yeager and his colleagues feared. But it also became far safer to fly, both for passengers and pilots.

Similarly, providing health care that is predictably safe and evidence-based depends to some degree on excellence of the physician, but just as much on a system that has appropriate rules, regulations, and infrastructure, such as computerized decision support, that directs the physician to do the right thing, even when memory fails, when the to-do list becomes overwhelmingly long, and when distracted by personal issues. Creating such systems will depend on physicians participating in their construction and upkeep.²⁶

Some may argue that physicians who help develop and maintain safety and quality systems are planning for their own obsolescence, commoditizing themselves so that they become interchangeable because, the reasoning goes, any clinician can follow a series of computer-generated directives. There may even be foreboding additional parallels to commercial aviation: as pilots helped design safer systems and protocols, the excellence of the individual captain mattered less than that of the system, leading pilots to become all-but-interchangeable (in modern aviation, it is highly unlikely for any pilot to crash over a 30-year career).²⁷ This commoditization, worry some, may have sowed the seeds for today's pilot pay cuts and job losses. Others are concerned that computerization and standardization will promote the outsourcing of medical services or the growth of alternative models that may exclude physicians.^{28 - 29}

The most important outcome should be creating high-quality, more effective, and safer systems for physicians as well as patients. The well-functioning health care organization of the future is likely to be one in which rote, algorithmic practices (eg, pneumococcal vaccine administration, evidence-based antibiotic choice for pneumonia) are guided by a relatively nonintrusive, user-friendly computer system armed with the appropriate guidelines, accompanied by supportive evidence for each practice. Such systems should appreciate clinical workflow, allow appropriate overrides in special circumstances, and not harm (and perhaps even improve) clinicians' efficiency.

Even if such an optimal system were in place, there will be enough art remaining—the care of patients with complex multidimensional illness, the skillful performance of procedures, and the empathic application of ethical decision-making, patient counseling, care coordination, and palliative care—that physicians' professional lives should become more interesting and fulfilling. More important, the shift toward a system in which patients predictably receive safe, high-quality care would help to meet health care's highest purposes. Although there will continue to be challenges and unforeseen consequences, efforts must be directed to ensuring that the quality and information technology revolutions produce the desired consequences of better patient care and improved health outcomes.