A Computer Alert System to Prevent Injury From Adverse Drug Events:  Development and Evaluation in a Community Teaching Hospital FREE

ADVERSE DRUG EVENTS (ADEs) are the most common type of iatrogenic injury occurring in hospitalized patients.^1- 2 Adverse drug events have been reported to occur during 1% to 30% of hospital admissions, depending on the operational definition of ADE and the rigor with which they are sought.^2- 9 A recent meta-analysis reported an overall incidence of 6.7% for serious adverse drug reactions (a term that excludes injury secondary to errors in prescribing and administration).¹⁰ For every 1000 patients admitted to a hospital, approximately 3 will die^3,10- 11 and 1 will suffer serious long-term disability² due to ADEs. The mean direct cost of an inpatient ADE ranges from $1900 to $5900.^4,6,12

From 28% to 56% of ADEs are preventable,^3- 4,6- 7 and these are most commonly caused by errors in order writing.^3,13 Such errors occur in up to 5% of medication orders.^14- 15 Prescription of the wrong drug or wrong dose is often due to lack of information regarding the drug or the patient.^3,13- 14 A recent study concluded that 78% of errors leading to ADEs are due to systems failures that could be corrected by improved information systems.¹³

We have developed a computer alert system that provides patient-specific information to clinicians, with the specific aim of correcting prescription errors that might lead to ADEs (primary prevention) and detecting ADEs before harm occurs (secondary prevention).

Good Samaritan Regional Medical Center (GSRMC) is a 650-bed teaching hospital and regional referral center in Phoenix, Ariz. In 1994, a group convened at GSRMC to develop a method of reducing ADE-related injury using the decision support capabilities of our hospital information system (Discern Expert, Cerner Corp, Kansas City, Mo). The group included physicians and representatives from pharmacy, clinical pharmacy, nursing, laboratory, and information services.

Our hospital information system contained integrated patient-specific data including demographics, pharmacy orders, drug allergies, radiology orders, and laboratory results. Other clinical information such as major diagnoses andphysicians' notes were not part of this database. Efforts focused on using information from the integrated databases to detect situations that might lead to ADE-related patient injury. The group devised a plan to do so through primary prevention alerts, which detect prescription errors with high potential for resulting in ADEs (eg, inappropriate dosing of imipenem in a patient with renal failure), and secondary prevention alerts, which detect potential ADEs before maximal patient injury has occurred (eg, new-onset thrombocytopenia in a patient receiving heparin sodium).

Specific ADEs were selected for inclusion based on clinical significance and the presence of specific risk factors for injury in our databases. Adverse drug events resulting from drug interactions and allergies were already being addressed at our institution through computerized decision support, and therefore were not included.

Thirty-seven drug- or drug class–specific ADEs were targeted. These are listed in Table 1 and represent the most common categories of ADEs described in the Harvard Medical Practice Study,² with the exception of allergic ADEs.

We developed and pilot tested computer programs to generate alerts for each of the targeted ADEs. The logic statements within the programs each contained a trigger premise (describing a clinical situation in which injury secondary to an ADE might be imminent) and a recommendation to avoid injury (eg, if a verified serum potassium level exceeds 6.0 mmol/L and the patient is receiving potassium chloride, then print an alert recommending discontinuation). Table 1 includes simplified logic for each alert.

Programs involving drug dose adjustment in renal failure use the method described by Jelliffe¹⁶ to estimate creatinine clearance¹⁷ and the American College of Physicians' recommendations for appropriate drug dosing.¹⁸ Programming was performed using Cerner Rule Editor (Cerner Corp). Systems for physician alert notification were developed.

Pharmacists evaluated each alert that printed out in the pharmacy. This involved confirmation of the information that triggered the alert and discussion with nurses regarding the patient's clinical condition when necessary. The pharmacist contacted the attending physician when the alert recommendations seemed appropriate given the clinical situation. Alerts designed to prevent radiocontrast media nephrotoxicity were evaluated by radiology technicians and brought to the attention of the attending radiologist when appropriate.

We collected data on consecutive alerts that fired between July 1, 1997, and January 1, 1998. The pharmacist or radiology technician who contacted the physician recorded (1) whether the physician had already recognized the problem identified by the alert, (2) whether the physician made order changes consistent with alert recommendations, (3) the reason for disagreement if the physician did not make order changes, and (4) the time spent evaluating each alert.

A research nurse prospectively collected this information and confirmed physician order changes by paper chart review. Each firing was classified as a true-positive or false-positive alert based on whether the attending physician wrote orders consistent with the alert recommendations. Systat version 5.2.1 (Systat Inc, Evanston, Ill) was used for all descriptive statistical analyses.

During the 6-month study period, there were 13521 admissions at GSRMC, of which 4215 were labor and delivery admissions. Consistent with a published observation that ADEs are extremely uncommon in obstetrical patients,⁷ there were only 7 alert firings among these patients. The following results apply only to the 9306 nonobstetrical admissions.

The ADE alert system fired 1116 times. In 794 cases, the evaluator felt the alert warranted physician notification. Physicians were not notified when the adverse event was clearly not drug related (eg, thrombocytopenia secondary to disseminated intravascular hemolysis), or when the triggering laboratory result was misleading (eg, hyperkalemia secondary to hemolysis of blood specimen). A total of 596 (53%) of 1116 alerts were true positives. Thus, opportunities to potentially reduce patient injury secondary to ADEs were identified at a rate of 64 per 1000 admissions (596/9306). Physicians stated they were previously unaware of the potentially dangerous clinical situations leading to alert firings in 265 (44%) of the 596 true-positive alerts. The order changes in these patients were directly attributable to alert notification and occurred at a rate of 29 per 1000 admissions (265/9306).

Primary prevention alerts fired 803 times, identifying 490 potential opportunities to prevent ADEs (positive predictive value of 61%) (Table 2). Of these, 238 (49%) were unrecognized by the clinician before alert notification. The most common prevention alert firings were for radiocontrast media nephrotoxicity and digoxin toxicity.

Secondary prevention (early detection) alerts fired 313 times, identifying 106 cases in which the physician agreed that action was required to evaluate or treat a possible ongoing ADE (positive predictive value of 34%) (Table 3). Twenty-seven (25%) of these were previously unrecognized. The most common detection alert firings were for possible pseudomembranous colitis and drug nephrotoxicity.

The most common reasons for false-positive primary prevention alerts were (1) importance of the radiocontrast media study was felt to outweigh the risk of nephrotoxicity (n = 81), (2) disagreement that renal drug clearance was inadequate (n = 26), and (3) planned short-term or as-needed-only use of medications (n = 20). For secondary prevention alerts, the most common cause for a false-positive alert was the determination that the observed complication was not drug related (n = 127).

Incidentally, true-positive alerts were associated with appropriate reductions in drug dosages in 135 patients. Eighty-four of these were previously unrecognized, and resulted in a savings of 254 drug doses (146 doses of antibiotics and 108 doses of nonantibiotic medications). The mean time spent by pharmacy technicians evaluating each alert was 15.9 minutes (SD, 12.8 minutes; range, 0-180 minutes).

Our system detected opportunities to reduce ADE-related injury at a rate of 64 per 1000 patient admissions. Previous measures of this rate are not available for comparison because this is the first study to prospectively evaluate a computer support system with real-time intervention for reducing injury from a broad range of ADEs. However, previous noninterventional studies have quantified the rate of opportunities to prevent ADEs. Leape and colleagues¹³ combined preventable ADEs and potential ADEs (medication errors with the potential to cause ADEs) to determine the total number of preventable events. A rate of 69 per 1000 patient admissions can be calculated from their reported data,^3,13 which is similar to the rate in our study. Others have reported preventable event rates of 106 per 1000 admissions¹⁵ and 117 per 1000 admissions.⁷ Although we would strive to develop a system to circumvent all such preventable events, our set of alerts represents only a subset, and probably includes events that would not be classified as preventable or potential ADEs by other researchers.

Nevertheless, several examples in which our preventive intervention failed to illustrate the serious potential consequences of a true-positive alert. In one instance, an alert identified an elderly woman with renal insufficiency and hyperkalemia who was receiving potassium chloride and quinapril. Use of the medications was discontinued on alert notification; however, the patient suffered a fatal cardiac arrest less than 1 hour later with a serum potassium level of 7.0 mmol/L. Another patient, identified by a pilot alert, was receiving metformin and had a serum creatinine level of more than 350 µmol/L. Within 24 hours, the patient developed fatal lactic acidosis.

Cost considerations are important in determining the generalizability of our approach. In a 1993 survey of 166 hospitals, 83% reported the ability to identify patients based on medications received, but only 30% could integrate this information with laboratory data (a prerequisite for an alert system such as ours).¹⁹ An integrated hospital information system with decision-support capability may cost several hundred thousand to several million dollars, depending on the size of the institution (Steve Hawthorne, Cerner Corp, written communication, September 4, 1998). Our working group spent approximately 400 person-hours developing the specific system described in this article, but the entire process need not be duplicated at every institution implementing such a system. The overall positive predictive value (53%) and the average time spent evaluating an alert (15.9 minutes) suggest that the average incremental cost of each true-positive alert is approximately 30 minutes of pharmacist or radiology technician work time. Given the rate of alert firings, this amounts to approximately one fourth of a full-time equivalent at our institution.

The benefit of an highly effective ADE prevention program can be estimated for a hypothetical 650-bed hospital. If ADEs occur in approximately 7% of admissions,^3,10 1800 would be expected annually. A conservative estimate is that 28% of ADEs, in general, and 42% of life-threatening ADEs are preventable.^3,6 Therefore, a fully functional system might avert 500 ADEs and save 36 lives per year. The average preventable ADE adds $5857 to the cost of hospitalization,⁶ therefore cost savings as high as $3 million annually might be achieved. Prevention of ADEs should also reduce indirect costs associated with disability and medical-legal liability.

Previous studies have demonstrated the utility of computer systems in detecting ADEs.^8,20- 21 Researchers from LDS Hospital in Salt Lake City, Utah, have developed a computerized ADE monitor that detected 80 times more ADEs than conventional self-reporting methods.⁸ The most common method by which this system identified ADEs was the administration of antidotes (such as naloxone). Detection of this type of ADE would not qualify for our definition of secondary prevention, since the ADE in question has already been recognized and treated.

A subsequent study showed that reporting of computer-detected ADEs to physicians resulted in a 65% reduction of severe ADEs compared with historic controls.²² These impressive results were achieved with a focus on allergic and idiosyncratic ADEs. In contrast, we chose to focus on nonallergic ADEs in which real-time intervention might benefit the patient. Studies in which computer-assisted antibiotic dosing has been shown to decrease antibiotic-related ADEs^23- 24 exemplify this approach.

We felt it unethical to design our study with a concurrent, nonintervention control group. Reliable historical controls were not available because the established method of ADE detection at our hospital (self-report) is highly insensitive.^8,25- 26 Therefore this study did not directly measure a reduction in ADE-related injury. Instead, our study relied on changes in physician behavior as the main outcome variable, a common limitation of published research regarding computer-based clinical decision support.²⁷

Caution is warranted when interpretating our aggregate data. Adverse drug event alerts are quite heterogeneous. Some detect rare but immediately life-threatening ADEs (eg, metformin-induced lactic acidosis), and others detect common situations with a lower potential to result in injury (eg, hypokalemia in a patient receiving digoxin). Classen and colleagues⁴ have shown tremendous variation in the clinical and economic impact of various types of ADEs.

Several of our ADE alerts appear to have low sensitivity. Cognitive impairment is a common and important ADE in the elderly,^28- 29 but our delirium alert only identified a few cases. This alert uses pharmacy orders for haloperidol in elderly patients to identify delirium. We are developing methods to enter clinical data (such as alterations in mental status) into our computer database to improve the ability of our system to detect important clinical outcomes that are not well represented in our current databases.

We are also attempting to reduce false-positive alerts through refinement of alert-trigger logic. Excluding chemotherapy patients from thrombocytopenia alerts is an example of how this can be accomplished. Other systems improvements we are implementing include alerts to detect drug-induced pancreatitis and vancomycin administration in patients with methicillin-sensitive Staphylococcus aureus infections.

Computer alert systems can be used to identify opportunities to prevent or reduce patient injury associated with a broad range of ADEs. Prerequisites for a computer ADE alert system such as the one described herein include (1) an integrated computerized database (including clinical, pharmacy, and laboratory data), (2) the ability to program the system to generate alerts when opportunities to prevent injury occur, and (3) reliable clinical systems for physician notification.

Opportunity exists for greatly increasing the scope of computer-assisted decision making in clinical practice. Computer-aided diagnosis, preventive care reminders, and computer-aided quality assurance are examples of computer-based clinical decision support systems that have improved quality.³⁰ Computer systems with online physician order entry would enable decision-support systems to provide potentially critical information to the physician close to the moment of decision making.^31- 33 Improvements in hospital information systems and increasing utilization of this powerful tool by physicians should have an enormous beneficial impact on the quality of medical care.