0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Access to paid content on this site is currently suspended due to excessive activity being detected from your IP address 54.196.119.149. Please contact the publisher to request reinstatement.
Original Contribution |

Surveillance of Medical Device–Related Hazards and Adverse Events in Hospitalized Patients FREE

Matthew H. Samore, MD; R. Scott Evans, MS, PhD; April Lassen, RN; Patricia Gould, RN; James Lloyd, BS; Reed M. Gardner, PhD; Rouett Abouzelof, RN; Carrie Taylor, RN; Don A. Woodbury, MHA; Mary Willy, PhD; Roselie A. Bright, ScD
[+] Author Affiliations

Author Affiliations: University of Utah School of Medicine (Drs Samore, Evans, and Gardner), LDS Hospital/Intermountain Health Care (Drs Samore and Evans, Mss Lassen, Gould, Abouzelof, and Taylor, and Mssrs Lloyd and Woodbury), and Veterans Affairs Salt Lake City Health Care System, Salt Lake City; Center for Drug Evaluation and Research (Dr Willy) and Center for Devices and Radiological Health (Dr Bright), Food and Drug Administration, Rockville, Md. Ms Lassen is now with Alta View Hospital, Sandy, Utah.


JAMA. 2004;291(3):325-334. doi:10.1001/jama.291.3.325.
Text Size: A A A
Published online

Context Although adverse drug events have been extensively evaluated by computer-based surveillance, medical device errors have no comparable surveillance techniques.

Objectives To determine whether computer-based surveillance can reliably identify medical device–related hazards (no known harm to patient) and adverse medical device events (AMDEs; patient experienced harm) and to compare alternative methods of detection of device-related problems.

Design, Setting, and Participants This descriptive study was conducted from January through September 2000 at a 520-bed tertiary teaching institution in the United States with experience in using computer tools to detect and prevent adverse drug events. All 20 441 regular and short-stay patients (excluding obstetric and newborn patients) were included.

Main Outcome Measures Medical device events as detected by computer-based flags, telemetry problem checklists, International Classification of Diseases, Ninth Revision (ICD-9) discharge code (which could include AMDEs present at admission), clinical engineering work logs, and patient survey results were compared with each other and with routine voluntary incident reports to determine frequencies, proportions, positive predictive values, and incidence rates by each technique.

Results Of the 7059 flags triggered, 552 (7.8%) indicate a device-related hazard or AMDE. The estimated 9-month incidence rates (number per 1000 admissions [95% confidence intervals]) for AMDEs were 1.6 (0.9-2.5) for incident reports, 27.7 (24.9-30.7) for computer flags, and 64.6 (60.4-69.1) for ICD-9 discharge codes. Few of these events were detected by more than 1 surveillance method, giving an overall incidence of AMDE detected by at least 1 of these methods of 83.7 per 1000 (95% confidence interval, 78.8-88.6) admissions. The positive predictive value of computer flags for detecting device-related hazards and AMDEs ranged from 0% to 38%.

Conclusions More intensive surveillance methods yielded higher rates of medical device problems than found with traditional voluntary reporting, with little overlap between methods. Several detection methods had low efficiency in detecting AMDEs. The high rate of AMDEs suggests that AMDEs are an important patient safety issue, but additional research is necessary to identify optimal AMDE detection strategies.

Adverse drug events have been studied extensively,17 but the computer-based techniques used to detect adverse drug events have not been used to identify medical device events. Although the US Food and Drug Administration's mandatory and voluntary medical device reporting programs have played an essential role in identifying specific hazards associated with individual types of devices,813 they are limited by underreporting and the absence of denominator data. The diverse array of types of medical devices and comparable variety in potential problems associated with these devices11,1421 create challenges for the development and implementation of comprehensive surveillance programs. Human factors are more important than for drugs because devices have to be operated by a person,2224 and proper use depends on optimal design and instructions.2528

Because surveillance of adverse drug events was facilitated by the application of computer-rule–based methods for screening and detection,5,29,30 we hypothesized that computer-based surveillance could facilitate recognition of device events. To test this hypothesis, we conducted a pilot study of an electronic-flag–based system to detect device use problems that operated within the computerized patient record in the way that the computerized adverse drug event monitor does. Our primary objective was to measure the positive predictive value (PPV) of the electronic rules and to estimate the incidence of problems associated with devices identified by computer-based surveillance compared with postdischarge International Classification of Diseases, Ninth Revision (ICD-9) codes and the existing hospital incident event-reporting system. Secondary methods of detecting problems associated with devices were also evaluated in a pilot fashion: a prospective checklist system for telemetry-related problems, a postdischarge patient satisfaction survey, and a review of the clinical engineering logs.

Study Site

The main part of the study was conducted from January to September 2000 at a 520-bed tertiary teaching institution in the United States with experience in using computer tools to detect and prevent adverse drug events. The study was approved by the hospital's institutional review board. The target patient population comprised all hospitalized patients, excluding obstetric and newborn patients. Short-stay patients were expected to be discharged within a day of admission and included patients undergoing same-day surgery. The hospital's electronic medical record, Health Evaluation through Logical Processing (HELP), consists of an integrated clinical database and a frame-based medical decision system. Approximately 1300 terminals throughout the hospital facilitate routine clinical use. Other systems that interface with HELP include the Medical Information Bus,31,32 pharmacy, billing, laboratory, electrocardiography, medical records, digital radiology, and a collection of local area networks used by a variety of departments for local research and departmental management functions.

Definitions

Problems associated with devices were classified as device-related hazards or adverse medical device events (AMDEs) according to whether patient harm had occurred. Device-related hazards were events that had the potential to cause harm but manifested none, including a device malfunctioning or failing to perform as intended; problems in how a device was used, including misdiagnoses; failure to recognize and act on information from monitoring devices; and improper treatment. Examples were misplacement of a nasogastric tube in the esophagus in the absence of overt clinical consequences, failure of the cardiac monitor to initiate an alarm for an episode of unsustained ventricular tachycardia, false reading of oxygen saturation from a pulse oximeter, and malposition or clotting of a central venous catheter.

AMDEs were defined as any patient harm caused by device-related medical or surgical management rather than the patient's illness. Bleeding and infections associated with surgery were considered device-related only if they were associated with a specific device. A failed device that had to be replaced, if an invasive procedure that carried risk of harm to the patient was necessary to replace it, was considered an AMDE. Examples of AMDEs included electrocautery-device–induced burn; catheter-related bloodstream infection; failure to promptly detect asystole because of monitor failure; misprogramming of an infusion pump, resulting in excessive dosing of intravenous narcotic and leading to oversedation; and loosening of a prosthetic joint, requiring replacement. AMDEs were not classified with respect to preventability or severity because of the limitations of making this assessment on the basis of available data.

Spectrum of Devices and Device-Related Problems

During the study-planning process, commonly used devices were categorized by the part of the body or organ system for which they were used and by functional characteristics such as implantable (eg, artificial hip), reuseable (eg, ventilator), or disposable (eg, catheter). Meetings were held with clinical personnel in various hospital departments to learn which device problems were encountered most often. Nurses were surveyed about personal experiences and attitudes toward medical devices. Device-related incident reports sent to the hospital risk manager during a previous 6-month period were reviewed. Events were reviewed to determine whether they were documented in the patient's electronic or paper record and whether they were amenable to detection by a computer flag. We used this information to create a database of devices, including the spectrum of problems associated with each device, and their causes.

Surveillance Methods

Online Incident Event Reporting. The existing online incident event-reporting system relies on voluntary, nonanonymous submission of error and adverse event cases by health care workers. Incident reports were reviewed to determine whether medical devices were involved and to determine the outcome. Device-related incidents were classified by patient characteristics and medical department.

Computer-Flag–Based Surveillance. Concurrent with the incident event reports, surveillance was conducted with computer flags. Seven categories of computer flags were established: (1) local complications and hazards associated with a variety of catheter types (excluding peripheral intravenous catheters), including line replacement or removal for reasons other than "no longer needed," documented by nurse or intravenous teams; (2) infectious complications of urinary and intravascular catheters (excluding peripheral intravenous catheters) assessed by positive microbiological cultures and corresponding dates of collection; (3) device hazards manifested on chest radiograph (key words searched for included malposition, broken, sheared, tip, bent, "out of place," reposition, folded, and coiled, combined with terms indicating presence of a device, such as wire, device, tube, catheter, pacemaker, line, and lead; pneumothorax was another search term); (4) surgery lasting longer than the 90th percentile for the specific surgery type; (5) abnormal measurements from physiologic monitoring devices such as pulse oximeters, cardiac monitors, and automated sphygmomanometers31,32; (6) decrease in urine output as a proxy for obstruction of a Foley catheter; and (7) bloody sputum as a proxy for traumatic intubation. These categories were selected in part because electronic data were available to establish triggering rules.

These flags were added to the knowledge base and logic modules that had been developed for surveillance of adverse drug events.5,30,33 The flags were pilot tested to establish that electronic data were being appropriately processed. For example, among patients who were flagged as having had a more than 50% decrease in blood pressure, the electronic medical record was reviewed to establish that such a drop in blood pressure had been recorded. In addition, for independently identified instances of a more than 50% decline in blood pressure, appropriate firing of the rule was confirmed.

Research nurses who conducted the computer-flag–based surveillance were trained by individuals with extensive experience in electronic adverse drug event surveillance, in consultation with a US Food and Drug Administration expert on adverse device events (R.A.B.). From January through September 2000, the protocol involved the following procedures: a report was generated twice daily, listing all patients with flags, patient location and demographic information, and a brief summary of other pertinent clinical data. Each day, the research nurses reviewed the reports, assessed data in the electronic and paper medical record to determine whether criteria for an AMDE or device-related hazard were met, and, where possible, solicited additional information from the nurse caring for the patient. All flags verified in the medical record (true positives) were retrospectively reviewed by another nurse reviewer and discussed by the investigative team.

For flags based on physiologic monitoring devices, the goals were (1) to detect episodes of clinical deterioration that were complications of devices; and (2) to detect measurement errors on the part of the monitoring devices. We attempted to identify blood pressure, heart rate, and oxygen saturation values that were possibly due to measurement error and that might have led to erroneous interpretation. We searched for explanations of abnormal hemodynamic values, but we were limited by the lack of direct observation of possible device-related problems. For example, when we identified an instance of a single oxygen saturation value of 70%, we examined the record for evidence of a procedure or an acute condition that might have triggered desaturation or respiratory compromise. If no explanation was forthcoming and other recorded oxygen saturations from the patient before and after the isolated aberrant value were in the normal range, the episode was classified as a device-related hazard (possible measurement error). In contrast, if the physiologic characteristic had a value of zero, it was not categorized as a device-related hazard because it was assumed that such measurements reflected artifact or disconnection from the device. Flags related to catheters were classified as an AMDE when we identified an associated infection or local complication or as a device hazard if the catheter was misplaced or inappropriately detached from the patient.

Criteria for infection were based on National Nosocomial Infection Surveillance definitions.34 Catheter-related urinary tract infection was defined as fever or other sign of infection plus urine culture with a growth of more than 104 colony-forming units per cm3, coupled with the presence of a urinary catheter. Catheter-related bloodstream infection was defined as growth of an organism from 1 or more blood cultures (if coagulase-negative staphylococcus was the organism, at least 2 positive blood culture bottles or sets were required), presence of an intravascular catheter, and clinical signs of infection (fever or hypotension). Hazards detected on chest radiograph were defined as malposition or inappropriate breakage of a device.

Concurrent Method: Telemetry Checklist. Because the computer flags did not identify many of the nurse-reported problems with cardiac monitors, we added a telemetry checklist to the surveillance methods. For a 5-week period, from September 8, 2000, to October 11, 2000, telemetry technicians with the cardiac telemetry unit marked false alarms or missed ectopy during each shift on paper checklists. A false alarm was defined as each instance that the alarm was triggered when no arrhythmia was present, as judged by the telemetry technician. Missed ectopy was defined as each instance that an arrhythmia occurred when the alarm did not fire.

Retrospective Method:ICD-9-Code–Based Surveillance. We identified ICD-9 codes that specified devices in their definitions and therefore were considered to have a high likelihood of indicating a device problem. All were considered AMDEs because it was presumed that only events with overt clinical consequences were likely to be coded. Briefly, we included codes 996.0-996.79, except for codes that referred to tissue grafting or to a specific procedure such as coronary artery bypass surgery (eg, code 996.03, mechanical complication caused by coronary artery bypass graft) rather than a device or prosthetic implant, and injury codes E878.1, E878.2, and E879.4 (Box). The ICD-9 codes were classified according to organ system or device type (eg, genitourinary or orthopedic) or, if the text description was nonspecific, as "other." They also were classified according to their functional characteristics (implantable or disposable).

Box.

996.01 Mechanical complication due to cardiac pacemaker (electrode)
996.02 Mechanical complication due to heart valve prosthesis
996.04 Mechanical complication of automatic implantable cardiac defibrillator
996.09 Other mechanical complication of cardiac device, implant, and graft
996.1 Mechanical complication of other vascular device, implant, and graft
996.2 Mechanical complication of nervous system device, implant, and graft
996.32 Mechanical complication due to intrauterine contraceptive device
996.39 Other mechanical complication of genitourinary device, implant, and graft
996.4 Mechanical complication of internal orthopedic device, implant, and graft
996.53 Mechanical complication of prosthetic ocular lens prosthesis
996.54 Mechanical complication of breast prosthesis
996.56 Mechanical complication due to peritoneal dialysis catheter
996.59 Mechanical complication of other implant and internal device, not else where classified
996.61 Infection and inflammatory reaction due to cardiac device, implant, and graft
996.62 Infection and inflammatory reaction due to other vascular device, im plant, and graft
996.63 Infection and inflammatory reaction due to nervous system device, im plant, and graft
996.64 Infection and inflammatory reaction due to indwelling urinary catheter
996.65 Infection and inflammatory reaction due to other genitourinary device, im plant, and graft
996.66 Infection and inflammatory reaction due to internal joint prosthesis
996.67 Infection and inflammatory reaction due to other internal orthopedic de vice, implant, and graft
996.68 Infection and inflammatory reaction due to peritoneal dialysis catheter
996.69 Infection and inflammatory reaction due to other internal prosthetic de vice, implant, and graft
996.70 Other complications due to unspecified device, implant, and graft
996.71 Other complications due to heart valve prosthesis
996.72 Other complications due to other cardiac device, implant, and graft
996.73 Other complications due to renal dialysis device, implant, and graft
996.74 Other complications due to other vascular device, implant, and graft
996.75 Other complications due to nervous system device, implant, and graft
996.76 Other complications due to genitourinary device, implant, and graft
996.77 Other complications due to internal joint prosthesis
996.78 Other complications due to other internal orthopedic device, implant, and graft
996.79 Other complications due to other internal prosthetic device, implant, and graft
E878.1 Surgical operation with implant of artificial internal device causing abnor mal patient reaction, or later complication, without mention of misadventure at time of operation
E878.2 Surgical operation with anastomosis, bypass, or graft, with natural or ar tificial tissues used as implant causing abnormal patient reaction, or later com plication, without mention of misadventure at time of operation
E879.4 Aspiration of fluid as the cause of abnormal reaction of patient, or of later complication, without mention of misadventure at time of procedure

We identified all inpatients with 1 or more of these selected ICD-9 codes who were admitted from January to September 2000. We then reviewed 141 patient records randomly selected from the 1126 patients to assess the PPV of the ICD-9 codes. Events confirmed in the medical record were classified according to the level of harm, the cause of hospitalization, and the duration the device had been in place.

Postdischarge Patient Survey. In addition to the patient-record–based surveillance methods, we wanted to obtain a patient-centered perspective on device-related problems. From November 2000 through January 2001, pretested questions about device-related problems were added to our hospital's standard hospital postdischarge telephone satisfaction survey (available from the authors on request). Patients were randomly selected from the discharge lists from the previous calendar month until a predetermined, floor-specific number was reached.

Clinical Engineering Database. Service reports for electronic and mechanical devices entered in the hospital clinical engineering department's database of service and maintenance activities were reviewed for January through September 2000 and classified according to the type of device and corrective action. This information could not be linked to specific patients or events. This supplemental analysis was intended to provide an independent estimate of the frequency of problems associated with equipment from an engineering perspective.

Statistical and Data Analysis. For computer-flag–based surveillance, the PPV of computer flags was calculated by dividing the number of true-positive flags by the total number of flag alerts. Computer flags were categorized by type of device and whether harm occurred. Binomial confidence intervals (CIs) and P values for statistical comparisons were calculated with Stata 7.0 (College Station, Tex) or StatXact 3 (Cambridge, Mass). The χ2 test was used for count data and Wilcoxon rank sum test for nonnormally distributed continuous data. P<.05 was considered significant.

For AMDEs detected by computer flags, ICD-9 codes, and incident reports, AMDEs were classified by sex, age group, physician service, and primary diagnosis. Rates and confidence intervals of AMDEs per 1000 patient admissions were calculated. Device events detected by computer flags were expressed as rates per 1000 device-days for selected devices.

Pairwise comparisons of the incident report, computer flag, and ICD-9 diagnosis methods for finding device-related hazards and AMDEs were performed sequentially. Because no individual methods constituted a gold standard, sensitivity and specificity were not calculated.

Description of the Study Population

From January through September 2000, there were 19 704 regular admissions, representing 94 187 patient-days, and 7861 short-stay patient admissions. A total of 7124 patients were admitted to the obstetric service or were newborn; these patients were not included in the study because rates of detected medical device events in these populations were extremely low. Therefore, the study population comprised a total of 20 441 regular and short-stay patients (Table 1). The total number of device-days was 20 948 for urinary catheters, 3530 for peripherally inserted central catheters (PICCs), 3937 for central venous catheters, 8197 for arterial catheters, and 5687 for ventilators.

Table Graphic Jump LocationTable 1. Characteristics of the Hospital Population, Excluding Obstetric and Newborn Patients
Online Incident Event Reporting

The online incident reporting system yielded 80 device-related events, of which 21 were device-related hazards, 20 were AMDEs, and 39 could not be classified. Twenty-three involved disposable devices (2 hazard, 11 AMDEs, 10 unclassified) and 46 involved durable equipment (14 hazards, 7 AMDEs, 25 unclassified). Examples of device problems listed in the incident event reporting system included a sheared Jackson-Pratt drain, clotted central venous catheter, and malfunction of a patient-controlled analgesia pump.

Computer-Flag–Based Surveillance

Of the 7059 flags triggered during the study period, 552 (7.8%) indicated a device-related hazard or AMDE (Table 2). Patient medical record review of positive flags verified 95% of the flags that were initially classified as true positive according to the computer record alone.

Table Graphic Jump LocationTable 2. Positive Predictive Value of Individual Computer Flags

A total of 3687 (52%) flags were generated for changes in blood pressure, oxygen saturation, heart rate, or respiratory rate. The PPV for these hemodynamic flags was 3.4% (95% CI, 2.9%-4.1%). In most instances, the aberrant measurements were either clearly attributable to artifact or accurately reflected a deterioration in physiologic status that was not caused by a device-related complication. True-positive signals represented abnormal values attributed by the nurse reviewers to device failure; none were associated with patient harm. Change in urine output had a PPV of 0%.

Line insertion and removal flags had a PPV of 8.6% (95% CI, 7.1%-10.2%); 68% of the true-positive events were classified as AMDEs. The most common AMDEs were phlebitis or evidence of ischemia associated with arterial lines; typical device-related hazards were clotting and leaking.

Flags for detection of urinary tract–related and intravascular bloodstream infection had a PPV of 38% (95% CI, 34%-42%) and 35% (95% CI, 29%-42%), respectively; all true positives were classified as AMDEs. Radiology reports detected a small number of device-related hazards, usually tube malposition, with a PPV of 0.7% (95% CI, 0.2%-1.4%). Extended surgery duration had a PPV of 0% (95% CI, 0%-5.4%). The overall PPV of computer flags was 7.8% (95% CI, 7.2%-8.5%).

Table 3 shows the most common types of devices associated with AMDEs and device-related hazards. Foley catheters (57% of total AMDEs) and arterial catheters (17%) were the devices most commonly associated with AMDEs, whereas cardiac/blood pressure monitors and pulse oximeters were the devices most commonly associated with hazard (35% and 30%, respectively). Rates of AMDEs per 1000 device-days, as detected by computer flags, were 10.4 (95% CI, 9.1-11.8) for urinary catheters, 7.9 (95% CI, 5.3-11.4) for PICCs, 16.0 (95% CI, 12.5-20.7) for central venous catheters, and 9.9 (95% CI, 7.8-12.2) for arterial catheters.

Table Graphic Jump LocationTable 3. Computer-Flag−Based Surveillance: Number of Device Events by Device Type
Telemetry Checklist

During 516 hours of monitoring, the voluntary checklist system revealed a total of 593 false alarms and 4 missed ectopic events. The distribution of false alarms was 223 episodes of ventricular tachycardia, 131 of asystole, 104 of bradycardia, 83 of sinus pause, 20 of premature ventricular contraction, and 32 other. None of these events were considered AMDEs (ie, none of the false alarms or missed arrhythmias resulted in overt patient harm). Device events detected by this surveillance method did not overlap with events uncovered by other surveillance methods.

-Code–Based Surveillance

During the study period from January through September 2000, 1122 (5.5%) admissions had 1 or more of the target ICD-9 codes (Table 4). The device categories most often associated with ICD-9 events were orthopedic (25% of the total), vascular (19%), dialysis (15%), and urologic catheters (9%). Overall, 27% of patients identified by this surveillance method were short stay, for whom it can be reliably assumed that the device problem was the reason for admission. Most of the short-stay problems were related to dialysis (47%) and orthopedic devices (33%). Events uncovered by querying ICD-9 codes represented the primary discharge diagnosis in the majority (51%; 568/1122) of instances, more so in short-stay than regular-stay patients (93.5% [288/308] vs 34% [280/814]; P<.001). Analyzing the events by type showed that 52% involved permanent implants, 9% involved disposable devices, none involved durable equipment, and the remainder (38.5%) had codes that did not allow inference of device type. None of the diagnostic codes for short-stay admissions involved disposable devices.

Table Graphic Jump LocationTable 4. Adverse Medical Device Events (AMDEs) Detected by ICD-9 Codes During January-September 2000*

The record review of the random sample of 141 hospitalizations with 1 or more of the study ICD-9 codes revealed that 101 (72%) had a confirmed AMDE. Most instances of failure to confirm an AMDE were explained by the occurrence of a procedure-associated adverse event that was not device related. All of the patients experienced some harm, confirming the supposition that only events resulting in patient harm were documented among the ICD-9 discharge diagnoses. Ninety-one of 101 (90%) records of patients with a 996 code had a confirmed AMDE. Of 40 patients with an E878.1 or E878.2 or E879.4 code but not a 996 code, 10 (25%) had a confirmed AMDE. Excluding patients with code 996.64 (urologic catheters), the most common type of device injury was device failure requiring device replacement or repositioning (48%). For 84% of these patients, the device failure (generally an implant) was the cause of hospitalization. Fourteen of the records reviewed were of short-stay patients, and in all instances the AMDE was the cause of hospitalization.

Postdischarge Patient Survey

Of 888 patients who participated in the postdischarge survey, 72 (8%) responded affirmatively to the question, "Did you have any problems with any of the medical equipment used in your care or treatment?" For 58 of the 888 (7%) patients, we could infer from their narrative that a medical device was actually involved (some patients provided unclear or no narrative or cited nonmedical items such as televisions or windows); for 38 of the 58 (66%) individuals, the problem with the medical equipment bothered the respondent "somewhat" or "very much." For 29 of the 58 (50%) individuals, the device was an intravenous catheter, needle, or pump. Other types of devices reported included epidural catheter or pump (5 patients), blood pressure machines or cardiac monitors (6 patients), and beds or call buttons (4 patients). Overall, 16 patients reported device-related harm, resulting in an estimated rate of 18 (95% CI, 9-27) per 1000 patients; 2 instances of harm were related to durable equipment and 12 to disposable devices (the other 2 could not be classified). Twelve patients reported device-related hazards, 8 related to durable equipment, 3 to disposables, and 1 that could not be classified.

Clinical Engineering Database

During the study, a total of 1359 service reports were logged into the database. Devices most frequently sent to engineering for assessment were traction or hydraulic elevators (n = 262; 19%), cardiac monitors (n = 95; 7%), ventilators (n = 85; 6%), and anesthesia units (n = 78; 6%). Disposable devices and permanent implants were not included in the database. In 68 (5%) of instances, no repairs were necessary and in 1060 (78%), minor work was sufficient to restore equipment function.

Comparison of the Surveillance Results

Compared with the total number of AMDEs identified by any method, each method individually detected a minority of AMDEs. The line/insertion removal flag identified the same AMDEs as a vascular device ICD-9 code in 12 patients (16% of the line/insertion removal flags and 22% of vascular device–related ICD-9 codes). The catheter-related urinary tract infection flag identified the same AMDEs as the ICD-9 code for urologic catheters in 34 patients (16% of catheter-related urinary tract infection flags and 33% of urologic catheter-related ICD-9 codes). The catheter-associated bloodstream infection flag identified the same AMDEs as the ICD-9 code for vascular devices in 36 patients (41% of catheter-associated bloodstream infection flags and 22% of vascular device–related ICD-9 codes). None of the AMDEs detected by the Patient Satisfaction Survey were detected by the incident reporting, ICD-9, or computer-flag–based methods.

AMDE Rates by Patient Characteristics

Table 5 depicts the characteristics of regular-stay patients who experienced AMDEs, as detected by computer flags, ICD-9 codes, or incident reports. The AMDE detection methods yielded rates, expressed per 1000 regular-stay patients for the period, that were statistically significantly different from each other. The detected rate was 1.6 per 1000 patients (95% CI, 0.9-2.5) for incident reports, 27.7 per 1000 patients for computer flags (95% CI, 24.9-30.7), and 64.6 per 1000 for ICD-9 codes (95% CI, 60.4-69.1). The overall incidence of AMDE detected by any of these methods was 83.7 per 1000 (95% CI, 78.8-88.6). One AMDE, a catheter-related bloodstream infection, was directly associated with death during hospitalization. Longer-term follow-up was not performed.

Table Graphic Jump LocationTable 5. Characteristics of Regular-Stay Patients Who Experienced an Adverse Medical Device Event (AMDE), January-September 2000, by Surveillance Method*

As described above, in contrast to the other surveillance methods, ICD-9-code–based surveillance included AMDEs that were present at admission and those that were hospital acquired. The detected AMDE rates among regular admissions were not statistically different between men and women. Rates of AMDEs rose with increasing age; detected rates of AMDEs were quite low for gynecology. Patients with AMDEs identified by the computer-flag–based system had much longer lengths of stay than average (21.6 days vs 5.6 days; P<.001). Differences in AMDE rates across different types of primary diagnoses presumably reflected, at least in part, variation in device use. For instance, most AMDEs detected in patients admitted for rehabilitation care were Foley-catheter–associated urinary tract infections, and most AMDEs in patients admitted for prosthetic joint surgery involved orthopedic devices.

This study of methods for medical device event surveillance in hospitalized patients established a broad-based taxonomic scheme for identification and classification of medical device events. Explicit case criteria were developed that included many device-related problems and differentiated hazard, a state of increased risk related to device use, from harm, patient injury related to device use. However, neither of these states depended on the specification of error, a term that was difficult to apply in the circumstances encountered.

Active surveillance methods yielded substantially more information than voluntary reporting, as did the experience with adverse drug events.5,35 We had hoped that computer flags would facilitate timely detection of device events to allow secondary prevention and mitigate severity of harm, as they have for adverse drug events. However, aside from rules to detect device-related infections, the PPV of the flags was low. Anecdotal observations during the course of the study also suggested that device events were missed by the computer flag system. For instance, health care workers informed us that cardiac monitors were a source of frustration because of misreading of ectopy and that a problem with patient-controlled analgesia pumps was the lack of an obstruction alarm when the catheter segment above the pump was occluded. Electronic data were not available to generate flags according to absence or presence of alarms and, even when alarm problems occurred, documentation of such events in the medical record was minimal to nonexistent, making it more difficult to evaluate triggers to determine whether an adverse event had occurred. It appeared that the typical health care worker response to a device problem was to fix it or to retrieve a new device that worked and then move on, an appropriate solution at the individual patient level but not an effective systems approach.

ICD-9 codes represent an attractive method for adverse event surveillance because of the potential for consistent application across different health care systems and institutions. By reviewing a sample of records, we confirmed the predictive utility of 996 codes, whereas E codes were of less value by themselves. The disadvantages of ICD-9-code–based surveillance were that only episodes of overt harm were uncovered, classification of events by device type was only possible into broad device categories, and ameliorative intervention for the specific patient was impossible. ICD-9 codes appeared to be particularly useful for device-related events that caused hospitalization and less so for AMDEs that occurred during the hospitalization. Also, duplicate coding of the same event needs to be identified when events from administrative databases are tracked.

The postdischarge survey provided a novel, alternative source of data. The patients' attention focused on simple devices in common daily use that were a cause of discomfort. This information may be a useful element to include when patient satisfaction is assessed; whether it has a substantial influence on health care–related quality of life requires additional study. Patient interview has been a useful source of information for detection of adverse drug events in outpatients.36

The intensive, real-time telemetry detection system confirmed a high incidence of false alarms.37 The clinical engineering database indicated that telemetry monitors were the second most common device serviced.

Each surveillance method was useful for detecting different types of problems, but they exhibited relatively little overlap. Taken together, the complementary detection methods examined in this study indicated that device problems were frequent. However, none of the techniques described here were adequate to serve as a gold standard. Although our approach to the complex subject of device-related patient safety was more comprehensive than others previously reported in the literature, we were not able to derive a definitive measure of the incidence of device-related problems, nor would such an estimate necessarily be generalizable to other institutions. Just as medication errors and adverse drug events required extensive investigation to estimate incidence and develop interventions to prevent events, so also is intensive investigation in multiple centers required to define the epidemiology and establish surveillance and control of medical device problems.

Leape LL, Brennan TA, Laird N, Lawthers AG, Localio R, Barnes BA.  et al.  The nature of adverse events in hospitalized patients: results of the Harvard Medical Practice Study II.  N Engl J Med.1991;324:377-384.
PubMed
Brennan TA, Localio AR, Leape LL, Laird NM, Peterson L, Hiatt HH.  et al.  Identification of adverse events occurring during hospitalization: a cross-sectional study of litigation, quality assurance, and medical records at two teaching hospitals.  Ann Intern Med.1990;112:221-226.
PubMed
Bates DW, O'Neil AC, Petersen LA, Lee TH, Brennan TA. Evaluation of screening criteria for adverse events in medical patients.  Med Care.1995;33:452-462.
PubMed
Bates DW, Cullen DJ, Laird N, Petersen LA, Small SD, Servi D.  et al.  Incidence of adverse drug events and potential adverse drug events: implications for prevention: ADE Prevention Study Group.  JAMA.1995;274:29-34.
PubMed
Classen DC, Pestotnik SL, Evans RS, Burke JP. Computerized surveillance of adverse drug events in hospital patients.  JAMA.1991;266:2847-2851.
PubMed
Thomas EJ, Studdert DM, Burstin HR, Orav EJ, Zeena T, Williams EJ.  et al.  Incidence and types of adverse events and negligent care in Utah and Colorado.  Med Care.2000;38:261-271.
PubMed
Cullen DJ, Bates DW, Small SD, Cooper JB, Nemeskal AR, Leape LL. The incident reporting system does not detect adverse drug events: a problem for quality improvement.  Jt Comm J Qual Improv.1995;21:541-548.
PubMed
Brown SL, Morrison AE, Parmentier CM, Woo EK, Vishnuvajjala RL. Infusion pump adverse events: experience from medical device reports.  J Intraven Nurs.1997;20:41-49.
PubMed
Fuller J, Parmentier C. Dental device-associated problems: an analysis of FDA postmarket surveillance data.  J Am Dent Assoc.2001;132:1540-1548.
PubMed
White GG, Weick-Brady MD, Goldman SA.  et al.  Improving patient care by reporting problems with medical devices.  CRNA.1998;9:139-156.
PubMed
Dwyer D. Medical device adverse events and the temporary invasive cardiac pacemaker.  Int J Trauma Nurs.2001;7:70-73.
PubMed
Dillard SF, Hefflin B, Kaczmarek RG, Petsonk EL, Gross TP. Health effects associated with medical glove use.  AORN J.2002;76:88-96.
PubMed
Brown SL, Duggirala HJ, Pennello G. An association of silicone-gel breast implant rupture and fibromyalgia.  Curr Rheumatol Rep.2002;4:293-298.
PubMed
Ward RA. Ensuring patient safety: what lessons can be learned from device-related adverse events in hemodialysis?  Artif Organs.2002;26:305-306.
PubMed
 Medical device problem reporting for the betterment of healthcare.  Health Devices.1998;27:277-292.
PubMed
Austin A. Adverse incident reporting to the medical devices agency: a review of reports involving trilucent breast implants.  Br J Plast Surg.2000;53:313-316.
PubMed
Badea C, Kolitsi Z, Pallikarakis N, Papapanagiotou A. Medical device surveillance world-wide.  Stud Health Technol Inform.1996;28:157-174.
PubMed
Rosenheimer L, Embry FC, Sanford J, Silver SR. Infection surveillance in home care: device-related incidence rates.  Am J Infect Control.1998;26:359-363.
PubMed
Rosenqvist M, Beyer T, Block M, den Dulk K, Minten J, Lindemans F. Adverse events with transvenous implantable cardioverter-defibrillators: a prospective multicenter study: European 7219 Jewel ICD Investigators.  Circulation.1998;98:663-670.
PubMed
Cheney FW, Posner KL, Caplan RA, Gild WM. Burns from warming devices in anesthesia: a closed claims analysis.  Anesthesiology.1994;80:806-810.
PubMed
Domino KB, Posner KL, Caplan RA, Cheney FW. Airway injury during anesthesia: a closed claims analysis.  Anesthesiology.1999;91:1703-1711.
PubMed
Reid MH, Sawyer D. The human factors implications of peritoneal dialysis: cycler overfill incident reports.  Int J Trauma Nurs.1999;5:68-71.
PubMed
Brown SL, Bogner MS, Parmentier CM, Taylor JB. Human error and patient-controlled analgesia pumps.  J Intraven Nurs.1997;20:311-316.
PubMed
Lin L, Vicente KJ, Doyle DJ. Patient safety, potential adverse drug events, and medical device design: a human factors engineering approach.  J Biomed Inform.2001;34:274-284.
PubMed
Gosbee J. Human factors engineering and patient safety.  Qual Saf Health Care.2002;11:352-354.
PubMed
Gosbee J, Anderson T. Human factors engineering design demonstrations can enlighten your RCA team.  Qual Saf Health Care.2003;12:119-121.
PubMed
Chambrin MC. Alarms in the intensive care unit: how can the number of false alarms be reduced?  Crit Care.2001;5:184-188.
PubMed
Amoore J, Ingram P. Quality improvement report: learning from adverse incidents involving medical devices.  BMJ.2002;325:272-275.
PubMed
Jha AK, Kuperman GJ, Teich JM, Leape L, Shea B, Rittenberg E.  et al.  Identifying adverse drug events: development of a computer-based monitor and comparison with chart review and stimulated voluntary report.  J Am Med Inform Assoc.1998;5:305-314.
PubMed
Evans RS, Pestotnik SL, Classen DC, Bass SB, Menlove RL, Gardner RM.  et al.  Development of a computerized adverse drug event monitor.  Proc Annu Symp Comput Appl Med Care.1991;:23-27.
Kennelly RJ. Improving acute care through use of medical device data.  Int J Med Inf.1998;48:145-149.
PubMed
Kennelly RJ, Gardner RM. Perspectives on development of IEEE 1073: the Medical Information Bus (MIB) standard.  Int J Clin Monit Comput.1997;14:143-149.
PubMed
Evans RS, Pestotnik SL, Classen DC, Bass SB, Burke JP. Prevention of adverse drug events through computerized surveillance.  Proc Annu Symp Comput Appl Med Care.1992;:437-441.
Garner JS, Jarvis WR, Emori TG, Horan TC, Hughes JM. CDC definitions for nosocomial infections, 1988.  Am J Infect Control.1988;16:128-140.
PubMed
Hart GK, Baldwin I, Gutteridge G, Ford J. Adverse incident reporting in intensive care.  Anaesth Intensive Care.1994;22:556-561.
PubMed
Gandhi TK, Burstin HR, Cook EF, Puopolo AL, Haas JS, Brennan TA.  et al.  Drug complications in outpatients.  J Gen Intern Med.2000;15:149-154.
PubMed
Tsien CL, Fackler JC. Poor prognosis for existing monitors in the intensive care unit.  Crit Care Med.1997;25:614-619.
PubMed

Figures

Tables

Table Graphic Jump LocationTable 1. Characteristics of the Hospital Population, Excluding Obstetric and Newborn Patients
Table Graphic Jump LocationTable 2. Positive Predictive Value of Individual Computer Flags
Table Graphic Jump LocationTable 3. Computer-Flag−Based Surveillance: Number of Device Events by Device Type
Table Graphic Jump LocationTable 4. Adverse Medical Device Events (AMDEs) Detected by ICD-9 Codes During January-September 2000*
Table Graphic Jump LocationTable 5. Characteristics of Regular-Stay Patients Who Experienced an Adverse Medical Device Event (AMDE), January-September 2000, by Surveillance Method*

References

Leape LL, Brennan TA, Laird N, Lawthers AG, Localio R, Barnes BA.  et al.  The nature of adverse events in hospitalized patients: results of the Harvard Medical Practice Study II.  N Engl J Med.1991;324:377-384.
PubMed
Brennan TA, Localio AR, Leape LL, Laird NM, Peterson L, Hiatt HH.  et al.  Identification of adverse events occurring during hospitalization: a cross-sectional study of litigation, quality assurance, and medical records at two teaching hospitals.  Ann Intern Med.1990;112:221-226.
PubMed
Bates DW, O'Neil AC, Petersen LA, Lee TH, Brennan TA. Evaluation of screening criteria for adverse events in medical patients.  Med Care.1995;33:452-462.
PubMed
Bates DW, Cullen DJ, Laird N, Petersen LA, Small SD, Servi D.  et al.  Incidence of adverse drug events and potential adverse drug events: implications for prevention: ADE Prevention Study Group.  JAMA.1995;274:29-34.
PubMed
Classen DC, Pestotnik SL, Evans RS, Burke JP. Computerized surveillance of adverse drug events in hospital patients.  JAMA.1991;266:2847-2851.
PubMed
Thomas EJ, Studdert DM, Burstin HR, Orav EJ, Zeena T, Williams EJ.  et al.  Incidence and types of adverse events and negligent care in Utah and Colorado.  Med Care.2000;38:261-271.
PubMed
Cullen DJ, Bates DW, Small SD, Cooper JB, Nemeskal AR, Leape LL. The incident reporting system does not detect adverse drug events: a problem for quality improvement.  Jt Comm J Qual Improv.1995;21:541-548.
PubMed
Brown SL, Morrison AE, Parmentier CM, Woo EK, Vishnuvajjala RL. Infusion pump adverse events: experience from medical device reports.  J Intraven Nurs.1997;20:41-49.
PubMed
Fuller J, Parmentier C. Dental device-associated problems: an analysis of FDA postmarket surveillance data.  J Am Dent Assoc.2001;132:1540-1548.
PubMed
White GG, Weick-Brady MD, Goldman SA.  et al.  Improving patient care by reporting problems with medical devices.  CRNA.1998;9:139-156.
PubMed
Dwyer D. Medical device adverse events and the temporary invasive cardiac pacemaker.  Int J Trauma Nurs.2001;7:70-73.
PubMed
Dillard SF, Hefflin B, Kaczmarek RG, Petsonk EL, Gross TP. Health effects associated with medical glove use.  AORN J.2002;76:88-96.
PubMed
Brown SL, Duggirala HJ, Pennello G. An association of silicone-gel breast implant rupture and fibromyalgia.  Curr Rheumatol Rep.2002;4:293-298.
PubMed
Ward RA. Ensuring patient safety: what lessons can be learned from device-related adverse events in hemodialysis?  Artif Organs.2002;26:305-306.
PubMed
 Medical device problem reporting for the betterment of healthcare.  Health Devices.1998;27:277-292.
PubMed
Austin A. Adverse incident reporting to the medical devices agency: a review of reports involving trilucent breast implants.  Br J Plast Surg.2000;53:313-316.
PubMed
Badea C, Kolitsi Z, Pallikarakis N, Papapanagiotou A. Medical device surveillance world-wide.  Stud Health Technol Inform.1996;28:157-174.
PubMed
Rosenheimer L, Embry FC, Sanford J, Silver SR. Infection surveillance in home care: device-related incidence rates.  Am J Infect Control.1998;26:359-363.
PubMed
Rosenqvist M, Beyer T, Block M, den Dulk K, Minten J, Lindemans F. Adverse events with transvenous implantable cardioverter-defibrillators: a prospective multicenter study: European 7219 Jewel ICD Investigators.  Circulation.1998;98:663-670.
PubMed
Cheney FW, Posner KL, Caplan RA, Gild WM. Burns from warming devices in anesthesia: a closed claims analysis.  Anesthesiology.1994;80:806-810.
PubMed
Domino KB, Posner KL, Caplan RA, Cheney FW. Airway injury during anesthesia: a closed claims analysis.  Anesthesiology.1999;91:1703-1711.
PubMed
Reid MH, Sawyer D. The human factors implications of peritoneal dialysis: cycler overfill incident reports.  Int J Trauma Nurs.1999;5:68-71.
PubMed
Brown SL, Bogner MS, Parmentier CM, Taylor JB. Human error and patient-controlled analgesia pumps.  J Intraven Nurs.1997;20:311-316.
PubMed
Lin L, Vicente KJ, Doyle DJ. Patient safety, potential adverse drug events, and medical device design: a human factors engineering approach.  J Biomed Inform.2001;34:274-284.
PubMed
Gosbee J. Human factors engineering and patient safety.  Qual Saf Health Care.2002;11:352-354.
PubMed
Gosbee J, Anderson T. Human factors engineering design demonstrations can enlighten your RCA team.  Qual Saf Health Care.2003;12:119-121.
PubMed
Chambrin MC. Alarms in the intensive care unit: how can the number of false alarms be reduced?  Crit Care.2001;5:184-188.
PubMed
Amoore J, Ingram P. Quality improvement report: learning from adverse incidents involving medical devices.  BMJ.2002;325:272-275.
PubMed
Jha AK, Kuperman GJ, Teich JM, Leape L, Shea B, Rittenberg E.  et al.  Identifying adverse drug events: development of a computer-based monitor and comparison with chart review and stimulated voluntary report.  J Am Med Inform Assoc.1998;5:305-314.
PubMed
Evans RS, Pestotnik SL, Classen DC, Bass SB, Menlove RL, Gardner RM.  et al.  Development of a computerized adverse drug event monitor.  Proc Annu Symp Comput Appl Med Care.1991;:23-27.
Kennelly RJ. Improving acute care through use of medical device data.  Int J Med Inf.1998;48:145-149.
PubMed
Kennelly RJ, Gardner RM. Perspectives on development of IEEE 1073: the Medical Information Bus (MIB) standard.  Int J Clin Monit Comput.1997;14:143-149.
PubMed
Evans RS, Pestotnik SL, Classen DC, Bass SB, Burke JP. Prevention of adverse drug events through computerized surveillance.  Proc Annu Symp Comput Appl Med Care.1992;:437-441.
Garner JS, Jarvis WR, Emori TG, Horan TC, Hughes JM. CDC definitions for nosocomial infections, 1988.  Am J Infect Control.1988;16:128-140.
PubMed
Hart GK, Baldwin I, Gutteridge G, Ford J. Adverse incident reporting in intensive care.  Anaesth Intensive Care.1994;22:556-561.
PubMed
Gandhi TK, Burstin HR, Cook EF, Puopolo AL, Haas JS, Brennan TA.  et al.  Drug complications in outpatients.  J Gen Intern Med.2000;15:149-154.
PubMed
Tsien CL, Fackler JC. Poor prognosis for existing monitors in the intensive care unit.  Crit Care Med.1997;25:614-619.
PubMed

Letters

CME
Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.

Multimedia

Some tools below are only available to our subscribers or users with an online account.

Web of Science® Times Cited: 52

Related Content

Customize your page view by dragging & repositioning the boxes below.

See Also...
Articles Related By Topic
Related Collections
PubMed Articles