Medical Device–Associated Safety and Risk: Title and subTitle BreakSurveillance and Stratagems

Devices are ubiquitous in the delivery of modern health care. Diagnostics and therapeutics rely on a bewildering, constantly changing array of devices used to monitor and treat patients. Trends in device evolution include increasing complexity and autonomy of operation, device-device interactions, miniaturization, and integration with information technology. Managing risk associated with devices has thus long been a central concern of policy makers, manufacturers, and those providing health care.

The emerging international patient safety movement has called attention to the unacceptably high prevalence of preventable patient injuries as a consequence of medical management.^{1 - 6} The accompanying translation of safety science from other risky industries has brought new perspectives to how all stakeholders think about health care.^{1 ,7 - 8} The view of a clinical service supported by an administrative structure has broadened to include other elements that shape health outcomes. Newer ecological models incorporate the skills, knowledge, experience, attitudes, and values of people (clinicians, health care managers, leaders, and patients), as well as the characteristics of tools, environmental factors, tasks, goals, and their interrelationships.^{9 - 10} Additional dynamic factors significantly shape the experience of providing and receiving care. These factors include time pressure, change, availability and ambiguity of data, team interactions, organizational culture, and the way real or perceived risks mold what individuals actually do.^{11 - 12} Under the sum of such influences, the health care setting presents a field for activity with opportunities or constraints that shape behavior at all levels of the system. In the context of medical devices, the rationale underlying tool design, procurement decisions, and the ways and contexts in which tools are used as opposed to how they were intended to be used all result in unforeseen and unintended consequences under conditions of actual work. A growing dependence on devices, their complexity, and their influence on human and task performance therefore place device use at the heart of the patient safety question.

Understanding about the epidemiology and roles of devices in medical harms and hazards has lagged behind advances in other areas of safety concerns such as medications. Perhaps the reason lies partly in device heterogeneity and insinuation into practically all work processes. An additional challenge lies in the nature of safety reporting in health care in general,^{1 ,13 - 14} and in the case of devices in particular. The article by Samore and colleagues¹⁵ in this issue of THE JOURNAL offers a welcome addition to the dialogue on device and patient safety. The philosophical framework outlined above provides a backdrop for a consideration of the study, informing the tension between "top-down" and "bottom-up" approaches to understanding safety, and thinking about the relative merits of surveillance vs other means of learning and accountability.

What is a medical device? Samore and colleagues do not define the term or a particular focus of interest in their innovative investigation of surveillance methods designed to capture underreported device-related hazards and harms. The US Food and Drug Administration (FDA) categorizes a product as a medical device if the product does not meet the definitions of a drug and is essentially any artifact, article, or tool intended to affect the structure or function of humans or animals.¹⁶ The definition may well serve the purpose of regulation and offer some justification for the lack of constraint in the surveillance study. Device can also be defined as a stratagem designed to achieve a particular effect, as in "being left to one's own devices." Stratagems and work-arounds are often necessary given lack of training to use complex technology and well-intentioned device design that nevertheless results in devices that do not handle the trade-offs of actual work efficiently. Such difficulties increase because of the current nursing shortage, house staff working conditions, and device proliferation. The issue looms large given the primary objective of Samore et al¹⁵ —to measure the positive predictive value of electronic rules and to estimate the incidence of problems associated with devices identified by computer-based surveillance compared with other methods.

The underlying objective of the research likely was to learn about the scope of device hazards to better manage risk. The authors used the resources most easily available to them, trying to apply computer surveillance techniques successfully applied to medication error detection to the more diverse needs of device error detection. More can be learned from the results about the notion of various forms of surveillance and their current limits than about safety, risk, and human work with tools or the incidence of device use hazard and harm. That in itself may be a useful objective, but not surprisingly the signal-to-noise ratio of the data was low.

Samore et al studied all inpatient records except those of obstetric and newborn patients during 9 months at a single large tertiary-care institution with an advanced integrated electronic medical record. Incidents of device-associated hazards and harms were culled from nurse-reviewed computer-based flags, reported twice daily, and compared with a number of other concurrent methods, including a telemetry checklist, retrospective International Classification of Diseases, Ninth Revision (ICD-9) codes specifying devices in their definitions, the clinical engineering database, and a postdischarge patient survey. The institution also had an online incident reporting system, but the organization's safety and reporting culture, demographics of reporters, and structure and functionality of the reporting tool were not described. Nursing surveys and interviews and summaries of investigations of individual device-associated events were apparently performed but data were not reported. Limitations included the lack of direct observations and the lack of categorization by preventability or severity—arguably the 2 most important data points desired in patient safety investigations.

Interpretation of the extensive study results raises a host of interesting questions. Seven computer-flag categories were established, some of unclear value, for example, tracking bloody sputum as a means of identifying problems with intubation. Surveillance of fiberoptic or special airway cart use, resuscitation logs, tracheostomy kit use, otolaryngology consults, and other sources would provide a lower cost-benefit ratio (higher cost, much higher benefit), at least for case identification. While infectious complications involving the urinary tract and intravenous catheters led to 306 of the 383 identified adverse medical device events, the added value of construing catheter-related bloodstream infections as a device surveillance issue independent of nosocomial infection monitoring is not clear, and the data do not appear to provide useful new knowledge. Task failure must be defined first, and the process understood to the point of available knowledge, desire, and resource allocation. To achieve better results, the level of expertise must be raised to fit the task requirements, or the task must be redesigned to reduce demands on skill or tools.

The taxonomy may be too broad to provide useful information. The loosening of a hip prosthesis might be due to a poor decision to place it in a morbidly obese patient, the choice of the patient to engage in ill-advised activity postoperatively, the design of the device, some aspect of the patient's underlying pathological process, or the end of a long and expected period of benefit. A classification system that considered usability issues as well as potential for harm, injury, and costs would have been preferred.

What of the comparison between the lowest detected event rate (incident reports, 1.6 per 1000 admissions) and that for computer flags (27.7 per 1000 admissions)? While computer flags detected many events, the events are of unclear utility. For example, clinical practice for placement of nasogastric tubes may include calibrating placement by radiography. Malposition does not necessarily imply an error. Surveillance might be useful if undertaken as part of task redesign. Pulse oximeters were responsible for a third of all flagged device-related hazards (50/169) but are commonly affected by limb hypothermia, hypoperfusion, movement artifact, nail coloring, ambient lighting, and malpositioning, and clinicians adjust their behaviors accordingly.

During the more than 516 hours of telemetry monitoring, the telemetry log documented 4 ectopic events not captured by the technology vs 593 false alarms. The ectopic events missed by devices but captured by people are most interesting in terms of adding to the knowledge of reinforcing patient safety and managing risk, and they achieve greater significance given the large and likely underestimated number of false alarms. Failure analysis requires context, preventability, and severity; without those aspects it is difficult to assign meaning to this data or how learning should proceed after this surveillance.

The authors also used ICD-9 codes for surveillance of device-associated events. A total of 5.5% of admissions had 1 or more of the target codes. Random record review of 12% of this cohort revealed that nearly a third did not have an adverse medical device event but rather some sort of procedure-associated problem that was undefined. Short-stay patients represented 27% of the cohort; of these, the ICD-9 code was the primary diagnosis in 93%. A total of 80% of the short stays were related to dialysis and orthopedic devices. Device heterogeneity was more apparent in the regular admissions category. Fully a third of all devices were unspecified by type. How reimbursement and coding practices shaped these data is unknown. Using ICD-9 codes is relatively inexpensive, can serve as a starting point for other investigations, and is likely to be generalizable across institutions but, like all administrative databases, has limitations.¹⁷

Moreover, the postdischarge patient survey might have been more useful if patients were targeted who also had positive device-related ICD-9 codes as opposed to a random selection procedure. The available data are not detailed enough to draw valid or specific conclusions about device-associated harms and hazards. Other methods to improve the quality of information from patients would be to focus on patients who brought their own medical devices to the hospital, who received prescriptions for devices for home use, or who had chronic diseases known to involve devices in long-term therapy. Going forward, meaningful device surveillance programs should have a well-thought-out plan for soliciting patient expertise and experience with medical devices given the increasing role of patients in shared decision making and disease management.

Finally, clinical engineering logs likely represent an underutilized, rich resource for capturing and understanding device usability and safety issues. Determining whether equipment actually malfunctioned, or its operation was not understood by the user, would facilitate specific interventions to enhance device usefulness and safety such as evidence-based recommendations to manufacturers and sorely needed systematic device training.

Expertise of skilled clinicians, technicians, patients, and health care managers is an absolute requirement for success in device-use safety efforts – at the beginning, to plan and design effective surveillance, and at the end, to perform focused investigations and provide the context that yields meaning. Although active surveillance was thought to yield substantially more information than voluntary reporting, the value of the information was questionable in many cases, and sorting the wheat from the chaff required significant effort. The availability of better tools for voluntary reporting that are easy to use, capture just enough of event complexity and causal analysis, and facilitate intelligent, rapid database searching will go a long way to combine efficiency of event capture with the desired utility of involving users more deeply in safety improvement.^{18 - 19} Arm's length computer-based surveillance techniques also neglect event recovery data, an important source of learning about adaptation to working conditions in real time that is naturally suited to capture by advanced voluntary reporting tools. Advanced reporting tools combined with quality safety education, with emphasis on case-based training about device-associated issues, have great potential to enhance the safety of medical devices.

Public resources for device safety issues derived from mandatory and voluntary report analysis are available on the FDA Web site.¹⁶ However, much remains to be accomplished in terms of improving the usefulness and detail of reported information.²⁰ Future developments will likely include integration of lessons learned from the wide range of safety reporting projects currently funded by the Agency for Healthcare Research and Quality.^{21 - 22} ECRI (formerly the Emergency Care Research Institute, available at http://www.ecri.org), an independent not-for-profit health services research agency, has maintained a device-associated event database for more than 30 years but data are fully identified and not available for research and dissemination. That said, ECRI represents a substantial resource for health technology safety and risk improvement, from event investigation to alerts and position papers.

Ultimately, the value of measurement is determined by the insight gained, whether for learning or accountability. Surveillance in the service of sustainable, meaningful long-term improvement in patient safety for medical devices will succeed to the extent that it is deployed in concert with the needs and experiences of clinicians and patients.