Automated External Defibrillators—Device Reliability and Clinical Benefits

Sudden cardiac arrest accounts for more than 300 000 deaths annually in the United States and is a leading cause of disability and health care costs.^{1 - 4} Successful resuscitation, survival, and optimal clinical outcomes depend on time to definitive treatment, primarily early defibrillation of ventricular tachycardia or fibrillation. A substantial body of evidence demonstrates improved survival resulting from reduced time to defibrillation with automated external defibrillators (AEDs).^{1 - 16} Increasing recognition of the clinical benefits of AEDs and minimal legal risks of ownership, medical oversight, or use have contributed to an exponential growth in AED use over the last decade.^{1 - 2 ,17} As lifesaving devices, AEDs require the highest possible standards of reliability. Despite careful design, engineering, and manufacturing, AEDs—like all electronic devices—have finite rates of component failures; however, prior analyses of AED reliability are notably limited.

In this issue of JAMA, Shah and Maisel use 2 separate data sources to address AED performance.¹⁸ Using data from weekly US Food and Drug Administration (FDA) Enforcement Reports over a 9-year period, the authors identified recalls and safety alerts, collectively known as advisories, involving AEDs and accessories and estimated the number of AED advisories, rate trends, and types of malfunctions prompting AED and AED accessory advisories. These advisories are markers of AED device performance that indicate the potential for malfunction, not an actual malfunction; as such, they are a surrogate of AED reliability but not a true measurement. Accurate assessment of actual device performance is not possible based on limitations of the postmarketing surveillance system for medical devices. Moreover, some advisories were issued when the risk of AED malfunction was less than 1%.

Based on these data, Shah and Maisel report that during 2.78 million AED device-years of observation, 52 advisories were issued affecting a total of 385 922 AEDs and accessories. This represents an average of 5.1 devices affected by advisories per 100 AED device-years. Overall, 21.2% of AEDs distributed during the study period were recalled, most often due to electrical or software problems.

In their second analysis of reports of AED malfunction, the authors¹⁸ reviewed AED-related adverse events reported to the FDA Manufacturer and User Facility Device Experience (MAUDE) database and classified the limited available information in this database into malfunctions if manufacturer analysis confirmed the malfunction, or the device was not returned to the manufacturer for analysis but the AED malfunction was witnessed and confirmed by trained rescue personnel. Only malfunctions that occurred during sustained ventricular arrhythmias were included. The authors did not have access to the primary data, such as rhythm strips or clinical flow sheets for analysis, and therefore had to rely on reports of observations of rescue personnel on the scene. The authors classified 370 of these reports as “confirmed” AED-related device malfunctions. While failure of a potentially lifesaving device certainly is concerning, accurately assessing the importance of this finding is difficult due to incomplete information regarding the denominator of actual AED deployments during cardiac arrests.

This study by Shah and Maisel¹⁸ should be carefully considered in the context of benefits and risks of AEDs. The analysis of advisories represents potential, not actual, AED malfunction. The AED advisory rate did not increase significantly during the period of analysis. With the 10-fold increase in numbers of AEDs sold during the study period,¹⁸ an increase in annual absolute number of AED advisories would be expected. The 370 reports of AED malfunction in the MAUDE database appear relatively infrequent when compared with the 385 922 AEDs affected by advisories. However, underreporting of medical device malfunction via this database is common,¹⁹ and reports of malfunctions are infrequent compared with the hundreds of thousands of uses of AEDs during attempted resuscitation. As noted by the authors, the study reassuringly demonstrates that despite increasing AED complexity, the advisory rate did not increase during the study period, even though the analysis clearly justifies measures to further improve AED reliability.

In multiple clinical studies, reports of failures of AEDs are notably absent.^{1 ,5 - 15} For example, in the Public Access to Defibrillation (PAD) Trial, which used devices from 3 different manufacturers, no device failures or clinical harm from AEDs were reported from the deployment of 1600 AEDs accessible to more than 11 000 volunteers in 622 locations during an average period of 21.5 months.¹ With detailed and rigorous analysis of the clinical data related to each AED deployment by a clinical events committee in the PAD Trial, adverse events related to AED use were rare and consisted mostly of transient psychological trauma to the volunteers and stolen devices.¹ Despite the low frequency of AED malfunctions resulting in adverse clinical outcomes, these studies were underpowered to detect performance problems.^{1 ,5 - 15} Signals of uncommon adverse events are rarely noted in such investigations. Only with an analysis of a considerably larger number of AEDs for a longer time, with more than 2.7 million device-years of observation in the study by Shah and Maisel, could the very low frequency of AED clinically significant malfunctions be noted.¹⁸

This analysis of AED reliability,¹⁸ like other analyses regarding pacemaker and implantable cardioverter defibrillators,^{20 - 21} focuses attention on multiple salient issues regarding monitoring and reporting of device performance. Device reliability standards, regulatory oversight, information dissemination, and responsibilities of all stakeholders merit careful evaluation to further improve AED reliability. The report by Shah and Maisel serves a major purpose of highlighting the inadequacies of manufacturer and FDA postmarketing device tracking, performance assessment, and notification mechanisms. Preclinical testing and clinical trials are not designed to identify device reliability with relatively small numbers of devices followed for short periods of time. Postmarketing surveillance provides the mechanism for monitoring performance of a large number of devices in long-term follow-up and should serve as the basis for determination of the mechanism, time course, and rate of product failure. Postmarketing surveillance also serves as the basis for redesign of components, modifications of manufacturing, and identification of devices at risk of failure. This same process is used by regulatory agencies or manufacturers to make decisions to issue advisories regarding reliability problems.

Depending on the nature and scope of the problem, the FDA may choose to issue a recall or a safety alert, collectively considered advisories. Recalls are issued when a device with the potential for risk is identified and are classified as I, II, or III with decreasing probability of serious adverse health consequences. A class I recall indicates a reasonable probability that the product will cause serious adverse health consequences. In the analysis by Shah and Maisel, approximately 10% of AED and accessory advisories were class I advisories. A class II recall indicates that the product may cause temporary or medically reversible adverse health consequences or that the probability of adverse health consequences is remote. In this report, 83.7% of the AED and 90.3% of the accessory advisories were class II. A safety alert is a communication issued by a manufacturer to inform physicians of the potential for substantial harm from a medical device. Other communications may be sent from manufacturers to notify physicians of performance issues that fall below the threshold for recalls or safety alerts.

Recent recommendations call for establishing a simpler, clearer, and more intuitive nomenclature to communicate information about device performance.²² These recommendations include elimination of the term “recall” and use of the terms “advisories” and “safety alerts” to provide more clarity in communication. While AED manufacturers must report to the FDA all events brought to their attention, manufacturers are not required to seek out device malfunctions actively. The limitations of this passive adverse event reporting system are compounded by lack of complete information regarding owners, medical directors, users, or service status of AEDs. The confluence of these multiple factors makes it currently impossible to precisely determine the nature and frequency of AED malfunctions.

Similar, but not identical, challenges regarding monitoring of device reliability have arisen with pacemakers and implantable cardioverter defibrillators.^{20 - 21} The Heart Rhythm Society, representing physicians who implant cardiac devices, established a task force to examine assessments of device performance and has proposed recommendations and guidelines.²² Many of their recommendations are relevant to AED reliability,^{22 - 23} including greater transparency in postmarketing surveillance, analysis, and reporting of information; enhanced systems to increase the return of devices to manufacturers and to improve the analysis and reporting of device performance and malfunction information; and cooperation among industry, the FDA, and physicians to make every effort to prevent injuries and deaths due to device malfunctions.^{22 - 23} The complex issues related to surveillance, performance standards, threshold for advisories, terminology for advisories, reporting, analysis, communication, and responsibilities of stakeholders^{22 - 23} merit careful consideration, as many of them are relevant for improving performance of AEDs.

Early intervention in cardiac arrest with AEDs improves survival rates, with effectiveness diminishing with each passing minute. Well-designed clinical studies^{1 - 14} provide conclusive scientific evidence of the substantial benefits and minimal risks of AEDs. The rational response to the analysis reported by Shah and Maisel is to enhance efforts to improve AED reliability by reforming the process of monitoring and reporting on AEDs in postmarketing surveillance.

The same standards of evidence-based medicine used to define the benefits of AEDs should be the foundation of postmarketing surveillance for AEDs. These devices, like all electronic medical devices, will always fall short of perfection, but there is a shared responsibility to make AEDs as reliable as possible. No manufactured product can ever be completely free of design or manufacturing flaws.²³ However, because an AED malfunction might result in failure to deliver lifesaving therapy during a cardiac arrest, surveillance and corrective strategies should aim to achieve a risk of malfunction that is as low as possible.²³

While efforts to develop rigorous mechanisms to accurately monitor device performance will undoubtedly have costs, these efforts are essential to achieve the best possible clinical outcomes. Device manufacturers and the federal government should allocate the funds required to make this effort successful. Initial steps should be guided by knowledge that many advisory AEDs remain in service and have not been appropriately evaluated. Additional measures to accurately track those who purchase and use these devices are needed. Subsequent measures will require the best efforts of health care professionals, regulators, manufacturers, and the public working collaboratively to develop a transparent, rigorous, and systematic approach using the best practices of evidence-based medicine in postmarketing surveillance. From these efforts, guidelines for monitoring, communicating, and performance should be established. During cardiac arrest, when each minute really matters for survival, all stakeholders share the duty of ensuring the highest possible standards of AED reliability.