Advancing Cardiac Resuscitation: Title and subTitle BreakLessons From Externally Controlled Trials

At least 250,000 people experience sudden cardiac death outside of hospitals each year in the United States.¹ Despite application of techniques ranging from basic cardiopulmonary resuscitation (CPR), through "first responder" defibrillation using automatic external defibrillators, to advanced cardiac life support (including airways and drugs) by emergency medical service (EMS) personnel, survival rates for out-of-hospital cardiac arrest are as low as 1.4%,² although rates vary widely.³ Even more troublesome than the low survival rates is the difficulty in maximizing the functional ability of those who survive. Clearly, any process that is as widely applicable but has such poor success as resuscitation needs continuous reevaluation and evidence-based strategies for improvement.

In this issue of THE JOURNAL, 2 groups of investigators present their attempts to improve survival for out-of-hospital cardiac arrest. The study by Stiell et al,⁴ conducted in 19 Ontario communities representing 2.7 million people, was designed to decrease the interval from the receipt of an emergency call by an ambulance dispatch center until the arrival of a responder with a defibrillator at the scene of the patient in cardiac arrest. Each community reduced the response interval using its own strategies, such as improving dispatch techniques, deploying mobile ambulances, and equipping firefighters with automatic external defibrillators. For all people in cardiac arrest, regardless of the initial cardiac rhythm, the proportion who had an EMS response within 8 minutes increased from 76.7% for the 4690 patients during the 36-month preintervention phase to 92.5% for the 1641 patients during the 12 months after the intervention. The corresponding rate of survival to hospital discharge improved from 3.9% to 5.2% (P<.05). Following establishment of the rapid defibrillation program, 79.7% of the 66 patients who were resuscitated and alive at 1 year had "very good" cerebral performance scores and functional status.

The study by Cobb et al⁵ was conducted in the Seattle, Wash, EMS system and was designed to improve survival for out-of-hospital ventricular fibrillation. During the 42-month preintervention period, emergency medical technicians administered defibrillation as soon as possible after arriving at the scene of the patient in cardiac arrest. During the 36-month intervention period, Cobb et al modified the standing orders so that emergency medical technicians performed CPR for 90 seconds before performing defibrillation. For patients in ventricular fibrillation, survival to hospital discharge improved from 24% for the 639 patients during the preintervention period to 30% for the 478 patients during the intervention period (P=.04). The proportion of survivors who had good neurologic function at hospital discharge increased from 71% of 150 to 79% of 138 in the preintervention and intervention periods, respectively (P<.11).

Both of these studies are externally controlled trials, also known as before-after trials, studies with historical controls,⁶ or observational studies.⁷ An advantage of this design is that the same sites and populations are used in both the control and intervention periods.⁶ This approach avoids social, cultural, or genetic differences that may be introduced by comparing different populations at different sites,⁷ and also avoids bias in selecting patients for one treatment rather than the other.^{7 - 8}

Another advantage to an externally controlled trial is that it can be used when a randomized controlled trial is not feasible⁸ or not ethical.^{6 ,8 - 9} The collection or study of existing data is exempt from regulation by the US Department of Health and Human Services, provided those data are available publicly or the subjects cannot be identified.¹⁰ Stiell et al indicate that their study was approved by a research ethics committee, but informed consent was deemed unnecessary because patients were not randomized to receive different therapies. Cobb et al did not request approval from an institutional review board for changing the standing orders of the Seattle EMS system. Yet, both investigations should be considered research because they were designed to test a hypothesis, permit conclusions to be drawn, and develop or contribute to generalizable knowledge.¹¹

These 2 studies have several strengths. They address an issue, sudden cardiac death, that has great importance for patients and society. Each group of investigators gathered data consistently and systematically during the preintervention and intervention periods by using the EMS report forms.⁷ The researchers examined not only return of spontaneous circulation and survival to hospital admission and discharge, but also neurologic function among survivors. Stiell et al also estimate the financial aspects of their intervention and analyze costs per life saved.

Bailar et al⁶ suggest 5 interrelated features that can be used to evaluate the strength of evidence and assess the inferences provided by an externally controlled clinical trial. These features can be applied to the studies by Stiell et al and Cobb et al.

First, before the study, did the investigator express that the intervention was applied to affect the outcomes reported? Statistical tests may falsely identify chance occurrences as being significant, especially if these occurrences are noticed or sought after a change in procedure. This criterion supports that the improved outcome observed was not a chance observation, but a direct result of the intervention. Both interventions had the objective of improving survival for cardiac arrest, and both studies describe survival to hospital discharge as the primary outcome and also assess neurologic function among the survivors.

Second, was the analysis planned before the data were generated? This supports the investigators' claim that they first defined their hypothesis, then designed the intervention to test it. Stiell et al published their protocol,¹² whereas Cobb et al refer to their study as "prospectively defined" with an a priori hypothesis about subgroups.

Third, did the investigators articulate a plausible hypothesis before the results were observed? If changes in outcomes are noted retrospectively, the temptation is then to develop a hypothesis that supports the observation. Prior to this study, the Ontario group performed a meta-analysis¹³ and cost-effectiveness analysis,¹⁴ the results of which drove the hypothesis for the intervention. The investigation tested by Cobb et al was prompted by lack of overall improvement in survival for cardiac arrest and was based on experimental work published 2 years¹⁵ before the change in their protocol.

Fourth, would the results have been of interest if they had an "opposite" effect? To achieve therapeutic equipoise, there must be a real possibility that the hypothesis being tested might be incorrect—that the intervention could have an opposite effect. Rapid defibrillation has been a central tenet of resuscitation.¹⁶ If more rapid defibrillation did not improve outcomes, Ontario (and other municipalities) would not invest resources in an ineffective system. If performing CPR before defibrillation did not improve survival, Seattle would revert to rapid defibrillation without initial CPR by the rescuer.

Fifth, do the investigators provide reasonable justification for generalizing their results to broader groups of patients? Results from externally controlled trials may not apply beyond the population or setting tested. Stiell et al support applying their results to North American and European communities with populations of less than 1 million. Cobb et al suggest that their results apply to patients with out-of-hospital ventricular fibrillation for whom defribillation had been delayed.

Although the studies by Stiell et al and Cobb et al meet most of the features used to assess the strength of evidence derived from externally controlled trials, even the best of such trials raise serious concerns. Ideally, in clinical studies, confounding variables are balanced between treatment groups. In randomized controlled trials, this balance usually is achieved through unbiased, random assignment. In externally controlled trials, researchers adjust for imbalances statistically.⁸ Stiell et al and Cobb et al examined the patients' demographic characteristics, the circumstances of the cardiac arrest, and details of the EMS response. The investigators then adjusted for the differences in these factors between groups using a regression analysis. However, neither study examined or adjusted for differences in the victims' severity of underlying cardiac disease, concurrent medical problems, or behavioral risk factors.^{7 - 8}

Accordingly, it is possible that the patients, the disease, or the treatment may have changed over time.⁹ In fact, there are some hints this occurred. For instance, in the study by Stiell et al, the accrual rate increased from 130 patients per month in the preintervention phase to 137 patients per month following implementation of the intervention; in the study by Cobb et al, the accrual decreased from 15 patients per month during the preintervention period to 13 patients per month following the intervention. In addition, a pharmacologic trial was being conducted concurrently with the postintervention phase of the study by Cobb et al. The study drug, amiodarone, may have had subtle yet nonsignificant benefits or study personnel may have been more aggressive in their efforts during the pharmacologic trial.

Thus, even though the studies by Stiell et al and Cobb et al have many strengths, they also have the primary limitation inherent to their design—possible uncontrolled confounders. For this reason, their findings may not be borne out by subsequent trials. Yet, these 2 studies fit those situations for which externally controlled trials are considered important⁶ : when results are not intended to be definitive (Cobb et al consider their work "an encouraging pilot study"); for potentially lethal disease (certainly the case for cardiac arrest); for innovation in desperate circumstances (such as the 2.5% survival rate for out-of-hospital cardiac arrest in Ontario); and for dramatic results (such as the 33% relative increase in survival for all cardiac arrests achieved in the study by Stiell et al, and the 25% relative increase in survival for ventricular fibrillation achieved in the study by Cobb et al). Moreover, Bailar et al suggest that " . . . the first clues to real breakthroughs in medical science may come from comparisons with external controls"⁶ and that " . . . any research program or journal that will take no risks at all with externally controlled (or even uncontrolled) studies of potentially important advances runs some risk of becoming irrelevant to the most important current developments in science and medicine."⁶

Resuscitation research has proceeded slowly and deliberately in the nearly 40 years since CPR was described.¹⁷ However, despite numerous investigations involving various therapies during that time, CPR and defibrillation are the only interventions that have been convincingly demonstrated to improve long-term survival for cardiac arrest.¹⁸ We hope that the studies by Stiell et al and Cobb et al will help generate new hypotheses, stimulate more studies using stronger methods, advance resuscitation research, and ultimately improve neurologically intact survival for those who experience cardiac arrest.