Testing Protocols in the Intensive Care Unit: Title and subTitle BreakComplex Trials of Complex Interventions for Complex Patients

In this issue of JAMA, Meade and colleagues¹ and Mercat and colleagues² report the results of 2 large international trials of alternative strategies for setting positive end-expiratory pressure (PEEP) in ventilated patients with acute lung injury or acute respiratory distress syndrome. Both trials asked whether higher PEEP would reduce mortality, and both concluded it did not. Many readers not familiar with intensive care might reasonably wonder why such a seemingly innocuous intervention would deserve such attention, but the story behind PEEP is a long one, and these latest, largest trials do not provide a conclusion. They do, however, serve to demonstrate that answering even the simplest questions can become an endeavor of immense proportions.

Mechanical ventilation is lifesaving for patients with acute lung injury or acute respiratory distress syndrome, but the ventilator can injure the lung, causing a condition known as ventilator-induced lung injury.³ The most successful way to minimize ventilator-induced lung injury was demonstrated by the NHLBI Acute Respiratory Distress Syndrome Network: a protocol to lower tidal volumes, compared with a protocol that required higher tidal volumes, improved survival.⁴ This important result was both praised and criticized: proponents argued that low tidal volume protocols should be implemented broadly, but others cautioned that simply reducing tidal volume to a specific level (6 mL/kg of predicted body weight) failed to fully consider the complex mechanics of the injured lung.^{5 - 6} For example, the benefits of avoiding overdistention with lower tidal volumes may be negated in some patients by worsened atelectasis, collapse, or repetitive alveolar open-close shearing with each breath.⁷

Experimental data favor the combination of lower tidal volumes with higher PEEP to achieve an optimal balance of maximal recruited lung with minimal overdistention. However, such titration at the bedside is complex and was previously considered the domain of the expert clinician.^{5 ,7} The 2 trials reported in this issue tested sophisticated protocols that were designed to set PEEP in a reproducible fashion yet titrate it to individual requirements. The Lung Open Ventilation Study¹ took the simplest approach, titrating PEEP to oxygenation, whereas the Expiratory Pressure (Express) Study² required that clinicians adjust PEEP based on pulmonary pressure and volume. In their accompanying editorial, Gattinoni and Caironi⁸ discuss the physiologic and clinical implications. But there are also a number of key study design issues that bear comment.

First, these trials are unblinded. Blinding a complex set of clinician instructions is obviously not practical, but failing to do so exposes the study results to potentially important biases. In particular, clinicians may alter their behavior with regard to other aspects of care based on knowledge of treatment assignment. To reduce the risk of such bias, all aspects of care that affect study outcomes could be described in the protocol. In the Express trial, for example, the investigators disseminated standard instructions for managing hemodynamic changes secondary to ventilator adjustments. However, writing a protocol to include all potential cointerventions used in the care of these critically ill patients, especially in a multicenter environment, is a huge task that may distort usual practice and engender considerable resistance. An alternative is collecting the information necessary to determine whether use of cointerventions was dramatically different across treatment groups. Although more feasible, this effort still represents a considerable logistic and financial burden, does not prevent confounding, and provides only circumstantial evidence as to whether confounding occurred. For example, in both trials, patients who received higher PEEP were less likely to require rescue therapy. Possible explanations are that higher PEEP protected lungs and facilitated recovery; that higher PEEP simply made the patient appear less sick by improving oxygenation without fundamentally changing the risk of death; or that clinicians had greater trust in higher PEEP and therefore felt less need to take additional measures.

Second, defining the control intervention is of crucial importance. The ideal approach is to compare a new intervention with the best current standard of care. However, when the intervention is a complex set of instructions, the alternative could be myriad expert clinician behaviors that are difficult to quantify and reproduce. Both the Lung Open Ventilation and Express studies chose to standardize the care in the control intervention by promoting the use of low tidal volume protocols similar to that tested by the NHLBI Acute Respiratory Distress Syndrome Network. The advantage of this approach is that the intervention in the control group is more explicit, thereby promoting greater confidence in the study's generalizability to other settings where protocols with low tidal volumes are the current standard of care. The main disadvantage is that it is impossible to know whether any of these protocols outperform expert-directed ventilator management.

Third, inferences of causality can only be drawn about the entire intervention and not about any specific piece. These protocol instruction sets can last for days and involve many individual adjustments not just to the ventilator, but also to other important components of care, such as intravenous fluid management. Small flaws in 1 part of the logic could damage the overall intervention's effectiveness. Similarly, it is possible that only 1 part of a protocol is necessary for clinical improvement. This trade-off is a necessary limitation of studies testing complex interventions. A study of individual components of the intervention might provide greater mechanistic insight but would present other challenges, such as the need for multiple intervention groups, each containing separate pieces of the overall intervention; greater sample size; and reduced likelihood that any 1 step could meaningfully impact outcome.

Fourth, dissemination of study results requires careful understanding of the full dimension of the intervention, as well as these methodological and clinical issues. The intervention includes not only the written protocol, but rather the entire process used to put the protocol in place and ensure its adoption. Failure to comply properly with the protocol instructions could compromise efficacy. Investigators of both studies collected an admirable amount of data on ventilator and respiratory parameters and demonstrated strong separation in the care patterns between the intervention and control groups, as well showing that higher PEEP had beneficial effects on key indices, such as oxygenation. Nevertheless, more formal evaluation of compliance with the protocols, and prespecified secondary analyses that accounted for intersite variation in compliance, may have provided insight into the failure to detect a difference in mortality. Information on steps taken to ensure compliance would also be helpful should there be attempts to disseminate higher PEEP strategies to clinical practice.

In summary, despite the relatively straightforward physiologic basis for the individualized titration of the “best” PEEP, generation of robust clinical evidence in its favor is bedeviled by a number of complicated study design choices and implications. Issues largely solved for placebo-controlled drug trials resurface when testing these complex interventions. Nevertheless, both the Lung Open Ventilation Study and the Express Study demonstrated that it was possible to convert the physiologic principles on which experts base their care into a set of reproducible instructions and then test these instructions in a broad multicenter environment. Although neither study demonstrated a significant improvement in mortality, their findings appear to have implications for future practice.⁸ Finally, these studies made important steps toward increasingly rigorous assessment of increasingly sophisticated protocols for the best care of critically ill patients.