Infection Prevention and Comparative Effectiveness Research

Health care–acquired infections, particularly those due to antimicrobial-resistant bacteria, have received significant attention in recent years. Despite work focused on elucidating the epidemiology and effects of such infections, success in curbing their emergence remains elusive. Few new classes of antibiotics are even in the earliest stages of development, making efforts to prevent the emergence and spread of antimicrobial-resistant bacteria even more crucial. However, the evidence base on which potential infection-prevention strategies must be built is severely limited because very few of the necessary clinical trials have been conducted. The generation of such scientific evidence is constrained by the perceived difficulty of completing the necessary studies and limited federal funding available for assessing infection-prevention interventions.

An increasing number of legislative mandates, such as the state of Illinois' methicillin-resistant Staphylococcus aureus (MRSA) active surveillance mandate, have been enacted to force the implementation of infection-prevention strategies.¹ Many of these mandates lack a strong scientific foundation and considerable resources are now devoted to supporting them without a clear template for how to evaluate their clinical and economic ramifications. To effectively protect patients, rigorous studies must be conducted to assess the comparative effectiveness of different infection-prevention interventions. Thus, scientists and funding agencies need to recognize the strengths and limitations of the epidemiological methods that could be used to address these critical questions. This commentary focuses on 3 complementary methods for comparative effectiveness research in infection prevention: cluster randomized trials, quasi-experimental studies, and mathematical models.

A well-designed and adequately powered randomized controlled trial (RCT) provides the most rigorous evidence for or against the efficacy of a given intervention. In health care epidemiology, interventions to reduce device-related infections (eg, antimicrobial-coated central venous catheters) are often amenable to an RCT investigative approach because the intervention and the observed benefit occur at the level of a single patient and the effect of the intervention for one patient is independent of the effect on a different patient.

However, many interventions focus on population-level strategies and are not amenable to an RCT approach. For example, MRSA screening programs test patients for MRSA carriage and isolate colonized patients to prevent transmission of MRSA. These screening programs indirectly benefit patients who are not isolated. To assess population-level interventions, alternatives to RCTs are needed.

The cluster randomized trial is well suited to study the comparative effectiveness of population-level interventions.² Cluster randomized trials may involve randomization at different levels including the full hospital or individual hospital units. These trials are complicated, costly, and time-consuming but are absolutely vital if population-level interventions are to be adequately evaluated.

In health care epidemiology, clinicians and infection control personnel frequently are compelled to act quickly due to patient safety concerns (eg, an outbreak). Using a cluster randomized trial approach in this setting is both infeasible and often unethical. It is also of interest to study changes that occur outside the control of the investigator (eg, legislative mandates). Assessing the potential effects of these broader initiatives is critical for building the evidence base for health care epidemiology. In these instances, a cluster randomized trial approach also would not be feasible.

An alternative design to RCTs and cluster RCTs is the quasi-experimental (QE) study (sometimes called a “before-after” study). Quasi-experimental studies aim to evaluate interventions but do not use a randomized control group. In the simplest QE design, a population serves as its own control during a baseline period of observation. An intervention is then implemented, and a subsequent period of observation is completed. Changes in the outcome of interest are then compared before and after the time of the intervention.

The role of the QE design in the investigation of infection-prevention interventions has been described.^{3 - 4} If not properly designed, QE studies will have several threats to internal validity including uncontrolled confounding and selection bias. Several approaches may be used in QE studies to address these limitations, including assessment of outcomes during a prolonged baseline period, use of nonequivalent control sites in which no intervention is implemented, and collecting data on confounding variables.³ In addition, QE designs that match intervention and control groups on pre-intervention measures of the outcome of interest (eg, infection rates) can help reduce the effects of selection bias and thereby import higher internal validity.⁵

The statistical analysis of QE studies requires careful attention. Segmented regression analysis is an optimal method used to assess both changes in infection rates from the preintervention to the postintervention period while also assessing for changing trends in outcome rates.⁶ For example, by only comparing mean infection rates before and after an intervention and ignoring the trends in infection rates, the analysis could miss a trend toward declining infection rates when one exists. Efforts must also be made to control statistically for potential confounding and selection bias with multivariable regression, propensity scores, or instrumental variable methods.

However, many previous studies using a QE study design have failed to acknowledge these limitations or otherwise address them.⁴ It is possible that the shortcomings of prior QE studies have hindered wider acceptance of this approach. However, QE designs have significant potential advantages, including efficiency, generalizability, and the unique ability to assess the temporal relationship between nonrandomizable events (eg, legislative mandates) and outcomes. If this study design were more broadly embraced and rigorously implemented, the evidence base for health care epidemiology might expand significantly.⁷

In both QE studies and cluster randomized trials, the interventions assessed are often implemented at a unit level. Thus, participation of all individuals within these units is paramount. As such, seeking individual consent may be infeasible as well as problematic. For example, if individual consent is sought and only 50% of patients agree to participate, a unit-level intervention cannot be studied. It is possible that the perceived barriers imposed by ethical concerns have limited the use of QE and cluster randomized trials.⁸ Although further efforts are needed to inform the proper ethical conduct of these studies, researchers planning these studies must ensure appropriate steps for institutional review board evaluation and oversight are followed.

Both cluster randomized and multicenter QE studies require large collaborative networks of sites willing to participate. Given the demonstrated differences in health care–associated infections across institutions, broad representation of different types of institutions from diverse geographic regions is necessary to expand multicenter health care epidemiology research. Early work has demonstrated the potential resources of a large health care epidemiology research network,⁹ but sustained funding for such networks is needed.

Traditional studies, such as RCTs, are critically concerned with causal inference—determining which exposures are linked to a specific outcome. These studies must maximize internal validity by limiting confounding through randomization, matching, or statistical control. On the other hand, mathematical simulation models assume statistical association (eg, improving hand-hygiene compliance is associated with reduced transmission of infectious agents) and predict the consequences of the assumption, in many potential hospital types (eg, urban vs rural). Even though modeling studies are subject to limitations (eg, based on available data and assumptions), these studies allow for interpreting existing epidemiological studies and testing hypotheses, such as the conditions under which an intervention might be most cost-effective. A recent Institute of Medicine panel acknowledged that certain comparative effectiveness questions cannot be answered using clinical trial methods and noted that “when properly constructed and independently validated, these models not only serve as useful tools to identify, set priorities in, or facilitate the design of new trials, but also can be engaged to conduct virtual comparative effectiveness trials.”¹⁰

The focused and coordinated use of well-designed quasi-experiments, cluster randomized trials, and mathematical models offer significant potential opportunities for improving the scientific understanding and targeting infection prevention efforts. The scientific community, including investigators, scientific societies, and funding agencies, must be willing to consider these complementary methods and facilitate the creation and support of the collaborative research networks in which to complete the necessary investigations.