Monitoring and Ensuring Safety During Clinical Research

In recent years, it has become increasingly clear that clinical practice should be based on empirical evidence.^{1 - 2} This tenet, coupled with the burgeoning number of potentially beneficial interventions, has intensified the need for rigorous clinical trials to test medical interventions for safety and efficacy. Despite the urgency of gathering these data, it is essential that patient-subjects be protected in the process.³

Substantial responsibility for ensuring the protection of human subjects is now vested in institutional review boards (IRBs). These boards are charged with reviewing and approving research involving human subjects before it is conducted and with reevaluating it at regular intervals to assess continued safety of the protocol.^{4 - 6} In all trials, ensuring the safety of participants is not solely the responsibility of IRBs, but also of a multitude of others: investigators, sponsors, the US Food and Drug Administration (FDA), the Office for Human Research Protections (formerly the Office for Protection from Research Risks), and data monitoring committees (DMCs), also called data and safety monitoring boards or data and safety monitoring committees. Nevertheless, serious concerns have been raised regarding the processes by which the safety of participants in clinical trials is currently monitored.⁷ Moreover, the emergence of large, multicenter, and sometimes international clinical trials and the increasing shift of funding for clinical trials to industry have made apparent the inadequacy of mechanisms for protecting patient-subjects that were developed during a period when clinical research was generally carried out on a small scale at single institutions.

To address these concerns regarding the protection of human subjects in the current and evolving environment in which clinical research is conducted, a group of professionals with expertise in various aspects of clinical trials was assembled by the authors in May 2000 (the names of members of this group are listed at the end of this article). Specifically, we included individuals with expertise in the design, conduct, sponsorship, analysis, and reporting of clinical trials, as well as those with expertise in the regulations governing research with human subjects, IRBs, DMCs, research ethics, and government oversight.

At our initial meeting, participants described and evaluated the mechanisms by which clinical trials are monitored, focusing on adverse event (AE) reporting and the processes by which various parties with oversight responsibilities may or may not interact in the course of these trials. Based on this information, we describe the manner in which AE reporting would ideally function to enhance safety and the role of DMCs in using aggregate data from these reports along with other data evolving during a clinical trial. We then frame a statement of the problems that currently exist for IRBs in fulfilling their obligation to conduct meaningful ongoing review clinical trials, and offer recommendations for improving the current approach. Although we benefited greatly from the insights, opinions, and perspectives of the participants at our initial meeting, the views presented herein do not necessarily represent those of the group because we did not focus attention on reaching consensus concerning these matters.

Adverse event reporting ideally should provide useful information regarding safety in a clinical trial. In current practice, when AEs are observed in a trial, investigators submit adverse event reports (AERs) to the local IRB and the study sponsor.^{8 - 9} General AEs are typically recorded on case record forms; serious AEs are reported in an expedited fashion to the research sponsor and to the IRB. To ensure the safety of participants in a clinical trial, AEs need to present an accurate and expanding picture of potential harms. This includes making a rational assessment of the risks and benefits of an experimental therapy and of the overall welfare of patient-subjects in a trial. Doing so assumes that the random variation in the background rates of adverse medical outcomes prohibits the assessment of causality in an individual event except in very unusual circumstances. Indeed, in many situations the overall positive or negative profile of a therapeutic approach has been created by events considered part of the natural history of the disease of interest until the excess events in a treatment group statistically exceeded the rate of events in a control group. For example, since sudden death due to cardiac arrhythmia is part of the natural history of myocardial infarction, a single death due to arrhythmia would not, in and of itself, engender concern in a clinical trial testing a new antiarrhythmic agent.¹⁰

To conduct a valid assessment of an AER, it is necessary to have information beyond that contained in the report itself, such as the number of patients in the study as a whole, the expected frequency of the AE reported, and, in a blinded study, information about whether the patient-subject in question is receiving the test agent. Information on efficacy is also necessary to weigh risks and benefits. For example, a fibrinolytic agent tested in the setting of acute myocardial infarction may be associated with an increased risk of intracranial hemorrhage in a small fraction of patients, yet in a larger number of patients the agent results in long-term survival benefits due to a decrease in cardiac death.¹¹ An AER without information about benefit would justifiably be alarming. In addition, information such as the demographic characteristics of patients enrolling in the trial is necessary to determine if any group of individuals is unfairly bearing the burden of the risks or is barred inappropriately from receiving the potential benefits of study participation (as well as if the trial will provide sufficient information to support use of the intervention if it is shown to be effective).

Data monitoring committees have emerged as a means of assessing appropriateness of continuing clinical trials based on evolving trial data.¹² Such committees typically consist of experts in the disease or condition under study, biostatisticians, and sometimes ethicists and patient representatives.¹³ So as to not bias the conduct of the trial by revealing early data, DMCs act independently of the sponsor and the study investigators.

The sponsor or steering committee of a study charges the DMC to protect patient-subject safety by examining the accruing data for indications that clear benefits or harm may be occurring to individuals participating in the trial.¹² The DMC then makes a judgment as to whether the trial should continue. A decision to stop a trial is usually made either because excessive benefits or AEs have been observed or because one study group shows results significantly different from the other, with obvious ramifications for patients and subjects. In performing its work, the DMC is usually looking globally at the community of patients and not at an individual patient-subject. This is possible because investigators typically report AEs to a data coordinating center, which then compiles data for the DMC to review at predefined intervals. The DMC then uses preplanned statistical analyses to determine if any serious AEs or clinical end points differ between the treatment groups; it does not ordinarily evaluate the specific circumstances of individual events not considered end points. The DMC traditionally reports directly to the sponsor or steering committee, not to the IRB, investigator, and, generally, not to regulatory authorities, although it has obligations to all these groups.¹²

Although many large-scale trials use a DMC, individual oversight of research by an IRB is the mainstay of federally mandated subject protection.^{5 - 6} However, the extent to which an IRB can evaluate accurately the potentially critical information in AERs is questionable, especially in large-scale, multicenter trials, due to a variety of factors. First, IRBs receive individual AERs, both from their own site and other participating sites, yet these reports are difficult to compile and interpret because they are submitted to the IRB as individual reports typically without any explanation of how the event relates to any other events previously observed.¹⁴ Second, in international trials, differences in standards of care and language may make the clinical significance of reported AEs even more difficult to interpret. Third, in blinded studies, this information is provided to IRBs without a treatment code and information on efficacy. Fourth, multiple trials with an experimental agent are often in progress simultaneously, but each IRB may have access only to the data from trials over which it has authority. Similarly, an IRB might not consider other relevant data from studies at other institutions testing the same or a similar agent or other treatments for a particular disease completed during the ongoing trial that it is reviewing.⁸ Fifth, it is likely that many IRBs do not have sufficient statistical or clinical expertise or access to appropriate information to allow them to evaluate properly the issues of safety and benefit that arise in the course of a trial. Perhaps this is not surprising because there are no federal requirements to have such expertise on IRBs. As a result of these factors, IRBs frequently are unable to translate observations regarding individual AEs into a coherent assessment of the overall risks and benefits for a trial.

Moreover, IRBs have become inundated with AERs, with as many as several hundred reports submitted to IRBs that oversee a large number of studies each month.⁸ Some of the excessive burden that AERs create for IRBs may be attributed to the following: confusing terminology in the regulations that govern trials, differing requirements of the governmental regulatory bodies involved in ensuring patient-subject safety, and inconsistencies in the regulations themselves. In certain cases it is not clear precisely what AEs must be reported, how rapidly they must be reported, who shoulders the main responsibility for reporting, or to whom completed reports must be submitted. For example, the FDA requires the investigator to "promptly report to the IRB all unanticipated problems involving risk to human subjects or others."⁹ Regulations from the Department of Health and Human Services require prompt reporting to the IRB of "any unanticipated problems involving risks to subjects or others."¹⁵ However, the word "others" in the first regulation is confusing since it could refer to risk to other patients in the trial or even to those not in the study. In both regulations, the term "unanticipated problems," left undefined, could encourage the reporting of any problems, related or unrelated to the trial. In another section of the FDA regulations, investigators are required to report (to the sponsor, but also, by implication, to the IRB) "any adverse event that may reasonably be regarded as caused by, or probably caused by, the drug" being studied.¹⁶ These requirements could be interpreted to mean that all AEs, whether serious or not and whether they occur locally or at another site, must be reported by the investigator. What about minor medical events of unknown relationship to the study drug? What about events expected as part of the natural history of the disease of interest?

The confusion in reporting responsibilities is further compounded by FDA regulations that require sponsors to notify investigators of "new observations discovered by or reported to the sponsor of the drug [under study], particularly with respect to adverse effects and safe use."¹⁷ This directive could be interpreted as a requirement that all AEs be sent to all investigators. Eager to avoid liability, sponsors of clinical trials might then require investigators to submit all AERs to their local IRBs; in some cases, even in international multicenter trials, all AERs are sent to multiple individual IRBs.

In contrast to myriad requirements for reporting AEs, US regulations lack provisions about how IRBs should handle these reports once they have been received. The FDA does not actually require that the IRB review all AERs but depends more on the study sponsor to accrue data concerning AEs. To date, federal auditors have been interested in determining whether investigators are complying with the requirement to report AEs and whether the events recorded on case record forms are consistent with the medical records of the patients who have experienced the AEs. This is not a systematic approach to controlling the quality of clinical investigation; it is instead a blunt, nonquantitative assay of whether investigators or study coordinators are sloppy or duplicitous in their work.

Flooded by AERs and poorly positioned to interpret the emerging trial data, IRBs have tended to focus on optimizing regulatory compliance instead of using AERs to determine whether the risk-benefit assessment for locally enrolled patients is affected. When the prospect of many individual IRBs in large studies all attempting to replicate an assessment of the safety and efficacy of the therapy of interest is considered, the implications are magnified. At the same time, the enormous amount of work performed by IRB administrators and members to complete these functions is likely to be costly.

If IRBs are usually unable to determine accurately from isolated AERs whether undue threats to the safety of patient-subjects exist, then who should characterize the emerging safety profile of an intervention, whether an investigational medical product, a behavioral intervention, or an administrative practice intervention? Ultimately, no single group can provide complete protection of patient-subjects. A systematic plan is required for each trial so that appropriate input comes from each entity involved in its oversight. Furthermore, communication among the different entities needs to be enhanced and these groups need to come to an understanding of their complementary and unique roles in the conduct of trials.

In a small, single-center study, the IRB, in conjunction with the investigators, must assess the collected data on AEs and positive clinical outcomes. Performing this task requires that those with adequate expertise in clinical trials, statistics, and the clinical condition being studied periodically review the data to ensure proper reporting and interpretation of AEs. A knowledgeable and experienced study statistician can often serve this role. Investigators or others charged with data monitoring should plan to provide aggregated data summaries to the IRB at prespecified intervals and explanations about the seriousness and relatedness of the AEs to the study intervention. This plan should be explicit and should be agreed to by the investigators, the IRB, and those assuming responsibility for monitoring before the trial begins.

In most multicenter trials, an independent DMC is required to provide the type of aggregated data that permit meaningful oversight. The DMC, the sponsor, and the investigators must develop a systematic approach for reporting to local IRBs so that the IRB's function of assessing continued subject safety is not abrogated. Institutional review boards should not be forced to function as DMCs, and there should be no overlap of their functions.

Unfortunately, the composition and function of DMCs and how they are to communicate with IRBs and other parties in multicenter studies is often vague.^{18 - 19} As a result, the DMC's specific operational and statistical approach to interim analysis of incoming data should be constructed by the sponsor, the study statistician, and the steering committee for the study, and assessment of this approach should be part of the IRB approval process. Safety parameters should be established at the outset of the study. These parameters might include specified rates of certain toxicities or beneficial outcomes that would require discontinuation or modification of the study. Although the DMC should not set the rules for interim analysis, it should indicate whether the proposed rules are feasible. In addition, how the DMC will interact with the IRB should be well outlined.

Once research has begun, in most cases, the DMC should provide a simple report to each IRB at appropriate and regular intervals indicating that the safety parameters established at the start of the study have not been exceeded and that there are no reasons based on evolving information concerning risks and benefits that the study should not continue.²⁰ However, direct communication between the DMC and IRB also could create problems, including difficulties in communicating information in a timely and usable fashion. This topic is worthy of continued evaluation. The requirement that all studies, both single center and multicenter, have a clear plan for monitoring safety also is appropriate.²¹ At the same time, the responsibility that an IRB currently shoulders should not merely be transferred to a DMC. The important point is that the information provided by the DMC to the IRB needs to be well defined so that members of the IRB can expect that they will obtain the data needed to make decisions about whether local modifications to the protocol or informed consent process are required without unblinding the study or adversely affecting the investigators or sponsors.

In addition to these efforts at enhanced communication, a variety of other modifications to the current system seem in order (Table 1). Regulatory agencies ought to define their nomenclature for AEs with more precision and harmonize their requirements for reporting. For example, it would seem to make sense that only serious, unexpected, and potentially related AEs (not those that are part of the disease process under study or those that are related to the known mechanisms of the experimental agent) should be reported in an expedited fashion as they occur. While the mechanisms for such reporting would need to be developed and evaluated, having persons knowledgeable about the experimental agents and the disease processes evaluate AEs to determine the need to provide expedited vs routine reporting is of central importance. Perhaps lists of AEs that would not need expedited reporting could be created at the outset of a trial.

Serious, unexpected, and potentially related AEs should be assessed by the IRB in the context of an analysis provided by the study sponsor that ideally includes a recommendation for whether modifications in the informed consent and additional safety measures are warranted. These, as well as other reports, should be compiled on a case record form and sent in summary statements to the data coordinating center. Adverse events known before the start of the study to be critical events in the assessment of safety and efficacy should be reported in a systematic fashion as either present or absent. This approach will allow summary reports to be interpreted with confidence that the observed incidence rates are accurate.

Institutional review boards should have 3 major roles in a multicenter trial in addition to reviewing the research protocol, including the procedure for obtaining informed consent, before the study begins. First, the IRB should examine and approve the plan for studywide monitoring. Second, each IRB should be able to certify that the investigators under its purview understand the regulations governing the continued safety of patient-subjects in the trial, such as AE reporting and the collection and maintenance of study documents. Third, each IRB should review aggregate AERs and DMC communications as part of a thorough, continuing review of research. After reviewing this information, IRBs may determine that protecting the welfare of the patient-subjects for whom they are responsible requires that the investigator be queried as to whether additional safeguards may be necessary.²²

Institutional review boards should also seek input from the study sponsors (whether government agencies, foundations, or industry groups) in making assessments of safety in trials. Sponsors often possess a considerable amount of information from other studies (animal and human) and usually are in an excellent position to characterize AEs and their relationship to the intervention being investigated. This may be best accomplished by requesting that sponsors compile and maintain up-to-date information with a cumulative summary of AEs and a statement of the probability of relationship with the intervention so that IRBs can decide on an ongoing basis whether additional safeguards are required. By receiving a summary of relevant data, IRBs would have access to the type of information they need to protect human subjects without being overburdened by extraneous data. This approach would allow a more systematic approach to be taken to analyzing and assessing the aggregate data collected from trials.

Nonetheless, potential conflicts of interest should prevent any sponsor from being the sole monitor for a trial. This approach is especially important in phase 1 and 2 trials, which are typically conducted specifically to assess toxicity and are not designed to make a definitive evaluation of whether a therapy should be applied in practice. Therefore, some independent means of monitoring must be used. In definitive phase 3 trials, we recommend that the sponsor allocate responsibility for safety evaluation to an independent DMC to minimize conflicts of interest among those making the necessary judgments during a trial.

The current system for protecting human subjects in large-scale multicenter trials is outdated. It charges IRBs with functions that they cannot credibly perform. We believe that adopting the approaches outlined herein would likely enhance the safety of human subjects who participate in multicenter trials. Nevertheless, we advocate the continued evaluation and assessment of this system and refinements where necessary so that important clinical research can be conducted without jeopardizing patient safety and without creating unnecessary work.