Problematic Variation in Local Institutional Review of a Multicenter Genetic Epidemiology Study FREE

Protection of human subjects in research is an evolving process. The current system of institutional review board (IRB) assessment of human subjects protection was established in 1974 in response to highly publicized human research scandals in the 1960s and early 1970s.^{1 - 2} Federal regulation of research conduct and IRB function was implemented in 1979. When IRBs were created, the common paradigm for human subjects research consisted of a single investigator at one institution enrolling local participants, with the major emphasis of regulation on the review of clinical trials.

Over the past 25 years, research strategies and technologies have changed, often bringing together investigators from multiple institutions to enroll geographically diverse pools of participants into epidemiological studies. However, IRB procedures and their federal underpinnings have not correspondingly kept pace.^{2 - 3} Because of the focus of IRBs on clinical trials, others have asserted that IRBs "often have little insight into the needs of epidemiology."⁴ Indeed, it is worth noting that one infamous human subjects research scandal—the Tuskegee Syphilis Study—involved an epidemiology study rather than a clinical trial.¹

The incorporation of genetic information into clinical and epidemiological studies has raised additional problems for the current IRB system. There are few standards set by the Office for Human Research Protections for genetic studies, and substantial disagreement exists within the research community about what constitutes minimal risk in studies that are not clinical trials.⁵ According to Greely,⁶ "Research into human genetics has stretched current regulations of human subjects research beyond the breaking point." The inherent rarity of the outcome and the large number of subjects needed to unravel complex gene-gene and gene-environment relationships often require a multicenter study design to attain sufficient statistical power to generate meaningful results.

Although Silverman et al⁷ have reported variability in the review of multicenter clinical trials, there have been no published reports examining IRB approval for sites involved in multicenter genetic epidemiological studies. In addition, current regulations are not well suited to the complex issues raised by genetic studies. According to Jamrozik, "The current systems of ethical oversight designed primarily to regulate intervention studies involving individual patients associated with single institutions have been completely overtaken by developments in clinical, molecular, and epidemiological research."⁴ When IRB committees do not allow a consistent method of consent among the participants, "selection bias may be introduced and statistical power is certainly decreased."⁴ Therefore, IRBs are largely without guidance in the review of studies that incorporate genetics.

The current method of multicenter review involves approval by each local IRB involved in the study. This results in variability in the type of review, type of consent form, time to approval, changes requested, and the quality of human subjects protection afforded.^{7 - 8} Compounding the problem is the variability inherent in the interpretation of regulations by the estimated 3000 to 5000 IRBs in the United States.² To assess the burden imposed by review of all types of multicenter studies, we determined the yearly volume of single-center and multicenter studies published in the literature since 1974. One method of evaluating the impact on the current IRB multicenter process is to submit a common protocol to multiple IRBs.⁷

We are conducting a multicenter genetic epidemiology study to identify modifiers of cystic fibrosis (CF). Implementation of this study required the participation of CF care centers across the United States. Institutional review board review of the same study protocol varied considerably. Here, we present results of a survey of participating CF centers to document current IRB issues in conducting a multicenter genetic epidemiology study.

A PubMed search was performed to assess the number of English-language human multicenter studies in the literature since 1974.⁹ Abstracts, letters to the editor, review articles, and publication types not containing original results were excluded. In addition, collaborative studies of disease mechanism, disease treatment, or health care delivery were excluded. The frequencies of citations of published, multicenter, English-language human studies in PubMed from 1974 through 2002 were tabulated in 5-year increments. (The algorithm used in the PubMed search is available on request from the authors).

The CF Twin and Sibling Study is a multicenter genetic epidemiology study that was used as a case study to illustrate variability in IRB review. The study involved collection of medical record data along with a blood sample from CF patients who attended CF care centers throughout the United States. A study protocol and consent form developed by the researchers at Johns Hopkins Medical Institutions and approved by the Johns Hopkins University School of Medicine IRB, Baltimore, Md, was distributed to each center. This protocol and consent form was provided as a template for the IRB application at each center. Each center was provided with additional information that included the guidelines for genetic banking provided by the American Society of Human Genetics.¹⁰

To document the process of IRB approval, a 7-question survey (available on request from the authors) was sent to all participating CF centers asking the study staff to provide information on the following: frequency of IRB meetings, dates for submission and approval, use/nonuse of the Johns Hopkins University consent form, type of review performed, types of consent forms required, preparation time, and number of changes requested by the IRB at each CF center. Each center was also asked to provide a copy of all consent forms generated at their center and all IRB correspondence regarding the CF study. Variability among IRBs regarding approval of this study was derived from review of IRB correspondence and approved consent forms. Issues raised by centers and differences among consent forms were categorized and tabulated.

A matrix of consensus statements published from 1987 through 2001 was created to assess the most frequently cited guidelines for genetic studies and to illustrate variability in use of these statements in the consent forms. Data on number of beds, obtained from the American Hospital Association, were used as a proxy for the size of the institution.¹¹ Extramural research revenues obtained from the National Institutes of Health (NIH) were used as a surrogate for volume of research performed at each institution.¹² Number of beds and level of NIH extramural funding in centers that did and did not respond to the survey were compared by t test. t Tests were also performed to assess differences in number of days to approval between centers requiring full review vs those that used expedited review and between centers with children vs those with adults. P<.05 was considered statistically significant. A stepwise linear regression analysis was performed with number of days to approval as the outcome variable. All statistical analyses were performed using SAS software.¹³

The overall number of multicenter studies and the number of epidemiological and genetic epidemiological research multicenter studies published since the establishment of IRBs are presented in Table 1. The number of citations for multicenter studies increased by 1.6- to 3-fold for each of the 5-year periods from 1985 to 1999. However, the number of epidemiology and genetic epidemiology multicenter studies increased 4- to 5-fold every 5 years during the same period. Between 1985 and 1999, the number of multicenter epidemiology and multicenter genetic epidemiology studies increased approximately 8- and 9-fold, respectively, while the increase in single-site studies in the literature was 1.3-fold. Numbers for 2000 through 2002 are consistent with this trend continuing. Thus, multicenter studies of epidemiology and genetic epidemiology comprise an increasing fraction of the multicenter studies reviewed by IRBs.

Thirty-one of 42 CF care centers involved in a multicenter genetic epidemiology study replied to the survey of their IRB approval process, yielding a 74% response rate. Twenty-four (77%) of the 31 institutions required full IRB reviews, and 7 (23%) considered the blood draw and medical record review in the protocol to be of minimal risk and eligible for expedited review based on their interpretation of the federal regulations.¹⁴ Twenty-nine centers (94%) required use of consent forms from their own institution. Three centers (10%) required 4 forms (adult, minor, parental, and assent). The number of centers requiring 2 or more consent forms was 15 (48%). Ten centers (32%) did not require an assent for children. Of the 21 centers that did require an assent, 10 (48%) provided a separate assent form that included an explanation of the study; the rest required a signature or initials of assent on a consent form written for an adult. The specific age range of patients for which assent was required varied considerably among centers. Ages specified for assent ranged from a minimum of 7 years to a variable maximum of 12 to 18 years. Nine assent forms (43%) did not specify age. There were no statistical differences between centers with children vs centers with adults. The number of consents required by a center was independent of whether the review was full or expedited. To assess the issue of response bias due to differences in the familiarity of centers with human subjects research, the number of beds and the level of NIH extramural funding in the centers were compared between the those who did and did not respond to the survey; no differences were found.

The mean time to obtain approval for an expedited review was 32.3 days (range, 9-72 days), and the mean time to obtain approval for a full review was 81.9 days (range, 13-252 days). The range of preparation time for the full review varied from 2 hours to as many as 40 hours. Predictably, the mean preparation time for an expedited review was shorter than that for a full review (5.8 vs 14.8 hours) and the mean number of changes requested was lower for an expedited review (5.7 vs 8.6). Preparation time was not separated into time to initial submission to the IRB and time to make changes and resubmit to the IRB. This information may be useful in future studies of this type. Prior to regression analysis, correlation analyses were conducted for all variables regardless of review type, all variables for centers using full review, and all variables for centers using expedited review.

Days to approval, an indicator of the difficulty of review, correlated with the number of changes requested when both review types were combined (P = .004) and with full review (P = .01) when the data were stratified by review type. No other significant correlations were observed. t Tests were performed for comparison of the full and expedited review groups. There were no significant differences between the groups except in preparation hours (P = .01). However, the paucity of numbers for the expedited review groups requires cautious interpretation of this result. Although the sample size was small, a stepwise regression analysis was performed with number of days to approval as the outcome variable. Number of changes required was the only variable that predicted number of days to approval.

The large variability in days to approval was not explained by the variability in meeting schedules, hours of preparation, number of consent forms, size of the institution, or volume of research dollars received by the institution. The large variability in the content of the consent forms and the number of changes requested was explained in part by differences in the amount of genetics-related information provided and the high percentage of questions regarding the genetic aspect of the study. Institutional review boards from smaller institutions with lower research revenues tended to ask more questions, which, in turn, led to longer preparation time.

Review of correspondence between the IRB and the study principal investigator at each center revealed that a substantial fraction (52%) of issues raised by local IRBs related to genetics. Most genetics questions related to DNA banking and risk-benefit analysis (Table 2). Questions related to nongenetic issues of confidentiality accounted for only 35%. There were also several questions that referred to clinical trial design tools that were not a part of this observational study. Only 2 centers (6%) explained that the study was observational and that there would be no treatment involved.

A review of consent forms consistently revealed language required by each individual IRB for all studies at their institution. In general, the required templates were not well suited to a genetic epidemiology study.¹⁵ For example, most consent templates provide information regarding data and safety monitoring boards. An IRB in this study requested that this information be included in its consent form despite the fact that this study did not contain an intervention for it to monitor. In addition, items necessary for genetic studies are not found in the templates, such as assurances of confidentiality for family members, since families are a unit of research in genetics. Finally, although each center was provided with DNA banking guidelines¹⁰ and other guidelines were available, few consent forms contained information relating to purpose/advantage, location, use, and confidentiality procedures or withdrawal procedures (Table 3).

Since the early 1980s, the growth of multicenter studies in the scientific literature has been dramatic. This increase has raised researchers' concerns about the adequacy of human subjects protection.^{17 - 18} In 1998, the deputy inspector general issued a report calling for the reform of IRBs. The report noted the inability of IRBs to cope with rapid advances in biomedical research and changes in the research environment, from conduct of small single-institution studies to larger multi-institution studies, and inadequacy of reviews due to increased workload, due in part to an increase in multicenter studies, lack of resources, and insufficient scientific expertise, with many IRBs spending "only 1 to 2 minutes of review per study."¹⁹

The dramatic increase in the number of all multicenter research studies supports previous reports of the impact of this research strategy on review of clinical trials.^{17 ,19 - 21} The number of multicenter genetic epidemiology studies found by our search of the PubMed database may be underestimated because of the infrequency of the term genetic epidemiology in the database during the early years. This limitation was in part overcome by use of a series of Medical Subject Heading terms that describe epidemiological studies (eg, case-control) and by combining them with multiple genetic descriptors (eg, hereditary). Indeed, a review of 10% of the results of the search strategy yielded a maximum of 8% false-positive results.

It was not possible to differentiate between multicenter studies with separate IRB approval and studies in which review was performed only at the originating center. Because of the nature of the PubMed database, publication bias may also reduce the number of genetic epidemiology studies found in our search. However, this would result in an underestimate of the numbers, thus implying stronger results. Although the number of multicenter studies constitutes a small fraction of all research studies, the amount of work involved in the multiple reviews of a multicenter study imposes a disproportionate burden on the IRB system. Thus, the rapid increase in use of the multicenter research strategy underscores the urgency for changing the current process of IRB review of multicenter studies.

Using a single multicenter genetic epidemiology study as a case study, we observed considerable variability in local IRB assessment of type of review required. There were differences among local IRBs as to what constituted minimal risk when coded rather than anonymous data were involved and when any genetic information was involved. The definition of minimal risk in research has been a source of debate during the last decade.^{5 ,7} Many IRBs struggle with the ideas of risk and benefit in nonintervention studies. Researchers and ethicists are divided as to whether genetic studies should always be considered to be of higher risk than other forms of research.²²

All participants in this study were patients with a well-defined genetic disorder, CF. However, IRBs seemed confused about what risk information they needed to provide the participants. It has been noted that IRBs lack experience in finding the equipoise in a risk-benefit analysis in which the risk is psychosocial and any benefit is solely scientific knowledge and, hence, indirect.²³ Genetics introduces probabilistic risk information that incorporates the concepts of penetrance and variable expressivity²⁴ and, often, unconfirmed estimates of risk perception, which further complicates determining risk-benefit ratios. These issues were illustrated by considerable variability in how IRBs dealt with DNA banking within their consent forms.

As shown in Table 3, guidelines for consent for genetic studies have been issued by several organizations.^{10 ,25 - 30} However, these guidelines are not consistent. Sometimes conflicting guidance has been offered, which, we speculate, contributes to the observed inconsistency among the IRBs. As noted by Francis Collins, "Many groups have made recommendations; researchers and IRBs are still confused. The IRB Guidebook is dusty and out of date for genetics research."³¹

The National Bioethics Advisory Commission found considerable disagreement across IRBs regarding "when informed consent should be required, and what constitutes proper consent."³² Variability in IRB review was also revealed in this study by the types and numbers of consent forms required and the content of the consent forms. In this study, the lack of consensus among IRBs regarding assent was exemplified by variability in the assent requirement. Institutional review boards are required to set ages of minority and majority based on local laws and their own judgment, taking age, maturity, and psychological state into consideration.¹⁴ State definitions of minority age range from at least 12 years to at least 17 years, while age of majority ranges from 18 to 21 years.³³ Most IRBs appear to apply the local legal definition when preparing assents.

We observed that some IRBs prepared an assent form with a grade 2 to grade 4 reading level, while others only furnish a space for a signature on a consent form requiring greater reading skills. This practice introduces considerable variability in the level of protection afforded to children participating in the same research. Thus, variability in multiple local IRB reviews uncovered differences in review criteria that could lead to uneven protection of human subjects. In addition, the inefficiency of multiple and variable IRB reviews of a single research protocol postponed the time to study initiation and resulted in redundant allocation of valuable IRB resources without adding substantially to the protection of human subjects.

A possible solution would be the creation of an independent national multicenter IRB review system overseen by the Office for Human Research Protections. An independent but federally accredited central multidisciplinary IRB program for multicenter studies could obviate concerns regarding inadequate staffing and education of IRBs, the burden multicenter review places on local IRBs, variability among IRB reviews, continuity of human subjects protections among all participating institutions, IRB availability at smaller institutions, and institutional conflict of interest.³⁴ Membership could be drawn from a pool of qualified individuals with various levels and types of expertise. To ensure the quality of the review, membership should be recognized within the scientific community with the same level of recognition attributed to membership in a study section at the NIH. Local IRBs would review multicenter research approved by a federally accredited, independent central IRB in an expedited fashion. Full local IRB review would be undertaken only if the expedited review revealed a potential for adverse effects on a community within their catchment area, conflict with local or state regulations, or overlap with ongoing studies within their institution. Cooperation would be essential to make it work.³⁵

Several other solutions to the multicenter dilemma have been proposed and implemented to greater or lesser effect.^{5 ,18 ,20 - 21 ,34 ,36 - 39} The NIH is currently considering establishing regional multicenter IRBs. Although this approach may decrease the burden on the local IRBs, it does not address the issue of variability among IRBs, does not deal with the issue of indemnification, and would require a centralized system for resolving conflicts among regions.³⁴

The debut of the new Health Insurance Portability and Accountability Act (HIPAA) privacy regulations, written with an emphasis on single-institution clinical trials, could cause review of multicenter research to be even more prone to variation in human subjects protection and inefficiency. This is particularly true for genetic epidemiology studies, which often require the participation of many centers. The requirement for detailed disclosure documentation has raised apprehension that "this is a very complicated and expensive task, and some healthcare organizations will simply choose instead to deny researchers access to the information."⁴⁰ Concerns regarding a criterion for minimal privacy risk seem to echo those previously expressed, and unanswered, regarding minimal risk in federal research regulations.⁴¹ No attempt has been made to create a standard individual privacy authorization or data use agreement. This leaves the regulation and the language to interpretation at each institution. Based on our experience with IRB review in this study, we anticipate that local IRBs will differ in their interpretation of the HIPAA, thus adding another layer of variability in the review of multicenter studies and further complicating the execution of studies evaluating the contribution of genetic variation to common disease.

In summary, the dramatic increase in multicenter studies has substantially increased the workload of local IRBs. A result of the Human Genome Project has been an increased interest in the application of epidemiological techniques to genetic research, which will lead to continued increases in the number of multicenter genetic epidemiology studies. However, the current multicenter approval process is onerous because of inexperience with new technologies and science, outdated regulations, and a lack of unified comprehensive national standards. The current approval process results in variability in the review of multicenter research. The observed variability is due to an absence of uniform standards to protect subjects in studies addressing disease etiology. The HIPAA will very likely add more variability. A centralized review board for multicenter studies, particularly genetic epidemiology studies, could reduce the variability in human subjects protection among medical centers, ensure that proper expertise is applied to each study, decrease the time required for review, and lessen the burden on local review boards. The need for reform appears necessary if we are to reap the full potential of the Human Genome Project.