Status of Clinical Research in Academic Health Centers: Title and subTitle BreakViews From the Research Leadership FREE

Clinical research, defined as research that involves living humans as subjects, is composed of a wide spectrum of research types such as clinical trials, translational research, epidemiological research, health services research, and outcomes research and is critical to translating the results of basic science into useable health care products and services.¹ The challenges facing the clinical research enterprise in the United States have become major concerns of the academic medical community. One reason is that these challenges could eventually reduce the flow of future medical knowledge by inhibiting the amount or reducing the quality of the research conducted.^{2 - 3} Realizing this potential, the Institute of Medicine recommended the establishment of a US $1 billion fund to incorporate, rapidly and appropriately, the results of scientific research into clinical practice—one of the causes of low-quality health care.⁴

Much of the "crisis" in clinical research has been attributed to numerous changes in the health care environment.^{5 - 7} First, insurers are less willing to pay the higher costs of patient care at academic health centers (AHCs). Thus, institutions in highly penetrated managed care markets have a reduced ability to cross-subsidize clinical research from patient care revenues.⁶ Second, health plans may limit the availability of patients to participate in clinical research at AHCs by directing them to lower-cost providers⁸ or by refusing to cover the additional costs of patient care that occur as a result of clinical research. Both of these actions could reduce the potential supply of patients to serve as subjects in clinical research. Third, young faculty may spend more time seeing patients to cover their salaries and generate clinical revenues for their institution. This may explain why clinical researchers in the high managed care areas have greater clinical care responsibilities and have a reduced amount of research funding from the National Institutes of Health (NIH).^{5 ,7} As a result, insufficient numbers of trained clinical researchers are currently available to meet the demand.⁹ Finally, AHCs now face competition from nonacademic contract research organizations (CROs) to supply clinical research services.¹

In response, some AHCs have taken steps to support their clinical research mission by streamlining the human subjects review process; improving the grant-writing capabilities of their faculties; or assisting researchers to identify new funding sources, establish collaborative relationships, and recruit patients.¹⁰ Unfortunately, empirical work is scarce regarding the nature or extent of the problems facing clinical research or the impact of the remedies AHCs are undertaking. Importantly, concerns about clinical research are rarely viewed in a wider context, such as relative to the status of nonclinical research.

One window into the world of clinical research in AHCs is the perspective of research leaders, including senior research administrators and department chairpersons in US medical schools. These individuals are likely to be close to the research being performed but far enough removed to achieve a sense of the problems and potential solutions occurring at departmental and institutional levels. Therefore, we conducted a national survey of the research leadership to examine the following questions: (1) How healthy are the clinical and nonclinical research enterprises in US medical schools? (2) What challenges do medical schools face in protecting their clinical research mission? (3) What strategic adaptations are they using to protect their clinical research mission and what impact have these adaptations had on the amount of clinical research performed?

The sample for this study consisted of the senior research administrator (SRA) and the chairs from 5 departments at each medical school in the United States. Using lists from the Association of American Medical Colleges, we identified the chairs of the departments of anesthesiology, internal medicine, neurology, pediatrics, and psychiatry at all 122 accredited, allopathic medical schools in the United States, excluding Puerto Rico. These departments were selected because they normally represent the most research-intensive departments in medical schools. Since not all medical schools contained all 5 departments and some positions were vacant at the time of the survey, our universe of department chairs was 587 subjects.

We also identified the SRA holding the title of vice, associate or assistant dean for research, research administration, research affairs, or academic affairs. For schools that did not have an individual holding one of the above titles, we asked the dean to identify the person in his/her institution who held the primary responsibility for planning and managing the research activities of the medical school. For 3 medical schools the dean identified 2 individuals as SRAs. Hence, our universe of SRAs was 125 rather than 122.

Our total sample consisted of 712 subjects (125 SRAs and 587 department chairs). Among the department chairs, there were 110 chairs of anesthesiology, 123 chairs of medicine, 112 chairs of neurology, 122 chairs of pediatrics, and 120 chairs of psychiatry.

We developed 2 survey instruments, 1 for SRAs and 1 for department chairs. To determine important content areas, develop and refine our hypotheses, and compose specific survey items, we held a focus group with potential respondents, interviewed informed colleagues, and reviewed the relevant literature. The instruments were pretested using 6 cognitive interviews conducted by professional survey researchers at the Center for Survey Research at the University of Massachusetts. The survey was administered by mail between December 1998 and March 1999. Of the 712 subjects, 478 completed a questionnaire, yielding an overall response rate of 67.1% (64.8% for SRAs and 67.8% for department chairs).

The Health and Robustness of the Clinical and Nonclinical Research Enterprises. Respondents were asked to rate the overall health and robustness of the clinical research in their department (for chairs) or institution (for SRAs). Rather than supply a definition of health and robustness we allowed the respondent to individually define these concepts, a procedure frequently used in survey research. The response categories were "excellent," "good," "fair," and "poor." An identical question was asked with respect to nonclinical research. For each question the responses were collapsed into dichotomous variables with "excellent" and "good" coded as 1 (interpreted as healthy and robust) and "fair" and "poor" coded as 0 (interpreted as not healthy and not robust).

Quality of Research. Respondents were asked to rate the quality, on average, of research performed in 5 areas: phase 1 and 2 clinical trials; phase 3 clinical trials; translational research (research that explores basic biological questions and uses humans as research subjects, often conducted in General Clinical Research Centers); epidemiological, health services, and outcomes research; and nonclinical research (ie, basic research). Response categories were provided as "very high quality," "high quality," "adequate quality," "poor quality," and "none performed." The responses to each question were collapsed into dichotomous variables with "very high quality" and "high quality" coded as 1 (interpreted as high-quality research) and "adequate" and "poor" responses coded as 0 (interpreted as less than high-quality research). Respondents who checked "none performed" were coded as missing.

Challenges to Performing Research. Participants were asked to rate the extent to which each of the following were problematic for clinical and nonclinical research: "the availability of external research support," "recruiting trained researchers," "finding research subjects," "pressure on clinical faculty to see more patients," "insufficient clinical revenues," "competition from CROs," and "the institutional review board (IRB) process." The response categories were "no problem," "small problem," "moderate problem," and "large problem." Again, the responses to each question were transformed into dichotomous variables with "moderate problem" and "large problem" coded as 1 (interpreted as a significant problem) with the "no problem" and "small problem" responses coded as 0 (interpreted as not a significant problem).

Urgency of Responding to Challenges. Respondents were asked: "Please rate your overall sense of urgency as you respond to the challenges facing the conduct of clinical research in your department or institution." The response categories were "extremely urgent—we are in a crisis with respect to clinical research," "urgent—delays in facing the challenges will be costly to clinical research," "not very urgent—normal management will address our clinical research issues,"or "not at all urgent—there are no important challenges to clinical research in my department or institution."

Strategic Adaptations. Respondents were requested to indicate whether their department (for chairs) or institution (for SRAs) had formal procedures or mechanisms in place (yes/no) to assist investigators with 6 research-related tasks: (1) identifying funding sources; (2) identifying potential collaborators; (3) writing grants; (4) reviewing grants prior to submission; (5) revising grants after a rejection; and (6) recruiting research subjects. For each "yes" response, we asked, "What has been the effect on the amount of clinical research performed?" The response categories were "no positive effect," "small positive effect," "moderate positive effect," and "large positive effect." Again, the responses to each question were transformed into dichotomous variables with "moderate positive effect" and "large positive effect" coded as 1 (interpreted as a significant positive effect), and the "no positive effect" and "small positive effect" responses coded as 0 (interpreted as not a significant positive effect).

Medical School Characteristics. The research intensity of each school was measured based on the amount of NIH research funding awarded to each medical school in 1999. The medical school that received the most funding was designated as 1 and the medical school with the least funding was designated as 122.¹¹ Similar to our previous work, each medical school was assigned to 1 of 2 categories: high research intensity (the 50 institutions that received the most money from the NIH) or low research intensity (institutions ranked 51 or higher).⁶

The competitiveness of the local market was indicated by the level of managed care penetration in 1998, obtained from InterStudy.¹² Each medical school was matched with the nearest metropolitan statistical area for which information about the managed care penetration was available. Respondents that could not be matched to a metropolitan statistical area were coded as missing for this analysis. Similar to previous work, the managed care penetration of each medical school was transformed into a categorical variable with 3 groups: low (1%-20%), medium (21%-40%), and high (>40%).⁷

All analyses were conducted using SAS (version 6.12, SAS Institute Inc, Cary, NC). Differences in simple proportions were tested using the χ² test. Multivariate analyses were performed using logistic regression controlling for type of respondent (SRA vs department chairs), research intensity (high or low), and level of managed care penetration (high, medium, or low). The regression formula was logit (p_i) = β₀+ β₁X₁+ β₂X₂ + β₃X₃, where p_i is the probability of an event, β₀ through β₃ are regression coefficients for the independent variables X₁ through X₃. Unless otherwise specified, regression-adjusted percentages are presented using the logistic model specified above.

Initial bivariate analyses suggested a possible interaction between research intensity and managed care penetration. Therefore, numerous multiple regression models were performed for each dependent variable with separate interaction terms for each combination of the research intensity and managed care variables (high managed care and low research intensity; high managed care and high research intensity; etc). However, in only 1 instance was the interaction significant in multivariate analyses, and for that case results are presented separately for high and low research-intensive institutions.

Also, a number of sensitivity analyses were conducted. We analyzed the primary dependent variables as ordinal in nature and compared differences in means; we included an additional variable in the multiple regression analyses for department specialty; and we tested alternative dichotomization approaches. The overall direction and significance of these analyses were similar to those presented and are not included here.

Of the respondents, 92% were male, 89% were physicians, and 92% had experience conducting clinical research (Table 1). Department chairs were more likely than SRAs to be male (94% vs 82%), physicians (99% vs 39%), and have conducted clinical research (98% vs 64%). Forty-five percent of respondents were employed in high research-intensive medical schools and 55% in low research intensive schools. Of the respondents, 14% were located in low managed care markets, 55% in medium penetrated markets, and 31% in high managed care markets.

Table 1 also compares the distribution of the respondents with the nonrespondents in terms of their research intensity and the level of managed care penetration. On both measures no significant differences were found between respondents and nonrespondents (P = .71 for research intensity and P = .98 for managed care penetration).

Slightly more than half (52%) of all respondents rated the health and robustness of their clinical research enterprise as good or excellent vs 63% for the nonclinical research enterprise (P<.001) (Table 2). SRAs were more likely than department chairs to rate the health and robustness of the nonclinical research enterprise as good or excellent (85% vs 55%, P<.001).

Perceptions of the health and robustness of the clinical research enterprise varied by the research intensity of the institution. At high research-intensive schools, 72% of respondents reported that the overall health and robustness of their clinical research enterprise was good or excellent compared with 35% in low research-intensive schools (P<.001). A similar trend occurred for nonclinical research, although the percentages reporting good or excellent were slightly higher in each category.

Perceptions of the health and robustness of the clinical and nonclinical research enterprise also differed significantly by the level of managed care penetration. Slightly more than a third (37%) of respondents in the low managed care markets felt their clinical research enterprise was in good or excellent health compared with slightly more than half (53%) of those in medium managed care markets (P<.049) and 58% of those in high managed care markets (P = .01). A similar trend was shown for nonclinical research, but no statistical significance was reached.

As shown in Table 3, respondents were most likely to rate nonclinical research as high in quality (79%) compared with 70% for phase 3 clinical trials, 67% for translational research, 65% for phase 1 and 2 trials, and 57% for health services research (all comparisons with nonclinical research significant at P<.001).

For each type of research, respondents in high research-intensive schools were more likely to feel that the research conducted in their department or institution was of high or very high quality compared with respondents in less research-intensive schools. For example, 87% of the respondents in high research-intensive schools rated the quality of their nonclinical research as high or very high compared with 65% of those in low research-intensive schools (P<.001). Virtually identical patterns of response were shown for phase 1 and 2 clinical trials (73% vs 56%, P = .002), phase 3 clinical trials (77% vs 63%, P = .002), translational research (76% vs 54%, P<.001), and health services research (68% vs 48%, P<.001). There were no significant differences by managed care penetration. SRAs and department chairs differed somewhat in their ratings, with department chairs significantly more positive about phase 3 clinical trials and SRAs more positive about nonclinical research and health services research.

Respondents reported an overall sense of urgency with respect to the challenges facing clinical research. Twelve percent felt that the situation was "extremely urgent—we are in a crisis with respect to clinical research"; 69% thought it was "urgent—delays in facing challenges will be costly to clinical research"; and 17% "not very urgent—normal management will address our clinical research issues." Table 4 shows respondents' views of the seriousness of the challenges research leaders face. Pressure on clinical faculty to see patients was perceived by the largest percentage of respondents as a moderate to large problem for clinical research (93%) followed by insufficient clinical revenues (89%), recruiting trained researchers (75%), a lack of external support for clinical research (72%), competition from CROs (48%), problems introduced by the IRB process (38%), and finding research subjects (37%). These figures are all 13 to 25 percentage points higher for clinical research than the comparable figures for nonclinical research (P<.001 for all).

The problems of availability of external research support and the ability to recruit trained researchers were most acute in the low research-intensive institutions. For example, 81% in low research-intensive schools felt recruiting trained clinical researchers was a moderate to large problem compared with 66% in the most research-intensive schools (P<.001).

The level of managed care penetration was associated with increased pressures to see patients, insufficient clinical revenues, and competition from CROs. In high managed care markets, 97% reported that pressure to see patients was a moderate or large problem for clinical research compared with 90% in low managed care markets (P = .03). Similar results were found for insufficient clinical revenues (93% high managed care vs 80% low managed care, P = .01). However, a significantly smaller percentage (66%) of respondents in high managed care markets felt the availability of external research funding was a moderate or large problem compared with 86% in low managed care markets (P = .006).

A significant interaction was found between research intensity, managed care penetration, and perceptions of the seriousness of CRO competition. Among the respondents in low research-intensive schools in low managed care markets, 29% felt competition from CROs was a moderate to large problem compared with 50% in medium managed care markets and 56% in high managed care markets (P<.001). There were no significant differences by level of managed care penetration for respondents in high research-intensive institutions.

A number of potential strategic adaptations available to medical schools to address the challenges facing the clinical research enterprise were addressed. Sixty-nine percent of respondents had formal processes for assisting faculty in identifying funding sources: 48% for identifying potential collaborators, 45% for writing grants, 45% for reviewing grants prior to submission, 34% for reviewing a grant after a rejection, and 34% for recruiting research subjects.

To measure the impact of these adaptations, respondents who had engaged in each of the above activities were asked to estimate the effect of the adaptation on the amount of clinical research conducted in their department or institution. Among those with a formalized process for assisting investigators to recruit research subjects, more than half (53%) reported that this innovation had a moderate to large effect on the amount of clinical research conducted. Similar results were found for procedures that assist investigators by reviewing grants prior to submission (47%) and revising a grant following a rejection (48%). The innovations that were reported least likely to have a moderate to large impact on the amount of research conducted were identifying research funding (30%), identifying potential collaborators (40%), and writing grants (43%).

We surveyed the research leadership in US medical schools and found substantial concern regarding the clinical research mission. Almost half (48%) of all respondents did not consider their clinical research enterprise to be healthy or robust. This finding is consistent with numerous reports citing a broad-based concern regarding the present status of the clinical research mission of AHCs.^{1 - 3}

A similar concern existed regarding the quality of clinical research. Our respondents, especially SRAs, believe that considerable amounts of clinical research do not meet their standards. One potential explanation for this finding is that less than two thirds of SRAs had conducted clinical research and thus may be less likely to consider such research as high quality. It may also be that the expectations of respondents regarding clinical research are unrealistically high given that clinical research as a discipline is younger than many of the basic science disciplines or that our survey question regarding the research quality may not adequately measure all aspects of the current research environment. Regardless of the explanation, clinical research is seen by our respondents as lower in quality than nonclinical research.

It is clear that several challenges to the clinical research mission of AHCs exist. First, 9 out of 10 research leaders thought that pressures to see patients and insufficient clinical revenues posed a moderate to large problem for clinical research, and these perceptions were greatest among those located in areas with high managed care penetration. These findings are consistent with previous research, which showed that clinical researchers in the most competitive markets have greater clinical care responsibilities⁵ and are less likely to receive internal support from their institution.⁶ These findings provide further evidence of the negative impact of market pressures on the clinical research mission of AHCs. Second, our respondents perceived an inadequate supply of trained clinical researchers. Seventy-five percent of respondents reported a moderate to large problem recruiting trained clinical researchers. Even among the most research-intensive institutions, two thirds reported difficulties recruiting clinical researchers. This finding may be the result of an underproduction of trained clinical researchers or an inability of medical schools to assemble competitive compensation packages sufficient to compete with nonacademic CROs or other job opportunities. Regardless of the explanation, this study supports the widespread consensus that demand for trained clinical researchers in academia may exceed the current supply.^{1 ,9 ,13}

The data on strategic adaptations leaders have made provide insight into potential management strategies to cope with these and other challenges. Despite a clear sense of urgency, less than half of respondents said that their department or institution had implemented various strategic policies or mechanisms. Even among those with such strategies in place, confidence in their impact was less than overwhelming. With the exception of helping recruit research subjects (53%), in no case did a majority feel their strategies had a moderate or large positive impact on the amount of clinical research performed. In fact, the most frequently used mechanisms, such as helping investigators identify funding sources, had the smallest positive impact on the amount of research conducted while the least used (recruiting research subjects) had the greatest impact.

However, concern regarding the clinical research enterprise may be tempered by some of our findings. First, given that the high research-intensive institutions receive approximately 80% of NIH funding,¹ it is somewhat encouraging that almost three quarters of those institutions rate the overall health and robustness of their clinical research enterprise as good or excellent despite the challenges discussed above. Second, despite greater pressures to see patients and insufficient clinical revenues, research leaders in high managed care areas were more likely to rate the health and robustness of their clinical research enterprise as good or excellent than those in low managed care markets. The explanation for this finding is not obvious. It is possible that other factors that occur in conjunction with high managed care markets, such as the presence of life science companies, provide resources that buttress the clinical research mission. Further studies should address this issue.

Our study has several limitations. First, we do not know the extent to which our respondents' perceptions adequately reflect reality. As a result, these findings should be seen as suggestive of topics and research questions that deserve future study. Second, our results are based on cross-sectional data. Some of the effects we observed in the most competitive markets or in the most research-intensive medical schools may be the result of adaptive behaviors or lack thereof brought on by historical, internal, or external stimuli instead of the static pressures occurring as a result of market competition or large amounts of funding from the NIH. Third, we did not ask about all the possible strategic adaptations that could be used by institutions. For example, one strategy we did not explore is developing training programs to better prepare clinical investigators with the tools for high-quality, patient-centered research and to provide a greater supply of young clinical researchers. Fourth, because of respondent confidentiality, we are unable to link individuals with their institutions. As a result, it is possible that respondents' perceptions of their research enterprise were affected by similar issues affecting their institution at large.⁴

Nevertheless, this research provides the first systematic, national data on how authoritative observers view the health of the clinical research enterprise compared with nonclinical research, and the prevalence and effectiveness of strategies to improve clinical research. As such, our study supports the frequently voiced concerns about the problems of clinical research, suggests that strategies that deal with these problems are underdeveloped, and that capacity opportunities may exist.