The Need for Improved Access to FDA Reviews

The medical literature is expanding rapidly, with thousands of new studies published each year. Increasingly, clinicians and scientists rely on rigorous syntheses of the scientific literature, such as high-quality systematic reviews and meta-analyses, which the Oxford Centre for Evidence-Based Medicine considers the highest level of scientific evidence available.¹ However, systematic reviews and meta-analyses are typically based on the published literature, which is known to be biased,^{2 - 4} creating an important question: can we trust the highest level of evidence available in MEDLINE?

Unlike clinicians, academic investigators, and the public, the US Food and Drug Administration (FDA) and other regulatory agencies do not rely on the published literature as the primary source of data for making regulatory decisions. Instead, companies seeking drug or device approval must submit the raw data from pivotal randomized clinical trials (RCTs) and summaries of other relevant studies they have conducted to the FDA for independent review. Comparisons with such regulatory reviews have found that trials with unfavorable results are less likely to be published than favorable trials,^{2 - 3} and published methods and outcomes sometimes differ from those prespecified in trial protocols.^{2 - 3} A recent study used the FDA reviews for 12 antidepressants to determine that the published literature overestimated the effect sizes for the antidepressants by a median of 32%, with a range of 11% to 69% for the different drugs.²

FDA reviews and reviews generated by other regulatory agencies should be more easily accessible. Linking regulatory reviews to MEDLINE would increase awareness of their existence and facilitate more comprehensive and accurate systematic reviews of clinical research.

Considerable progress toward reducing the effects of publication bias has been made. To be eligible for publication, many journals now require that clinical trials were publicly registered prior to participant enrollment. In addition, the FDA Amendments Act of 2007 requires that all clinical trials (except phase 1 trials) involving FDA-regulated drugs and biologics be registered in a publicly accessible registry.^{5 - 6} The law will also require the registration of trial results within 1 year of trial completion.

Trial registration enables accounting for clinical trials, whether published or not. Registration of trial methods allows peer reviewers and editors to compare the primary outcome(s) and the methods of analysis described in the registration (prior to data collection) with the submitted manuscripts to ensure that outcome reporting was not altered after data analysis. In this way, trial registries complement the published literature, allowing a more complete picture of both published and unpublished studies. Whether these steps reduce bias in the published literature remains to be seen.

There are many challenges to ensuring accurate and complete trial registration.⁶ As currently envisioned, registration of trial results will be subject to considerably less review than publication in the peer-reviewed literature,^{5 - 6} which is a major problem. Clinical trial statistical analyses can be complicated, and the peer-review process has not consistently rooted out biased statistical analyses. Recent episodes of intentional misrepresentation of clinical research in the peer-reviewed literature⁴ raise concern about the reliability of the unreviewed trial results that will appear in registries.

Beyond the complete and accurate reporting of relevant RCTs in an FDA review, comparing FDA statisticians' analysis of a trial's primary outcome with the sponsor's intention-to-treat analysis for the same outcome can be informative. For example, in the Summary Basis of Approval for pregabalin (Lyrica; Pfizer Inc, New York, New York) for pain associated with diabetic peripheral neuropathy, FDA statisticians disagreed with the sponsor's statistical analysis plans. The sponsor defined the primary end point for each patient as the average of the last 7 available pain scores while the patient was receiving medication and used a last-observation-carried-forward (LOCF) approach for handling missing data. The FDA review states that this analytic approach with this end-point definition “overestimates the benefit of the drug” because “patients achieving adequate symptom control but experiencing intolerable side effects often terminate the study with ‘good’ pain scores which are carried forward in the LOCF analysis,” even though they “are true treatment failures because they were unable to tolerate the dose necessary to achieve symptom control.”⁷ For these reasons, “the Agency prospectively expressed a primary interest in an analysis which compared change from baseline in mean pain scores using a baseline-observation-carried-forward (BOCF) imputation strategy.” The FDA statisticians recalculated the efficacy analyses using the pain scores during the last week of the trial as the end point and the BOCF approach for handling missing data.⁷ Four of the 5 RCTs reviewed by the FDA were ultimately published, and all presented the sponsor's prespecified end-point definition and LOCF approach without acknowledging the limitations of this approach.^{8 - 11} Relative to the FDA analyses, the published analyses overestimate the proportion of patients achieving a 50% reduction in pain by 20% and 28% for the 300-mg/d and 600-mg/d dosages, respectively.¹²

FDA reviews of unpublished observational data also may be valuable. A recent supplemental application to the FDA sought to expand the indication for the fentanyl buccal tablet (Fentora; Cephalon Inc, Frazer, Pennsylvania) from opioid-tolerant patients with chronic cancer pain to the substantially larger population of opioid-tolerant patients with chronic noncancer pain. In consideration of this application, the FDA compiled data from a variety of sources, including pooled unpublished clinical trial data and surveillance data from the National Survey on Drug Use and Health and the Drug Abuse Warning Network. This FDA review found that the available data suggested “an excess risk of events related to overdose, addiction, and CNS [central nervous system] depression related to opioids in the non-cancer population” relative to the cancer population.¹³ Although the sponsor demonstrated efficacy in the target population, the FDA ultimately rejected the application, likely due to tolerability and abuse concerns in the broader noncancer population. This is a controversial area of the medical literature without much published research, and the FDA's compilation of various sources, including unpublished data unlikely to appear in trial registries, provides important information that would add substantially to the published literature.

Although many FDA reviews are publicly available, legal barriers to the public release of research data considered proprietary by sponsors prevent or delay access to some FDA review data.⁶ Compelling arguments have been made for legislative reform that facilitates public access to FDA reviews to improve public health and safety.⁶

However, many of the publicly accessible FDA reviews are challenging to find, which undoubtedly interferes with dissemination of their findings. For example, to find the pregabalin medical review (discussed above) within the FDA's Web site requires successful navigation of a total of 6 screens, the second of which is the Drugs@FDA site.¹⁴ The links within the site are poorly named. For example, the 3 “FDA applications” for pregabalin are listed by number but do not specify the indication, so the searcher will have to use trial and error to find the review pertaining to painful diabetic neuropathy.

The Drugs@FDA search engine will identify a drug by either generic or brand name, but attempts to increase the specificity of the search (eg, pregabalin review or Lyrica approval) yield a consistent “no results” page. The search window on the FDA home page does not work well. Searching pregabalin review or pregabalin summary approval, for example, yields 79 and 37 hits, respectively, but none of the hits is the review. The aforementioned Fentora review does not appear under any of the Drugs@FDA Fentora links. Even the application number fails to identify this review with the FDA.gov search engine, although this search strategy using Google identifies the review.

Once found, reviews are difficult to navigate. The pregabalin review document does not reveal the indication being reviewed until page 19 of 390. The first table of contents appears on page 17 and lists incorrect page numbers.⁷ Locating specific information within the PDF file is challenging and time-consuming. While these are regulatory documents not designed for public use, the relatively simple additions of accurate labeling, tables of contents, and hyperlinks would help readers locate information.

The FDA has made efforts to make some of the FDA review information available. At least a few medical reviews of chemotherapy drugs have been summarized in the peer-reviewed literature, although exactly how many reviews have been published is difficult to discern. In addition, labels of approved drugs contain summaries of clinical trials based on the analyses of FDA statisticians. However, package labels may selectively report pivotal clinical trials. For example, the pregabalin label states that “efficacy of the maximum recommended dose” of pregabalin (300 mg/d divided into 3 doses) for painful diabetic neuropathy “was established in three . . . studies . . . , two of which studied the maximum recommended dose.”¹⁵ The label briefly summarizes 2 trials that showed efficacy using the FDA-approved dose without describing the trial that showed efficacy at a higher dosage or, of greater concern, mentioning that 2 pivotal trials assessing higher dosages were negative.¹⁵

Improved access to and awareness of the publicly available reviews conducted by the FDA and other regulatory agencies and authorities, such as the European Medicines Agency and the National Institute for Health and Clinical Excellence, are needed. Individual RCTs and meta-analyses conducted by regulatory agencies could be indexed in MEDLINE and searched in tandem with the published medical literature. Search filters could be included to allow users to focus on or exclude unpublished regulatory analyses.

It seems likely that the costs of linking FDA reviews to MEDLINE and improving the FDA Web site would be small relative to the expenditures for the FDA's extensive evaluations. Failure to access this information likely generates substantial costs. Overestimations of treatment efficacy may result in distortions of meta-analyses, systematic reviews, clinical practice guidelines, cost-effectiveness estimates,¹² sample size calculations for clinical trials, and risk-vs-benefit estimates in individual patients.

FDA reviews do have limitations. In some cases sponsors may withhold certain data from the FDA.⁴ Moreover, many RCTs do not appear in FDA reviews, including those completed after FDA approval and those performed for off-label clinical indications for which formal approval will never be sought.

Bias in the medical literature should be of great concern to scientists, clinicians, and the general public, all of whom can be adversely affected by incomplete or inaccurate analyses of clinical research. Clinical trial registration should improve the accurate identification of clinical trials and their prespecified outcomes but seems unlikely to ensure unbiased statistical analyses.

Independent analyses and summaries by experts with complete access to all of the relevant data and no financial conflicts of interest are incredibly valuable. Efforts to promote transparency of clinical research are certainly worthwhile, but awareness of existing regulatory agency “literature” also must be improved. At the time they are performed, regulatory agency reviews are the most complete and accurate syntheses of clinical trial data available—it is time to make better use of them.