Reporting of Conflicts of Interest in Meta-analyses of Trials of Pharmacological Treatments FREE

Conflicts of interest (COIs) related to the funding of biomedical research by pharmaceutical companies and financial relationships between researchers and pharmaceutical companies have come under increased scrutiny in recent years.^1- 3 COIs may influence the framing of research questions, study design, data analysis, interpretation of findings, whether to publish results, and what results are reported.^4- 5 Results from positive trials and from favorable analyses are more likely to be published than results unfavorable to sponsors.^6- 8 Compared with nonindustry-funded trials, pharmaceutical industry–funded studies more often yield results or conclusions in support of the sponsor's drug,^9- 16 and authors' relationships with drug manufacturers have been linked to favorable assessments of drug efficacy and safety.^17- 20

As a result, increased emphasis has been placed on the transparent disclosure of COI. The Consolidated Standards of Reporting Trials (CONSORT) guidelines require disclosure of study funding sources in trial reports.^21- 22 International Committee of Medical Journal Editors (ICMJE) guidelines recommend disclosure by authors of study funding sources and also of author-industry financial ties.²³ There are no guidelines, however, for the reporting in meta-analyses of COIs disclosed in included randomized controlled trials (RCTs). The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement requires meta-analysis authors to report the funding source of a meta-analysis, but does not address the reporting of COI from included RCTs.^24- 25 Meta-analyses are cited more than any other study design²⁶ and prioritized in grading evidence for practice guidelines.²⁷ Meta-analyses supported by the pharmaceutical industry, through study funding or author-industry financial ties, more often reach conclusions that favor sponsors' interests than meta-analyses not linked to industry.^28- 30 Without documentation in meta-analyses of COIs from included RCTs, users of meta-analyses may not have access to important information that could influence their evaluation of the risk of bias in the evidence reported.

The objective of this study was to investigate the extent to which pharmaceutical industry funding and author-industry financial ties or author employment disclosed in published reports of RCTs of pharmacological interventions are transparently reported in meta-analyses published in high-impact general and specialty medicine journals and the Cochrane Database of Systematic Reviews. We hypothesized that few meta-analyses would report COIs disclosed in original reports of included RCTs.

We selected a sample of meta-analyses published from January 2009 through October 2009 in 3 categories of high-impact publications: (1) general medicine journals, (2) journals representing the top 5 specialty medicine areas based on 2008 global pharmaceutical sales (oncology, cardiology, respiratory medicine, endocrinology, and gastroenterology),³¹ and (3) the Cochrane Database of Systematic Reviews. We prioritized recently published meta-analyses to reflect current reporting practices because standards are evolving.^23- 24

Within general medicine journals, we selected the 3 most recently published eligible meta-analyses from each journal with a 2008 impact factor of at least 10 (New England Journal of Medicine, JAMA, Lancet, BMJ, Annals of Internal Medicine, PLoS Medicine),³² with fewer included if there were not 3 that met eligibility criteria. Within each specialty medicine area, we also identified 3 recently published meta-analyses. We started with the most recently published meta-analyses in the top impact factor journal³² in each specialty area, then searched the second highest-rated journal if 3 eligible meta-analyses were not published in the top journal, and continued to search journals in declining order of impact factor until 3 eligible meta-analyses were obtained.

To obtain our sample, we searched the MEDLINE database via PubMed using limits of article type (meta-analysis) with journal names, supplemented by a manual search of each journal's table of contents for the term meta-analysis in article titles or abstracts. For articles published in the same journal issue, the article with the highest page number was considered most recent. Articles published online ahead of print as of October 31, 2009, were not eligible. In addition, meta-analyses from the most recent Cochrane Database of Systematic Reviews issue (issue 4, 2009) were selected for review based on random numbers generated in Microsoft Excel until 3 eligible meta-analyses were obtained.

Eligible meta-analyses (1) included a documented systematic review of the literature, (2) statistically combined results from at least 2 RCTs, (3) did not include non-RCTs, (4) evaluated the efficacy or harm of a drug or class of drug against an alternative treatment (eg, placebo, alternative drug), and (5) included at least 1 drug in the intervention or comparison study groups that was under patent in the United States at the time of publication based on the electronic US Food and Drug Administration Orange Book.³³ A drug was classified as under patent if any aspect of the active ingredient (eg, dosage, route, strength) was protected by an unexpired patent. We selected meta-analyses with at least 1 drug under patent in order to restrict the sample to drugs of potentially high economic importance to pharmaceutical companies. Meta-analyses that investigated biologics or that investigated a combination of pharmacological and nonpharmacological interventions (eg, psychotherapy) were included if a drug intervention alone was assessed as a study group.

If either of 2 reviewers independently deemed a retrieved meta-analysis to be potentially eligible based on title and abstract review, then a full-text review was conducted. Full-text reviews were conducted independently by 2 reviewers, 1 meta-analysis at a time in reverse temporal sequence until 3 eligible articles were obtained from each general medicine journal, each specialty medicine area, and the Cochrane Database of Systematic Reviews. Chance-corrected agreement between reviewers was assessed with the Cohen κ statistic with any disagreements resolved by consensus. Translators assisted reviewers to evaluate non-English titles, abstracts and articles and in data extraction.

Two investigators independently reviewed all meta-analyses and included RCTs, including disclosure statements, article texts and tables, author bylines and acknowledgments, and all online journal supplements (see eAppendix for data extraction forms) to identify (1) disclosure of COIs (study funding, author-industry financial ties or employment) from each selected meta-analysis; (2) disclosure of COIs for all RCTs included in each meta-analysis; and (3) to determine whether or not disclosed COIs from included RCTs were reported in meta-analyses. For included RCTs published only as abstracts, we verified whether a separate disclosure section was published and extracted data, as appropriate. For meta-analyses, we also determined whether a quality or risk of bias assessment of included RCTs was conducted and, if so, the instrument used.

Study funding sources for meta-analyses and included RCTs were classified as pharmaceutical industry, nonindustry (eg, public granting agency, private not-for-profit granting agency), combined pharmaceutical industry and nonindustry, nonindustry with drug supplied by pharmaceutical industry (RCTs only), no study funding, or not reported. Studies reported as funded “in part” by the pharmaceutical industry with no other indication of funding source were coded as industry-funded. Study funding included provision of financial support, resources (eg, statistical analyses), or inclusion of study personnel beyond those listed as authors.

Author financial ties to industry were defined per the October 2009 version of the ICMJE Uniform Disclosure Form for Potential Conflicts of Interest²³ and included current or former board membership, current or former consultancy, former industry employment, equity holdings (eg, stock, stock options), expert testimony, gifts, patents (planned, pending, or issued), payment for manuscript preparation, other research funding, royalties, speaker fees/payment for presentation development, travel reimbursement, or unspecified honoraria, as disclosed in the article. If an article did not contain a disclosure statement or acknowledgments, author-industry financial ties were coded as not reported. Authorship by individuals employed by the pharmaceutical industry at the time of article publication was coded separately as industry employment.

If a meta-analysis included citations to multiple articles reporting on the same RCT, each article was reviewed and COIs were coded as present if reported in any of the cited articles. For meta-analyses that included RCTs of both pharmacological and nonpharmacological interventions, only RCTs that assessed a pharmacological intervention alone as a study group were reviewed. Any discrepancies in data extraction were resolved by consensus.

Corresponding authors of meta-analyses were contacted via e-mail (as many as 3 attempts) to determine whether data extraction protocols included study funding and author-industry financial ties.

A total of 133 potentially eligible titles/abstracts were reviewed, including 52 from general medicine journals, 70 from specialty medicine journals, and 11 from the Cochrane Database of Systematic Reviews. Of these, 93 were excluded after title/abstract review and 11 after full-text review, leaving 29 eligible meta-analyses that were included in the review (eTable 1).^34- 62 The 29 meta-analyses included 11 from general medicine journals (3 each from JAMA, Lancet, and BMJ; 2 from Annals of Internal Medicine ; 0 from New England Journal of Medicine or PLoS Medicine), 15 from specialty medicine journals, and 3 from the Cochrane Database of Systematic Reviews. Impact factors of journals with included meta-analyses ranged from 12.2 to 31.7 in general medicine, 13.3 to 17.2 in oncology, 8.9 to 14.6 in cardiology, 5.2 to 5.5 in respiratory medicine, 6.4 to 7.3 in endocrinology, 7.4 to 9.8 in gastroenterology, and 5.2 for the Cochrane Database of Systematic Reviews. Cohen κ for chance-corrected agreement on inclusion/exclusion decisions was 0.94.

As shown in Table 1 and Table 2, the 29 selected meta-analyses evaluated a broad spectrum of pharmacological interventions, including 21 on treatment efficacy, 3 on harms, and 5 on both efficacy and harms. Between 2 and 65 RCTs were included in each meta-analysis.

As shown in Table 1 and Table 2, 0 of the 29 selected meta-analyses reported being funded by the pharmaceutical industry. Fourteen (48.3%) reported nonindustry funding, 4 reported no study funding (13.8%), and the funding source of 11 (37.9%) was not reported. At least 1 author of 16 of the 29 meta-analyses (55.2%) reported at least 1 financial tie to the pharmaceutical industry, all of the authors of 12 of the meta-analyses (41.4%) reported 0 financial ties to the pharmaceutical industry, and author financial ties were not reported in 1 meta-analysis (3.4%). Specific types of author ties to the pharmaceutical industry for each meta-analysis are shown in eTable 2. Only 1 of the 29 meta-analyses listed authors employed by the pharmaceutical industry.³⁷

The 29 selected meta-analyses synthesized data from a total of 509 RCTs. As shown in Table 3, 62.5% (318 of 509) of included RCTs reported funding source. Of these, 68.9% (219 of 318) were funded in part or whole by the pharmaceutical industry; 30.5% (97 of 318) by nonindustry funding sources, including 28 RCTs in which a study drug was supplied by the pharmaceutical industry; and less than 1% (2 of 318) reported that the trial received no funding. Characteristics of the 509 included RCTs, including COI data, are presented in eTable 3.

Author financial disclosures were reported in only 25.9% (132 of 509) of included RCTs. Among these, 68.9% (91 of 132) reported 1 or more authors having financial ties to the pharmaceutical industry. Author affiliations were reported in 94.7% of included RCTs (482 of 509), including 26.1% (126 of 482) with at least 1 author employed by the pharmaceutical industry.

As shown in Table 3, only 2 of the 29 selected meta-analyses reported the funding source of included RCTs.^47,62 One listed RCT funding sources in a table footnote⁴⁷ and the other in the Characteristics of Studies table that followed the main document and references.⁶² Neither mentioned RCT funding sources in the column of a core table, in the text, or in an assessment of potential bias. Both of these meta-analyses^47,62 reported nonindustry funding for the meta-analysis. One of the meta-analyses reported no author ties to the pharmaceutical industry,⁶² whereas the other reported that 1 of 3 authors had a link to the pharmaceutical industry.⁴⁷ None of the 29 meta-analyses reported author-industry financial ties or employment of included RCTs.

Of the 29 meta-analyses, 25 assessed quality or risk of bias in included RCTs. One of the meta-analyses that reported the funding source of included RCTs⁴⁷ used an ad hoc method to assess study quality that did not include an assessment of study funding. Five meta-analyses^{34,44,60- 62} used at least 3 of the 6 domains from the Cochrane Risk of Bias tool, which does not produce a single quality score, but rather provides ratings for individual risk components.⁶³ Only 1 of the 5 meta-analyses⁶² reported the funding source of included RCTs, but it did not include this information in the assessment of risk of bias.

In 7 of 29 meta-analyses (Table 3), 100% of included RCTs disclosed at least 1 form of COI in the original RCT publications.^{37- 38,40,44,47,54,60} In 4 of these 7 meta-analyses,^37,44,54,60 100% of included RCTs that reported study funding were funded by the pharmaceutical industry. Only 1 of the 7 meta-analyses, however, provided information on study funding of included RCTs,⁴⁷ and that was done in a table footnote.

Twenty-seven of 29 meta-analysis authors provided information on data extraction protocols. Two recorded and reported RCT funding sources^47,62; 5 recorded, but did not report funding sources; and 20 did not record funding sources. Only 2 of the 27 meta-analyses recorded RCT author-industry financial ties, but neither published this information.

The main finding of this study is that with few exceptions, information on COI disclosed in RCTs is not reported when RCT data are combined in meta-analyses. Pharmaceutical industry funding was present in 69% of the RCTs that disclosed funding. However, only 2 of 29 meta-analyses provided information on funding sources of included RCTs. None reported author-industry financial ties or employment disclosed in the original RCT publications. The 2 meta-analyses that reported RCT funding sources provided this information in a table that followed the main document and references⁶² and in a table footnote,⁴⁷ neither of which are typically reviewed by the average reader. Neither meta-analysis described sources of COIs in a core table, in the text, or in an analysis of potential sources of bias.

There is general agreement on the need for complete and transparent disclosure of COI in biomedical research.^21- 23 The results of the present study highlight a major gap in the reporting of COIs and suggest that, without a formal reporting policy, COIs from RCTs are unlikely to be reported when results are synthesized in meta-analyses. The PRISMA statement should be updated to require authors of meta-analyses to report funding sources of included RCTs or report that funding sources were not disclosed. Study funding was disclosed in the original reports of approximately two-thirds of RCTs in this review.

In addition to information on study funding, consumers of research, including patients and physicians, want researcher financial ties to industry to be disclosed⁶⁴ and consider author-industry financial ties in assessing the quality of research evidence.^64- 66 The authors of the PRISMA statement should also consider recommending that meta-analyses report author-industry financial ties disclosed in included RCTs or report that there was no disclosure statement. Although author-industry ties are less frequently reported than study funding sources in published reports of RCTs included in meta-analyses, the proportion of RCTs reporting author-industry financial ties will likely increase with the recent introduction of ICMJE disclosure guidelines. The nature and extent of author-industry ties disclosed in RCTs are complex. However, coding the proportion of authors with disclosed industry financial ties would not be a great burden and would flag studies with COIs from author-industry financial ties for interested readers.

Authors of meta-analyses are expected to transparently assess and interpret potential sources of bias from included studies that could influence outcomes. Items included in quality or risk of bias assessment tools (eg, sequence generation, blinding) are increasingly selected based on empirical evidence of an association with bias, including the mechanism, direction, and likely magnitude of possible bias.^63,67 Meta-analysis authors should document that they have evaluated all potentially relevant sources of bias, whether or not a particular source of possible bias is present in the studies reviewed and whether or not the magnitude of bias is expected to be large in comparison with other likely sources of bias. Risk of bias ratings for various domains are used by meta-analysts as stratification factors in sensitivity analyses or more qualitatively. For instance, meta-analysts might note that all RCTs in a given review have significant design shortcomings and a high risk of bias.

COIs from pharmaceutical industry study funding and author-industry financial ties meet the empirical criteria typically used to select other potential sources of bias for inclusion in evidence quality and risk of bias assessment tools. Pharmaceutical industry funding and author-industry financial ties are associated with a bias toward favorable results even when controlling for other study characteristics.^12- 13 Based on empirical evidence of bias related to COI, an Agency for Healthcare Research and Quality systematic review of tools used to rate evidence quality⁶⁷ included the category funding or sponsorship as a key evaluation domain and rated tools higher if they included an assessment of potential bias from industry sponsorship. Similarly, the recently developed Assessment of Multiple Systematic Reviews tool for grading the methodological quality of systematic reviews includes a score for whether COIs from included studies are clearly described.⁶⁸

Meta-analysts should evaluate the potential for bias due to pharmaceutical industry study funding and author-industry financial ties as part of their standard risk of bias assessment. As with other potential sources of bias, results from this domain should be transparently documented and used in sensitivity analyses or qualitatively. For instance, a set of positive results from industry-funded trials would likely be interpreted with more confidence if corroborated by at least 1 study with similar findings that was not industry-funded. The funding source and thus the risk of bias due to industry funding will be unclear for some studies, such as those conducted prior to the adoption of guidelines for declaring this type of information.

Although some risk of bias assessment tools include a domain for study funding source, most do not.⁶⁷ Currently, the Cochrane Collaboration's Risk of Bias tool includes an optional “other sources of bias” domain,⁶³ which meta-analysts could use to include information on COIs. We recommend that the Cochrane Collaboration consider formalizing the requirement to assess potential bias from COIs.

Several limitations should be considered in interpreting results from this study. First, the study was not designed to assess whether reporting of COI from RCTs included in meta-analyses was related to the quality of meta-analyses or whether COIs from included RCTs influenced the results of the meta-analyses reviewed. However, the motivation to conduct this study was based on extensive research that has shown that COIs can influence the results and conclusions of both RCTs and meta-analyses.^{11,13- 14,28- 30} Second, we reviewed a relatively small sample of meta-analyses from high-impact journals, and it is not clear to what degree these results can be generalized to other areas of medicine or to lower-impact journals. None of the 29 meta-analyses reviewed reported funding from the pharmaceutical industry, for instance, and it is possible that we undersampled industry-funded meta-analyses. However, given that only 2 of 29 (7%) meta-analyses mentioned COIs from included RCTs, it is likely that the main findings of the study are generalizable. Finally, the exact nature of COIs in included RCTs (eg, amount of industry funding, role of funding source) was not assessed. However, this information is not typically available and studies that have identified links between pharmaceutical industry funding and study outcomes have similarly relied upon dichotomous coding.^{10,12,15- 16}

In summary, this study found that meta-analyses of pharmacological interventions published in high-impact medical journals rarely reported the funding sources or author-industry financial ties of included RCTs, even when these sources of COIs were disclosed in RCT reports. PRISMA should require the reporting of study funding sources and author-industry financial ties of RCTs in meta-analyses, and this information should be included in risk of bias assessments.

Risk of bias assessment in meta-analyses due to COI or other sources of potential bias in included RCTs is imperfect, and disclosure is a necessary first step, but not sufficient to mitigate the effects of COI on biomedical research.⁶⁹ Nonetheless, if COIs disclosed by authors of RCTs are not reported when RCTs are synthesized in meta-analyses, efforts to achieve greater transparency in biomedical research through disclosure requirements may be less effective. Given that COIs are prominent in biomedical research and have been empirically linked to bias, it is the responsibility of authors of meta-analyses to transparently document for readers their efforts to evaluate the likely influence of COI on meta-analysis outcomes.

If COIs from included RCTs are not acknowledged in meta-analyses, 1 of 2 messages may be sent to readers—the first is that the authors of the meta-analysis have not considered COI in included studies, which leaves readers in a position of not knowing how to interpret possible biases that may have arisen because of COI in the original RCTs; and the second possible message is that the meta-analysts have in fact assessed the risk of bias related to COI in the original RCTs, concluded that the COIs did not create any biases, and therefore have chosen not to comment on the COIs. This interpretation may lead readers to trust the conclusions of a meta-analysis when they potentially should not. In either case, without acknowledgment of COI due to industry funding or author-industry financial ties from RCTs included in meta-analyses, readers' understanding and appraisal of the evidence from the meta-analysis may be compromised.