Evaluation of Conflict of Interest in Economic Analyses of New Drugs Used in Oncology FREE

Financial conflict of interest is a pressing issue for the medical research community.^1- 2 Physicians' economic ties to tobacco, alcohol, baby formula, and pharmaceutical companies have all been criticized as possible nonscientific influences on medical research.^3- 6 Recent studies of research on calcium channel antagonists in cardiology, nonsteroidal anti-inflammatory drugs for the treatment of arthritis, and the health effects of secondhand smoke all found that physicians with financial ties to manufacturers were significantly less likely to criticize the safety or efficacy of these agents.^7- 9 Similarly, a study of clinical trial publications determined that there was a significant association between positive results in general internal medicine clinical trials and funding from a pharmaceutical manufacturer.¹⁰

While the debate over financial conflict of interest has surrounded issues of clinical efficacy and safety, only 1 prior study has addressed concerns related to reports on cost-effectiveness.¹¹ In that study, Azimi and Welch¹¹ reported that industry-financed cost-effectiveness analyses were more likely to support additional expenditures with investigational drugs than standard treatments. To further examine the existing pharmacoeconomic literature, we evaluated cost studies for 3 recent breakthrough areas in oncology: hematopoietic colony-stimulating factors, serotonin antagonist antiemetics, and taxanes. Economic studies of these agents have reported varying assessments of costs and cost-effectiveness.^12- 17 This study was designed to determine whether the apparent financially motivated bias seen in clinical efficacy and safety evaluations is also evident in economic analyses in oncology.

The major objective of this study was to determine whether there was an association between pharmaceutical industry sponsorship and economic assessments of breakthrough oncology drugs. The following questions were addressed: were pharmaceutical company–funded economic studies more likely than nonprofit-funded studies to report favorable qualitative assessments and less likely to report unfavorable qualitative assessments? and were pharmaceutical company–sponsored studies more likely than nonprofit-funded studies to state qualitatively favorable conclusions despite neutral or unfavorable quantitative results?

Economic analyses of 6 recently marketed breakthrough cancer drugs in 3 categories were chosen. The agents included hematopoietic growth factors (granulocyte colony-stimulating factor [G-CSF] and granulocyte-macrophage colony-stimulating factor [GM-CSF]), serotonin antagonist antiemetics (ondansetron hydrochloride and granisetron), and taxane chemotherapy agents (paclitaxel and docetaxel). These drugs were chosen because their cost-effectiveness is controversial, and they account for a large fraction of total pharmaceutical expenditures in many hospital pharmacies. Clinical reports have demonstrated efficacy in specific settings, but high acquisition and administration costs have raised concern about the widespread use of these agents.

We searched the MEDLINE (1988-1998) and HealthSTAR (1988-1998) databases to identify original research articles that contained an economic analysis of 1 or more of the study drugs. The following terms were searched: cost(s), cost-effective(ness), economic(s), dollar(s), pharmacoeconomic(s), and cost-benefit. Drugs were searched under generic and brand names. Abstracts, letters, editorials, review articles, and non–English-language articles were excluded. Abstracts from the remaining articles were reviewed, and all articles including an actual analysis of costs were identified. This search yielded 54 articles, of which 8 were head-to-head comparisons between drugs in a given category (eg, G-CSF vs GM-CSF) and 46 were comparisons with placebo or standard treatment. Head-to-head comparisons were excluded because they could not be classified according to our criteria; ie, the results would always be either favorable or neutral for 1 or the other of the study drugs. Another 2 articles^18- 19 were excluded because we were unable to obtain information about the funding source, despite repeated requests. Of the 44 articles studied, there were 28 articles for hematopoietic colony-stimulating factors,^{12- 13,20- 45} 11 articles for antiemetics,^{14- 15,46- 54} and 5 articles for taxanes.^{16- 17,55- 57} The types of analyses included were cost-minimization or cost-identification (a comparison of the costs of treatment for 2 different agents with similar efficacy or outcomes) and cost-effectiveness (comparison of the costs of treatment for 2 agents normalized by their effectiveness, typically reported as cost per life-year gained). All of the articles fit 1 of these types, based on generally accepted definitions.^58- 59

Two investigators (M.F. and W.N.) independently abstracted information from each of the articles based on distinct, written, preset criteria. Information was collected on (1) the qualitative conclusion as stated in the abstract or manuscript conclusion, (2) the quantitative numerical results, (3) the timing of the study, and (4) the funding source.

Qualitative conclusions were rated according to the following criteria: favorable (the new drug "reduces costs" or is "cost-effective"), neutral (the new drug "is cost equivalent" or "may be cost-effective," or "does not require additional costs" over standard therapy), or unfavorable (the new drug has "higher costs" or is "not cost-effective"). Whenever the 2 investigators disagreed over an article's qualitative conclusion, a third investigator made the final decision.

Quantitative numerical results were also rated as favorable, neutral, or unfavorable. For cost-minimization studies, numerical results were classified as favorable when the costs of use of the new drug were less than standard treatment, neutral when there was no difference between the new drug and the standard, and unfavorable when the costs of use of the new drug were more than standard treatment. The total cost of treatment for each arm of the study was compared, including the cost of the study drug. When tests of statistical significance were available, significant differences were interpreted as favorable or unfavorable. Statistically insignificant differences were interpreted as neutral. For articles that did not include statistical analyses (typically, decision analyses), robust differences were interpreted as favorable or unfavorable. Nonrobust differences (which reversed direction under sensitivity analyses) were interpreted as neutral. For cost-effectiveness studies, any cost estimate of less than $50,000 per life-year gained was considered favorable, as is generally accepted in the literature.⁶⁰ More expensive results were considered unfavorable.

Study timing was interpreted as either prospective (the study was initiated alongside the clinical trial) or retrospective (the economic study was begun after the results of the clinical study were known).

Funding source was abstracted after recording a study's qualitative conclusion, quantitative results, and timing. Investigators were not specifically blinded as to funding source during abstraction. Articles were classified as either pharmaceutical company–sponsored or nonprofit-sponsored (government agency, professional organization, nonprofit foundation, or academic institution). For publications not including an acknowledgment of funding (17/46), first and last authors were contacted via mail, e-mail, and/or telephone and queried regarding the funding source of their study. Authors from 13 of 17 articles replied that their studies were either not externally funded or funded by nonprofit sources, while authors of 2 of 17 articles reported that their studies were funded by pharmaceutical companies. Authors of the remaining 2 articles^18- 19 did not reply, and their studies were not included in our analyses.

Relationships between funding source and (1) qualitative conclusion (favorable, neutral, or unfavorable), (2) overstatement of results (a favorable qualitative conclusion despite neutral or unfavorable quantitative results or a neutral qualitative conclusion despite unfavorable quantitative results), (3) study agent (hematopoietic growth factor, antiemetic, or taxane), (4) study timing (prospective or retrospective), (5) analysis type (cost minimization or cost-effectiveness), (6) journal type (peer-reviewed or non–peer-reviewed), and (7) author affiliations (all academic, or at least 1 pharmaceutical company or consulting firm employee) were analyzed using Fisher exact tests (for 2 × 2 tables with an expected cell value less than 5) or Pearson χ² tests. A 2-sided P value (against the null hypothesis of no relationship between conclusion and funding source) less than .05 was considered significant.

Of the 44 articles, 20 were funded by pharmaceutical companies and 24 by nonprofit organizations. For those studies funded by pharmaceutical companies, the funding source was always the manufacturer of the investigational drug. Approximately 65% of studies analyzed hematopoietic growth factors, 25% antiemetics, and 10% taxanes (Table 1). This distribution was similar for both pharmaceutical- and nonprofit-sponsored studies. Study timing, analysis type, and journal type also did not differ significantly by funding source. All authors of nonprofit-sponsored studies had academic affiliations, whereas 40% of pharmaceutical company–sponsored studies had at least 1 author with a pharmaceutical company or consulting firm affiliation (divided evenly between pharmaceutical company and consulting firm employees).

There was a statistically significant relationship between funding source and qualitative conclusions (P=.04). Unfavorable conclusions were reached by 38% (9/24) of nonprofit-sponsored studies but by only 5% (1/20) of pharmaceutical company–sponsored studies (Table 1). Reports including only authors who had an academic affiliation appeared more likely to report unfavorable conclusions (28% [10/36]) than those including pharmaceutical or consulting firm employees (0% [0/8]), although this difference was not significant (P=.18). The 2 investigators agreed on the classification of qualitative conclusions in 87% of the articles, with the third investigator determining the classification of the remaining 13%.

In addition, pharmaceutical company–sponsored studies were somewhat more likely than nonprofit-sponsored studies to overstate quantitative results; ie, a favorable qualitative conclusion when quantitative results were neutral or unfavorable, or a neutral conclusion when quantitative results were unfavorable (30% [6/20] vs 13% [3/24]), although this finding was not statistically significant (P=.26).

This study investigated financial conflicts of interest in the debate over economic analyses of breakthrough oncology drugs. We found a significant association between authors' stated qualitative conclusions regarding the costs and cost-effectiveness of these drugs and study sponsorship by the drugs' manufacturers. Studies funded by pharmaceutical companies were nearly 8 times less likely to reach unfavorable qualitative conclusions than nonprofit-funded studies and 1.4 times more likely to reach favorable qualitative conclusions. We also determined that 1 in 5 articles contained qualitative overstatements of quantitative results.

A number of hypotheses can help explain our findings. First, the retrospective methods used in 89% of our sample studies allow investigators and pharmaceutical companies "early looks" at clinical results and associated resource profiles. These early clinical data can be used to selectively identify the trials most likely to yield positive outcomes, and the pharmaceutical companies can fund economic studies accordingly and therefore, can potentially exercise a limited power to censor unfavorable studies simply by withholding financial support.

Second, there is an evident bias in the body of pharmacoeconomics research (also seen in other areas of medical research) toward the publication of studies with "positive" results. Regardless of funding source, studies with unfavorable preliminary evidence are less likely to be completed, less likely to be submitted for peer review, and, once submitted, less likely to be published.⁶¹

Third, pharmaceutical companies can influence research in a variety of ways. Studies may be funded through unrestricted research grants, educational funds, or consultancies (paid directly to investigators). These may include contractual agreements requiring pharmaceutical company review of manuscripts before being submitted for publication. Researchers also may receive funding from the same companies in the form of honoraria or travel awards for scientific meetings and have equity interests in companies and profit directly from increased drug sales.⁶² It is possible that these factors may result in some unconscious bias (perhaps when qualitatively interpreting results) that could influence study conclusions.

Fourth, the pharmaceutical companies can collaborate directly with investigators in devising protocols for economic analyses and indirectly shape the economic evaluation criteria.

Our study has several limitations. First, we considered only 1 type of economic relationship between pharmaceutical companies and researchers: direct funding of the analysis reported. Second, our ability to investigate direct financial sponsorship of the individual studies was limited because we were unable to review contracts or grants. While we used published information and direct communication with authors, the nature and degree of the financial relationship were not investigated.

The correlation between pharmaceutical company funding and favorable study conclusions might add to public uncertainty regarding company-sponsored medical research.^63- 64 Although other sources of funds for pharmacoeconomic studies are needed, limiting the publication of pharmaceutical company–sponsored studies is probably not feasible or practical. Pharmaceutical companies provide valuable resources to many areas of academic medicine and are a primary source of funding for pharmacoeconomic studies.^7,58 To improve the credibility of economic analyses, policies promoting full disclosure of all financial interests should be pursued. Conducting more prospective pharmacoeconomic analyses (in conjunction with phase 3 trials) would also increase credibility by eliminating the opportunity for selective funding based on clinical results.⁶⁵ Finally, pharmacoeconomic literature would be more balanced if managed care organizations, government agencies, and nonprofit groups increased their support for high-quality prospective pharmacoeconomic studies.