Fair Conduct and Fair Reporting of Clinical Trials

Meta-analysis, done properly, is a systematic effort to search for and winnow out all the best evidence and show how well a given intervention works. It is crucially dependent on the identification of all available data from clinical trials. In 1989, Gøtzsche,¹ who was performing a meta-analysis of 244 trials of nonsteroidal anti-inflammatory drugs in rheumatoid arthritis, drew attention to a practice that seemed to subvert the normal process of publication as well as of meta-analysis. Excluding abstracts, letters, and brief versions, Gøtzsche found 44 multiple publications of 31 of the clinical trials, 20 trials published twice, 10 three times, and 1 trial 5 times, with the overall proportion of multiple publications being at least 18%. The fact that the data had been published elsewhere was not noted in 32 of the 44 articles. Indeed, in about half of them, the first author and number of authors were different, and in half there were important discrepancies between the various versions of the same trial. Gøtzsche¹ pointed out that "some cases were so difficult to detect that in a meta-analysis they might have been mistaken for separate trials."

In 1996, Huston and Moher² found the same problem when they attempted to perform a meta-analysis of the effects of the antipsychotic agent risperidone. They had a frustrating time. They identified 20 articles and several unpublished reports describing randomized, double-blind trials, but after a search they described as "vexing," "bewildering," and "intolerably time-consuming" they concluded that there were probably only 7 small trials and 2 large trials, 1 of the latter being reported "in part, transparently, and not so transparently in six different publications . . . the authorship was different for each." Huston and Moher wrote: "Multiple renditions of the same information is self-serving, wasteful, abuses the volunteer time of peer reviewers, and can be profoundly misleading; it brings into question the integrity of medical research."² It certainly does. Who, for example, designed the trial in the first place? Why aren't all the names of those who did the work of the study consistently cited as authors? This practice, if it merely inflated the bibliographies of the researchers and distorted the process of promotion, might seem somewhat academic, but it has more pernicious effects. Huston and Moher took objection to a practice that "made a mockery of editors and the peer review system." While editors may be strong enough to withstand such mockery, other objections were more telling. The practice gave "an artificial impression of wide support for the efficacy of an intervention."² The multiple counting of the same data and patients would lead to a confirmatory bias in favor of the intervention.

The authors of this case study, and others,³ did not attempt to quantify the extent by which this unfortunate behavior might influence the estimate of the effect of the intervention reported in the meta-analyses. Tramèr and colleagues⁴ took the process further when they examined full reports of 84 trials presumably involving 11,980 patients, investigating the effect of ondansetron on postoperative emesis. They found that data from 9 trials had been published in 14 reports, without any clear cross-reference, so there were really only 8645 patients in 70 trials. Seventeen percent of published, full reports of randomized trials were duplicates, and data from 3335 patients (28%) were reported twice. Most crucially, these authors found that inclusion of these duplicate data led to a 23% overestimation of the efficacy of ondansetron. Moreover, the covert duplicate publication was not easy to uncover: it had not been noticed by the expert authors of at least 8 subsequent articles, reviews, or book chapters.

This evidence shows that covert reporting of the same data in clinical trials artificially skews the balance of opinion in favor of the new drug. Why is this happening? Reviewing these case studies, it is hard not to suspect that this practice, which serves commercial interests so well, is deliberate, and, because it confuses and biases information important to the care of patients, it has to stop. To make this happen, a heavy responsibility falls on the sponsors, who are the makers of the new drugs being assessed. For the manufacturer, increasing the number of reports on the same trials may confer a short-term commercial advantage: more reprints, by apparently different and unconnected authors, for company detail representatives to press into the hands of physicians, and a bigger dossier to file with the regulatory agencies. But over the long term, the manufacturers face a loss in credibility. If these practices, which exaggerate so many biases in favor of their drugs, become widely known and associated with specific companies, are physicians likely to retain confidence in their products?

An equal share of the responsibility for these practices must fall on researchers. The rules governing authorship and against redundant publication^{5 - 6} are well known. The reader is entitled to assume that the authors of a study actually performed its important functions. A clearer case for disclosing the actual contributions of coinvestigators could not be made.⁷ If the company, rather than the authors, designed the study, that would be made clear by such disclosure. Tramèr et al⁴ point out that all the duplicates in their study were published as full reports in journals that refer to the International Committee of Medical Journal Editors ("Vancouver") guidelines.⁶ These specifically forbid the submission of articles that are being considered for publication, or have already been accepted or published elsewhere.⁴ At the very least, good scientific practice dictates that there must be accurate cross-referencing of articles that are duplicates.⁶

In this issue of THE JOURNAL,⁸ further light is thrown on this practice and on new ones that bias the results of clinical trials still more. Gøtzsche and Johansen have already published a systematic, meta-analytic review of the effects of antifungal prophylaxis or therapy in patients with cancer complicated by neutropenia.^{9 - 10} Now these 2 authors describe their experiences when trying to get at the data behind another such review.⁸ If treatment is ever to be based on a rational examination of the facts, their story has lessons for us all.

To reduce publication bias,^{11 - 12} the authors followed the guidelines of the Cochrane Collaboration¹³ and performed an exhaustive search for all relevant trials that compared fluconazole and amphotericin B. A basic requirement of such a search is that it go beyond MEDLINE, which often produces only about 50% of relevant information.¹⁴ To try to identify all relevant data, those who perform meta-analyses should contact authors of published and unpublished trials and obtain fuller details of trials mentioned in abstracts, reviews, letters, and conference proceedings. In addition, those conducting systematic reviews are expected to contact relevant scientists employed by or sponsored by the drugs' manufacturers.

Johansen and Gøtzsche found 15 trials that met their criteria for inclusion, but they also found that 44% of the patients described were included in only 3 reports, each being 3-armed trials comparing fluconazole, amphotericin B, and nystatin.⁸ In the reports of these trials, the 2 "polyene" drugs, amphotericin B and nystatin, were combined for comparison with fluconazole. Since the insoluble agent nystatin was known from the start not to be useful in this context, the result of such reporting would be to dilute the effect of amphotericin B and to improve the apparent effectiveness of fluconazole. Because 79% of the patients randomized to amphotericin B received it orally, not intravenously, fluconazole seemed even more effective. In short, fluconazole raced against a heavily handicapped opponent. When, with difficulty, Johansen and Gøtzsche disentangled the effects due to the various comparison drugs, they found that nystatin was no better than placebo and that fluconazole was no better than amphotericin B.

Despite efforts to contact the relevant scientists, including those at the manufacturers of fluconazole (Pfizer Inc, New York, NY), Johansen and Gøtzsche⁸ were unable to get a justification for the drug's comparison with oral as opposed to intravenous amphotericin B and for the combining of the nystatin results with the amphotericin B results. They were also unable to sort out the polyene trials from each other. This is reminiscent of the difficulties experienced by Huston and Moher² when attempting to obtain information from the makers of risperidone and sponsor of some of the trials (Janssen Pharmaceutica NV, Beerse, Belgium).

As in the previous cases, Johansen and Gøtzsche encountered not only unnecessary and covert duplication, but 2 additional problems. First, this meta-analysis was beset by difficulties in obtaining information from researchers and manufacturers—this, despite the fact that a scientific duty assumed by a researcher who accepts the credit that goes with publication is to answer the questions and criticisms of fellow scientists and readers. This duty cannot be abdicated, and it stays with the authors if, for example, they change affiliation, nor can it be transferred to the sponsoring company. After all, it was the researcher who performed the study and who takes credit for it, and the researcher is the individual the reader reasonably expects to give credible responses about its design and conduct. It would be interesting to know whether the editors who published their work, or the readers who relied on it, were aware that the authors would be so unforthcoming to the meta-analysts, their scientific colleagues. The company must make sure that those researchers it sponsors understand this duty and, as sponsors, must do everything possible to provide the information. If the company's scientists designed and monitored the study, their scientists should be listed as coauthors on the article's byline.

Second, there was the problem of discerning and allowing for the peculiar comparisons made. Again, each of these problems seemed to commercially benefit the sponsors of the studies. When the patients agreed to take part in these trials, were they informed that the research design was deficient and that the reporting would be closed to inquiry?

What can be done to prevent and overcome these and similar problems, and to ensure fair conduct and fair reporting of clinical trials?

Registration of trials and publication of all results is the way to prevent publication bias—the tendency whereby favorable results are published more frequently and more rapidly.^{15 - 16} In 1986, Simes¹⁷ showed that pooled analysis of published trials demonstrated a significant advantage for combination chemotherapy in advanced ovarian cancer, an advantage that disappeared when the analysis was restricted to trials that had been registered in advance. Hetherington et al¹⁸ have shown that retrospective attempts to obtain data about unpublished trials are usually unsuccessful. Prospective registration overcomes this problem, will help avoid confusion as to whether different authors are describing the same trial, and will prevent double-counting of data and patients. Institutional review boards can insist that the trials they approve be registered at inception.¹⁹ Journals can help by requiring that the trials they publish be registered. There are now hundreds of trial registries, and the number grows monthly. Meta-registries to include the data in the many smaller registries are being set up. Early results of these positive efforts can be seen by searching, for example, Current Controlled Trials' metaRegister of Controlled Trials (http://www.controlled-trials.com/home_page.cfm). The Web site contains basic core information pertaining to ongoing clinical trials, such as design and eligibility criteria. However, the biggest problem standing in the way of successful, all-inclusive registries is the reluctance of drug companies that sponsor so much research to cooperate, even though the results of the trials are not what is being sought by the registries.

Fortunately, there are already signs from some companies that they put patients' welfare first. In response to pressure from members of the Cochrane Collaboration to register at their inception all clinical trials to prevent publication bias, 2 companies have agreed to do so.^{20 - 22}

First, in 1996, Schering Health Care agreed to put information about all of the company's ongoing controlled trials in the Cochrane Controlled Trials Register. Then, in 1998, the chairman of Glaxo Wellcome²¹ announced that, to reduce publication bias, his company would henceforth register all its clinical trials worldwide and make the information accessible to health care professionals and researchers outside the company. Protocols for completed phase 2 and phase 3 studies would be registered at the time of regulatory approval, "and the register will be updated at least annually with protocols for [Glaxo Wellcome's] largescale phase IIIb and IV studies." In addition, the chairman announced that his company was committed to publishing all the trials and to assigning each trial a unique identifier to be used in all publications.²³

It is hard to think of any step a pharmaceutical company could take that would so reassure its clients as to its ethical, clinical, and scientific good intentions. These 2 companies are to be congratulated on being the first to make this crucial decision. Two years have passed, yet none have followed. I urge other companies that sponsor clinical trials to follow their lead for the ultimate benefit of our patients.