Reporting Mortality Findings in Trials of Rofecoxib for Alzheimer Disease or Cognitive Impairment: Title and subTitle BreakA Case Study Based on Documents From Rofecoxib Litigation

Clinical trials registration is now the standard expected by the International Committee of Medical Journal Editors.¹ The Food and Drug Administration Amendments Act (FDAAA), effective October 1, 2007, requires not only the registration of all phase 2 to phase 4 clinical trials of new drugs but also the submission of trial findings to a publicly available results database.² The purpose is to reduce the selective publication of entire trials or their results.

Even in published articles, selective reporting, if present, can be difficult to detect. Typically, reviewers and journal editors rely on descriptions of study methods such as randomization or blinding to assess study quality.³ In an effort to improve reporting, JAMA has implemented a policy requiring that an academic statistician conduct the data analysis for all randomized trials.⁴ Another approach involves direct comparisons, when available, between study plans in design papers and study conduct in final publications. Even these methods may fail to identify irregularities in the representation of data from clinical trials.⁵

Legal work by one of us (R.A.K.) provided the opportunity to evaluate the quality and completeness of the data reporting at several levels, including the data presented in published papers, information provided to the FDA by the sponsor, and internal analyses conducted by the sponsor. In this article, these 3 representations of the mortality findings are summarized in this order.

In the course of work for a plaintiff's attorney in a lawsuit about rofecoxib,⁶ one of us (R.A.K.) reviewed a large number of company documents and materials submitted by the company to the FDA. Data sets from the trials evaluating rofecoxib in the setting of Alzheimer disease were also provided, and they were reanalyzed by one of us (R.A.K.). This analysis relied on and was based on the raw data files produced by the company and submitted to the FDA. The data were in the form of SAS data files, tables, program code, value, and variable labels. The Alzheimer disease studies differed from many other Merck trials because the analysis plan for safety data was based on an intention-to-treat analysis. As a result, mortality event information was available not only for the on-drug period but also for off-drug follow-up of most patients. Like the APPROVe and VICTOR trials,^{7 - 8} the Alzheimer disease trials followed patients after they had discontinued the study drug until the end of the trial, an approach that permitted intention-to-treat analyses for the mortality end point.

Documents consulted for this article (references 11-17) are available at http://www.biostat.washington.edu/research/Rofecoxib.

The sponsor conducted 3 clinical trials (known as protocol 078, protocol 091, and protocol 126) to assess the effects of rofecoxib on the occurrence or progression of Alzheimer disease.

According to the published report,⁹ protocol 078 enrolled 1457 patients older than 65 years with mild cognitive impairment and randomly assigned them to blinded treatment with rofecoxib, 25 mg, or placebo.⁹ Patients were recruited at 46 US study sites between April 1998 and March 2000.⁹ The planned duration of the study was 24 months, but because the event rate was lower than anticipated, the study was extended to 48 months. The primary end point was the development of Alzheimer disease. In this study,⁹ rofecoxib was associated with a statistically significantly higher risk of diagnosis of Alzheimer disease than placebo (hazard ratio [HR], 1.46; 95% confidence interval [CI], 1.09-1.94; P = .011).

According to the published report,¹⁰ protocol 091 randomly assigned 692 patients older than 50 years and diagnosed with possible or probable Alzheimer disease to blinded treatment with rofecoxib, 25 mg, or placebo. Patients were recruited at 31 US study sites from February to September 1999.¹⁰ The planned duration of the study was 12 months. The primary end point was reduction in cognitive decline measured by the Alzheimer's Disease Assessment Scale. In this study, there was no difference in risk of progression of Alzheimer disease between treatment groups.¹⁰

The sponsor had started another trial, protocol 126, which was similar to protocol 091.¹¹ This study had randomly assigned 382 patients to rofecoxib and 376 to placebo.¹¹ Protocol 126 was terminated early (in March 2001) because protocol 091 had shown no benefit. Details about this trial were never published, and it is unclear whether follow-up was complete.

In trials 078 and 091, the company collected investigator-reported adverse events and adjudicated certain cardiovascular events and all deaths. The published versions of both protocol 091 in 2004¹⁰ and protocol 078 in 2005⁹ provide information about mortality in the text of the articles, but without reporting any statistical analyses or statistical tests. For protocol 091,¹⁰ the report states: “There were no drug-related deaths during the study. Non-drug related deaths occurred in 11 patients (9 in the rofecoxib group and 2 in the placebo group) while taking study treatment or within 14 days of the last dose.” Deaths that may have occurred more than 14 days after the last dose were not reported. Regarding safety, the authors conclude that “Rofecoxib was generally well tolerated by the elderly patients in our study, which is consistent with results from previous clinical trials in patients with osteoarthritis.”^{10 (pp70-71)}

For protocol 078, the report states: “A total of 39 deaths occurred in patients who were taking study treatment or from fatal adverse events that started within 14 days of the last dose (24 or 3.3% for rofecoxib and 15 or 2.1% for placebo). . . . There were an additional 22 deaths in the off-drug period (17 in patients assigned to rofecoxib and five in patients assigned to placebo); 12 of these (11 in the rofecoxib group and one in the placebo group) occurred more than 48 weeks after treatment discontinuation.”^{9 (p1210)} The report also states: “In addition to evaluating efficacy, the present study provided important placebo-controlled data on the safety of rofecoxib 25 mg over periods of up to 4 years in an elderly population. . . . Rofecoxib was generally well tolerated by the elderly patients in the study, consistent with results from prior clinical studies in osteoarthritis (Langman et al, 1999; Reicin et al, 2002) and AD (Reines et al, 2004). The overall incidence of adverse experiences, serious adverse experiences, and discontinuations due to adverse experiences were [sic] similar or only slightly increased for rofecoxib vs placebo.”^{9 (p1212)} The overall impression created by this report, for which 8 of 11 authors are employees of the sponsor, is that rofecoxib was “generally well tolerated.”^{9 (p1212)}

In July 2001, the sponsor filed a Safety Update Report (SUR) with the FDA¹¹ and reported the mortality findings from studies 091 and 078. Regarding mortality, the SUR concludes: “Therefore, review of the deaths does not identify a specific increased risk with rofecoxib.”^{11 (p101)} The sponsor's mortality data included in the SUR are presented in the Table.¹¹ The report does not clearly describe how the sponsor arrived at the counts in the Table, although deaths that had occurred more than 14 days after discontinuation of the trial drug apparently were not included. In other words, it appears that the sponsor presented only an on-treatment and not an intention-to-treat analysis for total mortality.

On December 5, 2001, the FDA sent a letter¹² to the sponsor and asked about the ethics of continuing study 078 based on the excess mortality seen in study 091. The question posed to Merck was: “Please clarify whether the safety monitoring board and the IRB [institutional review board] overseeing these studies are aware of the excess in total cause mortality in the Vioxx 25 mg group as compared to placebo (p=0.026) and the trend against Vioxx 25 mg on CV mortality compared to placebo. . . . Have these oversight groups commented on the ethics of continuing study 078 in light of the mortality data?”¹² In its response to the FDA, the sponsor reiterated the mortality data and characterized the rofecoxib-placebo findings as “small numeric differences . . . most consistent with chance fluctuations.”¹³ The letter from the sponsor also stated: “With regard to dissemination of these data, individual study site IRBs, rather than a single, central IRB are providing oversight for the 078 study. There is no data safety monitoring board. MRL [Merck Research Laboratories] has not provided these data to the individual IRBs because MRL does not believe that a safety issue has been identified. Moreover, the 078 study is still under blind both to personnel at study sites and to personnel at MRL monitoring these studies. In the absence of a compelling and clear safety issue, MRL has not broken study blind to individuals involved in these studies.”¹³

In an April 8, 2001, internal memorandum,¹⁴ several months before the submission of the July 2001 SUR, one of the sponsor's statisticians summarized the combined mortality experience from protocols 091 and 078. The 091 study had been completed, and 078 was still in progress. The mortality data are summarized in the Table.¹⁴ The analysis was based on the intention-to-treat principle and included events occurring during the 1-year study period of protocol 091 plus 14 days and the last follow-up for patients in protocol 078 plus 14 days (median follow-up of 1.7 years). The analysis used the Cox model and adjusted for age and sex. In the sponsor's intention-to-treat analysis,¹⁴ rofecoxib was associated with an increased risk of mortality in each of the studies. For protocol 091, the HR was 4.43 (95% CI, 1.26-15.53).^{14 (p5)} For protocol 078, the HR was 2.55 (95% CI, 1.17-5.56).^{14 (p6)} In a combined analysis, rofecoxib was associated with a 3-fold increase in total mortality (HR, 2.99; 95% CI, 1.55-5.77).^{14 (p9)} For an internal meeting to be held on November 6, 2001, the sponsor circulated a slide set that “summarizes the current status of the three AD protocols including a summary of the completed mortality analyses.”¹⁵ For the combined data from the 3 protocols (078, 091, and 126), an increased risk of all-cause mortality was apparent for both the on-drug analysis (RR, 2.43; P = .015)¹⁵ and the intention-to-treat analysis (RR, 2.56; P = .001).¹⁵ In 2003, the sponsor submitted to the FDA the intention-to-treat mortality data without detailed analysis or comment.¹⁶

The Table summarizes the results of an independent analysis conducted by one of us (R.A.K.) of the data files provided by the sponsor in the New Jersey Vioxx litigation, including data through the completion of the 078 trial and the data from study 091.¹⁷ Classification of the cause of death into the categories shown in the Table was based on data presented in the July 2001 SUR¹¹ for study 091 and the Clinical Study Report (CSR) filed with the FDA in July 2003¹⁶ for study 078.¹⁷ This analysis provided evidence of the excess risk associated with rofecoxib for noncancer deaths (HR, 2.71; 95% CI, 1.57-4.68; P<.001). Most of this excess was due to heart disease deaths (HR, 3.84; 95% CI, 1.54-9.51). The independent analysis also confirmed the increased risk of total mortality (HR, 2.13; 95% CI, 1.36-3.33; P<.001) first identified by the sponsor's statistician in April 2001.¹⁴ The Figure provides the Kaplan-Meier plot for total mortality.

In April 2001, the sponsor conducted intention-to-treat analyses that clearly identified an increased risk of mortality associated with rofecoxib among patients in the Alzheimer disease trials. These combined intention-to-treat analyses were not submitted to the FDA until 2003. The data submitted to the FDA in 2001 used a variety of counting methods, including on-treatment rather than intention-to-treat analyses, an approach that minimized the appearance of the mortality risk. The FDA raised questions about the findings in the July 2001 SUR, and the sponsor indicated that it had not informed the IRBs of the findings.¹³

In the letter of December 5, 2001,¹² the FDA had also assumed that protocol 078 had an active data and safety monitoring board (DSMB). But the 078 study, which had IRB approval, did not have a DSMB.¹³ The only human-subjects protections available to the study participants were those provided by the investigators who were blind not only to the treatment allocation but also to the findings for study-wide adverse events, and by the unblinded Merck investigators, who did not discern a safety issue. The sponsor's submission of individual adverse event reports over time to the FDA is not adequate for active trial monitoring. The FDA depends on the sponsor and the DSMB to alert the agency about any evidence of harm that may be associated with the drug.

The sponsor, having failed to inform IRBs about its own intention-to-treat analysis conducted in April 2001, allowed study 078 to continue for about 2 additional years. In 2002, moreover, the study participants were reconsented for the extension of study 078. During the additional follow-up time, there were approximately 8 excess deaths among those randomly assigned to receive rofecoxib (20 additional deaths among those assigned to rofecoxib and 12 among those assigned to placebo). In this study, rofecoxib was also associated with an increased risk of progression to Alzheimer disease, a finding that was apparent early in the trial (Table 2 in the 078 publication⁹ ). The mortality findings and the Alzheimer disease findings would, in our judgment, have prompted a DSMB, if it had existed, to stop the trial early.

After the publication of a review that raised safety questions about cyclooxygenase 2 (COX-2) inhibitors,¹⁸ a meta-analysis of cardiovascular thrombotic events in 23 rofecoxib trials was published in November 2001.¹⁹ Although the 2 Alzheimer disease studies were included in this meta-analysis, the authors, 5 of them employees of the sponsor, did not take the opportunity provided by this publication¹⁹ to report the findings for total mortality.

The disparity between the mortality results reported in the 2004 and 2005 publications and the mortality findings summarized in the sponsor's internal intention-to-treat analyses is striking. Sponsors have a direct financial interest in their products and a fiduciary duty to shareholders to provide a return on their investment. These interests disqualify sponsors from other important duties, including those normally accorded to DSMBs and IRBs. Failure of the sponsor to inform IRBs of a safety issue violates the trust of those human participants who volunteered to advance science, medicine, and public health.

The recent report on the FDA's oversight of clinical trials by the Office of the Inspector General raises similar questions.²⁰ Lacking clinical trials registration, the FDA is not able to identify all trials, their sites, and their IRBs. The FDA relies on voluntary compliance to correct violations of regulatory significance. Few FDA inspections are conducted, and they tend to focus on verifying clinical trial data after the fact rather than on the protection of human research participants. All large clinical trials, especially for drugs with known serious risks, should have a DSMB. The minimal registration data set for clinical trials registration²¹ should also include information about the independent DSMB as well as the IRB.

Safety analyses can be conducted in 2 ways: by on-treatment analysis or by intention-to-treat analysis. If toxicity occurs when patients are actively taking the drug, the on-treatment approach may be more powerful than intention-to-treat analyses for detecting important associations with adverse events. Adverse drug effects, however, may persist after drug discontinuation and under these conditions, the on-treatment analysis will systematically underestimate risk, as apparently occurred in the sponsor's analyses. Further, adverse effects due to the active therapy may differentially cause dropouts in the active therapy group, resulting in biased results for any end point that may be associated with these adverse effects. The preferred method of analysis for randomized trials is the intention-to-treat analysis with complete follow-up of all randomized patients through the end of the trial. Even if the trial protocol specifies an on-treatment analysis for safety, all safety data should be collected and analyzed by the intention-to-treat method as well.

For reviewers and journal editors, attention to methodologic quality of a manuscript may provide an incomplete picture of study quality. Analysis by an independent academic statistician is an important additional protection.⁴ Direct comparisons between the original protocol of the study design and the final description of the study conduct, although perhaps burdensome to reviewers and editors, are also likely to improve the accuracy and the completeness of the reporting of important clinical trials.

Korn and Ehringhaus²² have published a set of principles for protecting the integrity of the conduct and reporting of clinical trials. These principles from the Association of American Medical Colleges would have protected patients in protocol 078, but they apply to academic scientists who work with sponsors. For sponsors that conduct their own studies or use contract research organizations to conduct studies, it is not clear how transparency in the reporting of results can be achieved if a sponsor chooses to represent its products in the best possible light. The commercialization of clinical trials has neither improved the quality of the science nor enhanced the protection of human research participants.^{23 - 25}

The findings from this case study suggest that additional protections for human research participants, including new approaches for the conduct, oversight, and reporting of industry-sponsored trials, are necessary. A clinical trials system in which sponsors fund the trials that are conducted by independent investigators would provide additional protections.