Composite Outcomes in Randomized Trials: Title and subTitle BreakGreater Precision But With Greater Uncertainty?

Randomized controlled trials are central to the evaluation of pharmaceuticals. They are used to provide evidence for the efficacy of pharmaceuticals in the licensing process that aims to ensure that drugs only become available for widespread prescription if they have a positive impact on symptoms, prognosis, or both. Furthermore, they play a major role in the evaluation of the effectiveness of treatments, providing the evidence base on which, for example, decisions on the inclusion of treatments in clinical guidelines are made.¹

The choice of outcomes measured (the outcome variables or end points) in clinical trials is an important design consideration. The primary outcome in particular has much invested in it, because it is normally the outcome alone that indicates whether or not the trial provides evidence at an acceptable level that the treatment is efficacious.²

Trials that examine treatments that are expected to have an effect on mortality and major morbidity often adopt a primary composite outcome measure that includes mortality along with other nonfatal end points. This article examines the use of composite outcomes in major clinical trials, focusing on those that include mortality. We assess the arguments for and against them and provide guidance on their application and reporting.

To assess the extent and nature of composite outcomes, we searched 9 journals, Annals of Internal Medicine, BMJ, Circulation, Clinical Infectious Diseases, Journal of the American College of Cardiology, JAMA, Lancet, New England Journal of Medicine, and Stroke, to identify trials that use composite outcomes, including mortality and major morbidity, for the 5 years from 1997 through 2001. We identified randomized trials using hand searching and a sensitive electronic search strategy within journals, searching on the words death, survival, and mortality as text words mentioned in the title or abstract. Altogether we identified 167 randomized trials published during this period, which together randomized 300 276 patients. The most common interventions across the journals we included were in the area of cardiovascular disease, and the most common single intervention during the period involved the use of stents. The therapeutic areas of the included trials are described in Table 1.

The frequency of the use of composite primary outcomes that included mortality during the 5-year period was fairly consistent. The New England Journal of Medicine published the most, with 58 during the period, whereas Clinical Infectious Diseases published only 1. The number of trials per journal per year is described in Table 2.

In 63 trials (38%) neither the composite outcome nor the mortality component was statistically significant at the conventional P = .05 level. In 60 trials (36%) the overall composite outcome was significant, but the mortality component was not significant. In only 19 trials (11%) were both the primary composite outcome and its mortality component significant. In 6 trials (4%) the composite primary outcome was not statistically significant, whereas the mortality component was. Nine trials (5%) reported multiple composite primary outcomes. Twenty-two trials (13%) did not report the statistical significance of the mortality component separately, although P values could be calculated for 12 of these trials. The results of the principal analyses of the remaining 10 trials could not be interpreted, despite being assessed independently by 3 of the authors (M.C., J.E., N.F.). Table 3 describes the assessment of each trial by journal of publication.

The finding that such a high proportion of trials that measure composite outcomes, including mortality, provide neutral results on the primary outcome may be unsurprising. However, the finding that a similar proportion are positive yet fail independently to identify an effect on the mortality component is striking and requires further consideration.

Randomized clinical trials can provide unbiased estimates of treatment effect. To achieve this, it is essential that trials follow a predetermined protocol describing the population to be studied, an appropriate method for random allocation, the procedure for follow-up, the outcomes to be analyzed, and the statistical methods that will be used.

If there is no obvious choice of primary outcome in a trial, then it is possible to adopt the composite of several outcomes. This situation was described by the International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH)³ as follows:

If a single primary variable cannot be selected from multiple measurements associated with the primary objective, another useful strategy is to integrate or combine the multiple measurements into a single or ‘composite' variable, using a predefined algorithm. . . . This approach addresses the multiplicity problem without requiring adjustment to the type 1 error.

Although dealing with multiple testing is an important factor in the design and analysis of clinical trials,² this may not be the only motivation behind the popularity of composite outcome measures. Instead, issues of statistical efficiency appear to be prominent, with composite outcomes in time to event trials leading to higher event rates and thus enabling smaller sample sizes or shorter follow-up (or both). Thus, to have power (1 − β) of .9 of finding a hazard ratio (relative risk) of 0.65 significant at the .05 level requires approximately 1400 patients with a 20% control event rate but only 700 with a 40% control event rate.⁴ Of course, the increase in power identified here depends on a common hazard ratio; this may or may not be the case.

A substantive risk associated with the reporting of composite outcomes is that the benefits described may be presumed to relate to all of the components. For example, Le May et al⁵ state in the abstract to their article, "Compared with accelerated t-PA [tissue-type plasminogen activator], primary stenting reduces death, reinfarction, stroke or repeat TVR [target vessel revascularization] for ischemia." In fact, only 5 deaths contributed to the primary end point, 3 in the stenting group and 2 in the t-PA group. Elsewhere in the abstract, the authors make it clear that there is no observed benefit on mortality, and the results for the components of the primary outcome are given together in a table. However, the statement describing the primary outcome may be considered misleading, especially if quoted out of context. A more appropriate description of this form of primary outcome could be "major event-free survival," albeit that descriptor, albeit technically correct, might also appear to suggest a reduction in mortality.

Some authors manage to communicate clearly the findings of their studies using composite primary outcome measures. For example, Leon et al,⁶ describing the results of a trial of localized intracoronary γ-radiation therapy, state that the composite primary outcome was statistically significant. Yet, they then state, "However, the reduction in the incidence of major adverse cardiac events was determined solely by a diminished need for revascularization of the target lesion, not by reductions in the incidence of death or myocardial infarction."

As our review establishes, the use of a composite outcome does not always lead to an increase in the evidence for benefit of an intervention. For example, the CAPRICORN [CArvedilol Post-infaRct survIval COntRol in LV dysfunctioN] trial investigated the effects of carvedilol, a β-blocker, in 1959 patients with left ventricular dysfunction following myocardial infarction.⁷ Originally, the primary outcome identified in the trial protocol was all-cause mortality. However, while the study was ongoing, the data and safety monitoring board (which has the key role of protecting the interests of patients in the trial)⁸ noted that the overall rate of accrual of deaths was lower than that predicted. The board communicated its concerns to the trial steering committee that the trial would not be powered to identify the primary end point as significant. The trial steering committee then took the unusual step of changing the primary outcome, dividing the available statistical power between a new composite outcome (all-cause mortality or cardiovascular hospital admissions), which the trial steering committee awarded a critical P value of .045, and the original primary outcome (all-cause mortality), which was awarded a prespecified critical P value of .005 to achieve statistical significance. Thus, if the P value for either primary outcome achieved statistical significance at the critical level, the study would be deemed positive.

The original primary end point (all-cause mortality) achieved a P value of .03 (ie, substantially larger than the .005 allocated to it), whereas the alternative primary outcome (composite of all-cause mortality and cardiovascular hospital admissions) had a P value of .30. Thus, the original primary outcome did not achieve statistical significance at the new and more stringent level determined by the revised policy for allocating statistical power in the trial. Although 12% of patients died in the carvedilol group, compared with 15% in the placebo group, 23% of patients in the carvedilol group and 22% of patients in the placebo group qualified for the composite outcome on the basis of hospitalizations alone, a result that undermined the relatively modest numerical reduction in mortality in the carvedilol group. Thus, strictly, CAPRICORN⁷ provides a neutral result, although ironically CAPRICORN would have been modestly statistically significant had the original primary outcome of all-cause mortality been maintained.

This example demonstrates the possibility that, by using composite outcomes, the measure of treatment effect can be diluted by an outcome that exhibits no effect being combined with a more critical measure that individually shows some evidence of benefit. It also demonstrates some of the arbitrariness of clinical trials; for example, the same trial may be considered positive or neutral on the basis of decisions concerning the statistical design. Fortunately, for those interpreting the results of trials, it is unusual for only 1 trial to be available. Although on its own CAPRICORN⁷ is neutral, it should be interpreted in the context of data from all relevant clinical trials, highlighting an important role for systematic review and quantitative meta-analysis.

Assessing the benefits of treatment across relevant available trials can highlight further challenges for the interpretation of composite outcomes. Glycoprotein IIb/IIIa inhibitors are used increasingly as medical therapy in the management of acute coronary syndromes. The pivotal trials used for licensing these agents used composite primary outcomes, including death, nonfatal myocardial infarction, and, occasionally, refractory ischemia. The US product insert for tirofiban hydrochloride, one of the glycoprotein IIb/IIIa inhibitors, states that:

AGGRASTAT [tirofiban], in combination with heparin, is indicated for the treatment of acute coronary syndrome, including patients who are to be managed medically and those undergoing PTCA [percutaneous transluminal coronary angioplasty] or atherectomy. In this setting, AGGRASTAT has been shown to decrease the rate of a combined endpoint of death, new myocardial infarction or refractory ischemia/repeat cardiac procedure.⁹

Reviewing the evidence for the medical use of glycoprotein IIb/IIIa inhibitors for acute coronary syndromes, Bhatt and Topol¹⁰ identified 4 trials that they state provide evidence for their use. The results of these trials, on the primary outcome identified by the protocols for each trial, are shown in Figure 1. The (theoretically) exact pooled odds ratio (OR)¹¹ for the effects of treatment at primary outcome is 0.86 (95% confidence interval [CI], 0.78-0.95), a benefit in favor of the treatment.

Although all-cause mortality is included in the composite outcome measure for every trial, the evidence that glycoprotein IIb/IIIa inhibitors are effective in reducing mortality is lacking. We pooled the all-cause mortality component in each of the trials, and the results are shown in Figure 1. For the 4 trials, the exact OR for the effect of glycoprotein IIb/IIIa inhibitors is 1.00 (95% CI, 0.82-1.22). Thus, although the results do not exclude a benefit for glycoprotein IIb/IIIa inhibitors on mortality, our best estimate of the effect is zero, and the plausible range runs from an 18% reduction in the odds of death to a 22% increase in the odds of death.

Investigation into the use of stents was the most common intervention in the trials we identified, with 20 (12%) overall. Twenty of the 24 predefined composite primary outcome measures (83%), including a mortality component, described in these trials were statistically significant. However, only 2 (8%) of 24 were significant for the mortality component of the analysis, and in both cases the comparison was with angioplasty, which was associated with increased hazard of death.^{12 - 13}

Simply identifying that the components of a primary outcome are consistent without considering statistical precision is insufficient, and indeed using this basis to claim efficacy of a treatment on the individual components of a composite primary outcome is misleading. For example, the abstract describing the results of the TARGET [Do Tirofiban and ReoPro Give Similar Efficacy Outcomes] trial¹⁴ states, "The magnitude and the direction of the effect were similar for each component of the composite end point (hazard ratio for death, 1.21; hazard ratio for myocardial infarction, 1.27; and hazard ratio for urgent target-vessel revascularization, 1.26." However, the estimate for mortality is based on approximately 20 deaths and the 95% CI stretches from 0.57 to 2.8 (P = .66), indicating that the play of chance is a highly plausible explanation for the result.

We hypothesized that the type of outcome included might influence the statistical significance of the composite primary outcome, specifically where components of the outcome were determined at least in part by the actions of clinicians rather than describing directly the disease. Thus, we considered that outcomes such as hospitalization or revascularization might be more amenable to change than a disease outcome such as nonfatal myocardial infarction.

Overall, 78 of 179 comparisons (including 20 primary outcomes from studies with multiple comparisons) included the following clinician-driven outcomes: revascularization, percuta neous mitral valvuloplasty, mechanical ventilation, hospitalization, transplantation (cardiac and liver), shunt, use of rescue therapy, institutionalization, initiation of new antibiotics, use of shock therapy, amputation, extra corporeal membrane oxygenation, and dialysis. We used a nonlinear mixed model¹⁵ to estimate the effects of clinician-driven outcomes on the likelihood of finding a statistically significant result, accounting for journal as a random effect. The inclusion of a clinician-driven outcome was predictive of a statistically significant result for the primary composite outcome (OR, 2.24; 95% CI, 1.15-4.34; P = .02).

There are 2 main advantages in using composite outcomes. First, the correct (a priori) identification of a composite primary end point can increase the statistical precision and thus the efficiency of a trial. When the aim of a trial is to provide proof of the efficacy of a treatment, a composite outcome can prove helpful, enabling a positive result from a smaller number of patients randomized. Thus, more trials may be conducted for a given investment in research, and results that provide guidance on the efficacy, or otherwise, of interventions may be available more quickly. Trials are costly to society and ideally they should provide reliable estimates of treatment effects in a timely and cost-effective manner. Although some trialists have advocated the use of large trials, these may also prove problematic. An example is the Heart Outcomes Prevention Evaluation trial, which estimated the effects of ramipril in patients who were at high risk but who "did not have left ventricular dysfunction or heart failure."¹⁶ Because of the scale of this trial (9297 patients), measurement of left ventricular function was only available for a minority of patients, undermining to some extent the interpretation of the findings. Second, as the ICH identified,³ a composite outcome may help investigators who are having difficulty in deciding which outcome to elect as the primary outcome measure in a trial and deal with the issue of multiplicity in an efficient manner, avoiding the need for arbitrary choices.

The disadvantages of composite outcomes may arise when the constituents do not move in line with each other. This appears most starkly when there is a principal end point (often all-cause mortality) supported by an additional and more common end point or end points. This disadvantage can happen in 3 ways. First, as in CAPRICORN,⁷ if an important component of the composite outcome is not substantially modified by the effects of treatment, then less rather than more statistical power to detect effects on the principal end point may be the result. Second, although the composite as a whole may appear to be affected by treatment, the evidence for benefit relating to its most important constituent may not exist or lack persuasiveness within the trial. This is the case with the trials of glycoprotein IIb/IIIa inhibitors described herein. Third, when clinician-driven outcomes, such as revascularization or hospitalization, are used, these appear generally to be more amenable to change, presenting further challenges for interpretation.

We have demonstrated that composite outcomes are commonly used in trials of mortality and major morbidity and identified several potential challenges in the interpretation of their results. A fundamental issue for composite outcomes, correctly identified by the ICH harmonised tripartite guideline (ICH E9),³ is that the outcomes that contribute to a composite outcome must be "associated with the primary objective." When this relationship is in question, the interpretation of composite outcomes is also in question. This requirement can be translated as being that the components should be measurable events that can sensibly be added together as being aspects of the same underlying disease process, and it helps if the composite can be given a single name. For example, in a trial of ursodiol for primary sclerosing cholangitis, a predefined composite primary outcome measure of "time to treatment failure" was used.¹⁷ The composite was defined as "death; liver transplantation; histological progression by two stages (of four) or progression to cirrhosis; the development of varices, ascites, or encephalopathy; sustained quadrupling of the serum bilirubin concentration; marked worsening of fatigue or pruritus; inability to tolerate the drug; or voluntary withdrawal from the study."

The more clearly a composite refers to an overall disease process, the less there is any problem of interpretation. The more likely it is too that the components of the composite will move in the same direction given an effective treatment. However, this is not guaranteed, as seen in CAPRICORN,⁷ the glycoprotein IIb/IIIa inhibitor trials,¹⁰ and coronary stenting trials, all of which used composite end points that were, presumably, a priori plausible.

It is particularly in the dissection of a composite that the differing interests of sponsors, licensing authorities, and interpreters become manifest. Sponsors are attracted by the hoped-for gains in statistical precision to improve their chances of a positive trial, and the increase in the use of composite outcome measures is largely due to their acceptance by licensing authorities. The latter might argue that their role is to ensure that licensed pharmaceuticals have an effect on an underlying disease process or symptoms rather than specifically on components of a composite primary outcome measure. Licensing authorities, however, have a role in the manner in which sponsors are permitted to market their products (the indication being agreed as a result of the evidence submitted by the sponsor). It is unsatisfactory for marketing claims to be made purely on the grounds of composite end points. Indeed, the product insert for tirofiban⁹ in the United States contains some information on the constituents of the primary outcome measure used in the trials that support licensing, although it is possible that this may be lost (literally) in the small print.

The licensing of pharmaceuticals is necessarily a game of fixed rules, and the judicial use of a composite outcome may serve this aim. However, the interpretation of the results of trials for other purposes should not allow rules to dominate well-informed good sense. A substantial number of major trials use composite primary outcomes that include all-cause mortality. The ICH E9³ states, "When a composite variable is used as a primary variable, the components of this variable may sometimes be analysed separately." To avoid the burying of important components of composite primary outcomes for which on their own no effect is observed, such as arguably may have been the case with the glycoprotein IIb/IIIa inhibitors described herein, the components of a composite primary outcome should always be described and analyzed individually. Thus, the components of a composite outcome should always be declared as secondary outcomes, and their results described alongside the result for the composite outcome. In addition, ICH E9 should be revised to address this point. The current version of the CONSORT recommendations¹⁸ makes no comment on composite outcomes and could usefully be revised accordingly. Patients often agree to participate in randomized trials because they believe their experience will inform the treatment of patients with similar disease in the future. However, in more than 1 in 10 trials in our review the reporting of data was inadequate. Recommendations on the appropriate use of composite outcomes are described in the Article .

1. Trialists should follow the CONSORT guidelines¹⁸ and identify precisely the prespecified primary and secondary outcome measures, reporting the results clearly in publications that describe the trial.

2. When trials report composite variables as primary outcomes, these should be interpreted together rather than as demonstrating efficacy of individual components of the composite.

3. Components of composite outcomes should always be defined as secondary outcomes and reported alongside the results of the primary analysis, preferably in a table.

4. Authors and journal editors should ensure that the reporting of composite outcomes is clear and avoids the suggestion that individual components of the composite have been demonstrated to be effective.

5. Systematic overviews and quantitative meta-analysis should be used to identify the effects of treatments on rare but important end points that may be included as part of composite outcomes in individual trials.

When describing the results of clinical trials, authors and journal editors could ensure that attempts are made to place the findings in the context of other available relevant research, as required by the CONSORT recommendations.¹⁸ The use of composite primary end points in clinical trials provides considerable support for the strategy of systematic overview and quantitative summary of the results of clinical trials to provide the best possible estimates on the effects on important outcomes.