Placebo Response in Studies of Major Depression:  Variable, Substantial, and Growing FREE

Contentious debate continues about the ethics of placebo administration in clinical trials in general and in psychiatry in particular.^1- 4 Supporters of the use of placebo maintain that, in the study of many illnesses, the assignment of individuals to receive placebo is scientifically critical and that such assignment is ethical if it is not expected to contribute to increased mortality or irreversible morbidity.^5- 8 Critics of the use of placebo assert that after the efficacy of 1 or more treatments for an illness has been established, it is not ethical to assign subjects in a research study to an intervention that is expected to be less efficacious.^9- 11 Such critics suggest that the scientific aims of placebo use (eg, determination of the utility of new interventions) may be achieved with alternative designs, including the use of active comparators (interventions with established efficacy) or previously collected information on the rate of placebo response.^1,12- 14

The study of pharmacologic treatments for major depressive disorder (MDD) has traditionally used assignment to placebo in randomized controlled trials. Based on trials using this design, the Food and Drug Administration (FDA) has approved for marketing antidepressant agents that are among the most frequently prescribed medications in the world and are widely viewed as effective for MDD. Paradoxically, the clinical success of these medications has contributed to ethical concerns about the continued use of placebo in trials to assess the efficacy of new antidepressant medications.¹¹ It has been argued that the use of placebo assignment in such trials remains justified because the demonstration of efficacy compared with placebo eliminates the risk to public health of approving ineffective medications and because there is no evidence that treatment delay or assignment to placebo results in permanent harm.^8,15 Along with these ethical arguments, investigators have suggested that, in recent years, it has become more difficult to demonstrate specific antidepressant effects compared with placebo, suggesting that the response to placebo may have increased.^16- 18 In this context of persistent ethical and scientific concerns about the use of placebo in studies of MDD, we examined data on the response to placebo and medication in controlled trials of adult outpatients with MDD published between 1981 and 2000 to determine whether the characteristics of such studies and the response to placebo have changed.

We collected peer-reviewed articles describing randomized, placebo-controlled trials of medication for outpatients with unipolar MDD published between 1981 and 2000. Studies published between 1981 and 1990 were taken primarily from an extensive, previously published literature review.¹⁹ To create a list of articles published between 1991 and 2000, we used similar methods. We specified key words, including placebo and generic names of all putative antidepressant medications, and conducted multiple computer searches using MEDLINE and PsycLit.

To be included in this review, articles were required to meet the following criteria: (1) reported in English; (2) published between January 1981 and December 2000; (3) primarily composed of outpatients with MDD according to research diagnostic criteria (RDC), Diagnostic and Statistical Manual of Mental Disorders, Third Edition, Diagnostic and Statistical Manual of Mental Disorders, Revised Third Edition, Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, International Classification of Diseases, Ninth Revision (ICD-9 ), or International Classification of Diseases, 10th Revision (ICD-10 ) criteria for major depression (we excluded trials that focused on bipolar disorder, but did include several early studies^20- 22 that contained a small number of bipolar depressed patients); (4) had at least 20 patients in the placebo group; (5) lasted at least 4 weeks; (6) randomly assigned patients to receive a putative antidepressant drug or drugs and placebo and assessed patients under double-blind conditions; and (7) reported the total number of patients assigned to placebo and medication group(s) and the number who had responded as determined by a reduction of at least 50% in their score on the Hamilton Rating Scale for Depression (HRSD)²³ and/or a Clinical Global Impression (CGI)²⁴ rating of markedly or moderately improved (CGI score of 1 or 2).

Data were extracted from the articles by 2 of the authors (S.S., R.S.) and discrepancies were resolved via discussion and additional review by a third author (B.T.W.). For each study, we calculated the response rate for the placebo and medication group(s) by dividing the number of patients in each group who had responded according to these criteria by the number of patients assigned to the group. In a few studies,^25- 29 only the number of patients randomized who returned for at least 1 visit was given, and this number was used as the denominator. If response data were provided only on study completers, we assumed that noncompleters had not responded.

To examine the strength of the linear relationship between continuous variables, we used Pearson correlation coefficients (2-tailed) and linear regression. To examine differences between groups in continuous variables, we used the t statistic. To assess whether the proportion of studies that used certain design features (eg, placebo lead-in before randomization) had changed linearly with time, we determined the number of studies that used the feature during each of the four 5-year periods between 1981 and 2000 and calculated a χ² statistic for linearity.³⁰ Tests of statistical significance regarding fractions of patients responding were also performed using an arcsin transformation of the fraction, specifically, 2 × arcsin. The results of the analyses using the untransformed and transformed data were virtually identical, and only results of analyses of untransformed response rates are reported herein. Effect size was calculated as the difference between the arcsin-transformed response rate on active medication and the arcsin-transformed response rate on placebo.³¹ Statistical calculations were performed using SPSS for Windows, version 10 (SPSS Inc, Chicago, Ill).

Seventy-five trials of medication for adult outpatients with MDD meeting our criteria for review were identified.^{20- 22,25- 29,32- 4546- 99} Data are presented as mean (SD). The trial length averaged 45.4 (17.9) days (range, 28-168 days) and increased significantly with year of publication (n = 74; r = 0.45; 95% confidence interval [CI], 0.25-0.61; P<.001). The average number of patients per group was 83.8 (59.7) (range, 20-336) for placebo and 85.0 (60.0) (range, 20-335) for medication; neither changed significantly with year of publication (placebo: n = 75; r = 0.12; 95% CI, −0.11 to 0.34; P = .30; medication: n = 75; r = 0.10; 95% CI, −0.13 to 0.32; P = .41). The diagnostic systems used have changed over time. For example, the proportion of studies using RDC declined from 28.6% between 1981 and 1985 to 0% between 1996 and 2000 (χ²₁ = 8.5, P = .004). The diagnostic systems generally require a minimum duration of 2 weeks of symptoms. Seventeen trials (22.7%) required a longer duration of illness, usually 4 weeks, to merit a diagnosis of MDD. Most trials (74.7%) used a 1- to 2-week initial placebo lead-in period; patients who responded during this time were not randomized. Neither the number of studies that required more than 2 weeks of symptoms nor the number using placebo lead in changed significantly with time (χ²₁ = 1.8 and 1.7, and P = .18 and .19, respectively).

Seventy-four studies used the HRSD to measure severity of depression, and 67 reported which version was used to assess severity of depression at study entry: the 17-item version was used in 33 studies (49%), the 21-item version in 32 (48%), and 24-item version in 2 (2.9%). The proportion of studies using the 17-item version did not change significantly with time (χ²₁ = 0.31, P = .58). Sixty-eight studies required a minimum score on the HRSD for study entry. To compare the HRSD scores across studies using different versions of the HRSD, we prorated the HRSD score by dividing the score by the number of items in the HRSD version and multiplied by 17. Among the 63 studies that provided sufficient information to perform this calculation, the prorated minimum entry score averaged 16.7 (2.1) (range, 12.9-22.0) and increased with year of publication (n = 63; r = 0.37; 95% CI, 0.13-0.57; P = .003). Forty-five of the studies provided explicit information on whether the use of concomitant medications (usually chloral hydrate or a benzodiazepine) was permitted during the trial for treatment of anxiety or insomnia. In 38 (84%) of these 45 trials, concomitant medications were permitted. The proportion of trials permitting concomitant medication did not change significantly with time (χ²₁ = 0.72, P = .39).

On average, 60.1% (9.7%) (range, 36.2%-85.0%) of participants were women, and this proportion did not change significantly with year of publication (n = 63; r = 0.00; 95% CI, −0.25 to 0.25; P = .99). The average age was 42.6 (7.3) years (range, 35.5-72.0 years) and increased significantly with year of publication (n = 61; r = 0.30; 95% CI, 0.05-0.51; P = .02). The average observed HRSD score at study entry, prorated for the number of items in the HRSD version, was 22.5 (2.7) (range, 13.8-30.2) and did not change significantly with year of publication (n = 61; r = −0.10; 95% CI, −0.34 to 0.16; P = .43).

Fifty-three studies provided information on the number of patients responding to placebo as indicated by a 50% or greater reduction in HRSD score. By this measure, the average proportion of patients per study who responded while receiving placebo was 30.0% (8.0%), and this proportion increased significantly with year of publication (n = 53; r = 0.43; 95% CI, 0.18-0.63; P = .001). Forty-seven studies provided information on the number of patients responding to placebo as indicated by a CGI rating of 1 or 2. By this measure, the average proportion of patients per study who responded while receiving placebo was 32.8% (11.1%), and this proportion increased significantly with year of publication (n = 47; r = 0.53; 95% CI, 0.29-0.71; P<.001).

We were interested in combining data from all studies to examine the relationship between study characteristics and placebo response rate. Twenty-two studies presented placebo response information only on the basis of CGI rating. To combine information from these studies with data from the other 53 studies that provided placebo response information on the basis of a 50% or greater reduction in HRSD score, we estimated the proportion responding according to HRSD as follows. Twenty-five studies reported both the number of patients receiving placebo who responded according to HRSD criteria and the number who responded according to CGI criteria. These proportions were highly correlated (n = 25; r = 0.79; 95% CI, 0.57-0.90; P<.001). We estimated the proportion responding according to HRSD criteria using the equation for the best-fit straight line obtained from the 25 studies that provided data on response according to both measures (HRSD response rate = 0.658 × CGI response rate + 0.0832). According to this combined HRSD measure, 29.7% (8.3%) (range, 12.5%-51.8%) of patients assigned to placebo responded. The proportion of patients taking placebo responding by this measure was significantly correlated with year of publication (n = 75; r = 0.45; 95% CI, 0.25-0.61; P<.001) (Figure 1), length of the trial (n = 74; r = 0.27; 95% CI, 0.04-0.47; P = .02), and prorated minimum HRSD score required (n = 63; r = 0.27; 95% CI, 0.02-0.49; P = .03). In a multiple linear regression model, with proportion of patients taking placebo responding according to the combined HRSD measure as the dependent variable and year of publication, length of trial, and prorated minimum HRSD score as independent variables, only year of publication remained significant (β = 0.366, t₅₈ = 2.7, P = .009). There were no statistically significant associations between the proportion of patients taking placebo who responded and prorated average initial HRSD score (n = 61; r = 0.10; 95% CI, −0.16 to 0.34; P = .46), average patient age (n = 61; r = 0.11; 95% CI, −0.15 to 0.35; P = .42), fraction of patients who were women (n = 63; r = 0.09; 95% CI, −0.16 to 0.33; P = .49), presence of a placebo lead-in phase (t₇₃ = 0.35, P = .73), minimum duration of symptoms required being greater than 2 weeks (t₇₃ = 0.23, P = .82), or concomitant medication use (t₃₅ = 0.86, P = .39).

To put the response to placebo in context, we examined the response to active medication in the same 75 studies. The 25 studies that provided information regarding placebo response on the basis of both a 50% reduction in HRSD score and CGI response contained similar information on medication response. Since most studies examined more than 1 active medication, we determined the best-fit straight line describing the relationship between CGI response and HRSD response to the first medication described in each of these studies: HRSD response rate = 0.744 × CGI response rate + 0.0805. The slope for this equation does not differ significantly from that obtained for placebo response rates (t₄₆ = 0.56, P = .72). For studies that provided response rates only using CGIs, we estimated the proportion responding according to HRSD criteria using this equation.

Sixty studies (80%) assigned more than 1 group of patients to active medications (average number of active medication groups, 1.97 per study; range, 1-4), and many assigned different groups of patients to receive different doses or preparations of the same medication. For each study with more than 1 active medication group, the greatest actual or estimated response rate using HRSD criteria in one medication group was designated the maximum medication response rate for that study. The 2 classes of medication most frequently studied were the tricyclic antidepressants (TCAs, 43 studies) and the selective serotonin reuptake inhibitors (SSRIs, 33 studies). For each study that examined a TCA and/or an SSRI, we determined the TCA and/or the SSRI response rates; for studies that assigned different groups of patients to different preparations or doses within the same class, we used data from the group with the greatest response rate.

The average maximum proportion of patients responding to medication in a study was 50.1% (9.0%) (range, 31.6%-70.4%). The proportion of patients responding to TCAs was 46.9% (10.6%) (range, 27.5%-65.6%) and to SSRIs was 48.9% (10.3%) (range, 25.0%-70.4%). The proportion of patients responding to medication was correlated with year of publication (for maximum proportion of patients responding to a medication, n = 75; r = 0.26; 95% CI, 0.03-0.46; P = .02; for TCA response: n = 43; r = 0.29; 95% CI, −0.01 to 0.54; P = .06; for SSRI response: n = 33; r = 0.47; 95% CI, 0.15-0.70; P = .006) (Figure 1). However, when significant covariates were entered into the regression models (trial length for maximum proportion responding and SSRI response; prorated initial HRSD score for TCA response), neither the relationship of medication response with year of publication nor with the covariate remained significant.

We calculated effect size, the difference between the response to medication and the response to placebo, using an arcsin transformation. The average effect size across studies for the maximum proportion of patients responding to a medication was 0.43 (0.22) (for TCA response, 0.38 [0.22]; for SSRI response, 0.40 [0.24]). There were no statistically significant correlations between any of these effect sizes and year of publication (for maximum proportion of patients responding to a medication, n = 75; r = −0.17; 95% CI, −0.38 to 0.06; P = .14; for TCA response: n = 43; r = −0.09; 95% CI, −0.38 to 0.22; P = .56; for SSRI response: n = 33; r = −0.16; 95% CI, −0.48 to 0.19; P = .38).

We were interested in determining the week at which a significant difference in the mean HRSD score between medication and placebo groups was first noted. Fifty-three reports found a putative antidepressant to be superior to placebo, and 40 of these provided information, usually a graph, that allowed determination of the first week at which a statistically significant difference (P<.05) between mean HRSD scores of medication and placebo groups was noted. If data on several medications or medication doses were included in one report, we used the week of first detectable medication vs placebo difference for the medication or dose with the greatest overall response rate. A significant drug effect was first noted, on average, 2.40 (1.48) weeks (range, 1-7 weeks) following randomization, and, in 92% (37 of 40 studies), by 4 weeks after randomization. In 19 studies that provided data on the time course of response to TCAs, a medication-placebo difference was noted in 2.53 (1.54) weeks (range, 1-6 weeks), and, in 19 studies with data on SSRIs, a medication-placebo difference was noted in 2.74 (1.63) weeks (range, 1-7 weeks).

This review of 75 placebo-controlled trials of medication for adult outpatients with MDD published between 1981 and 2000 leads to several conclusions relevant to considering the inclusion of placebo groups in future studies of antidepressant medication. First, the response to placebo across the trials varied, ranging from approximately 10% to more than 50%, and frequently was substantial; in approximately half of the studies, 30% or more of the patients assigned to placebo exhibited a clinically significant improvement. Second, in the last 2 decades, the proportion of patients responding to placebo has clearly increased, at the rate of approximately 7% per decade, and a similar increase has occurred in the fraction of patients responding to active medication. The effect size in published studies has not changed significantly, emphasizing the importance of examining unpublished data to determine whether the increased rate of placebo response may have contributed to the difficulty of establishing the efficacy of new antidepressants.^18,100

The change in placebo response rate does not appear to be directly explained by changes in study characteristics. Although the diagnostic systems used have shifted with time as psychiatry has adopted the American Psychiatric Association's editions of the Diagnostic and Statistical Manual of Mental Disorders, all the systems are close descendants of the RDC, and the specific criteria are extremely similar in the nature and duration of the symptoms required for diagnosis of MDD. Thus, there is little reason to think that the shift from RDC to Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition during the last 20 years is responsible for changes in response rates. The average age of patients has increased, but the relationship between age and placebo response rate was not significant. The length of randomized controlled trials has increased, and we found, as have others, that the proportion of patients responding to placebo increases with trial length.^101- 103 Presumably, this association reflects both the cumulative effects of the nonspecific interventions inherent in clinical trials and a longer period during which spontaneous recovery could occur. We also found that, after prorating for the number of items in the HRSDs used in different studies, more recent studies required a higher minimum HRSD score for study entry and that there was a significant positive correlation between minimum entry score and placebo response rate. However, in a multiple regression model that included year of publication, length of trial, and minimum required HRSD score, only year of publication was a significant predictor of placebo response. It is of note that the average placebo response rate of studies that used a placebo lead in did not differ significantly from that of studies that did not use a placebo lead in.¹⁰⁴ Data on other potentially relevant characteristics, such as duration of illness or number of prior episodes of depression, were not reported in a sufficiently similar form across studies to permit evaluation of their impact on placebo response rate.

A recently published meta-analysis¹⁰⁵ found little evidence that the course of patients receiving a placebo can be distinguished from the course of patients receiving no treatment, suggesting that the effect of placebo is usually no greater than the effect of the passage of time. This conclusion was not as clear-cut for studies examining more subjective outcomes such as pain, and the meta-analysis located only a few studies of the short-term treatment of MDD. Therefore, it is unclear whether placebo, when used in studies of MDD, has a greater effect than no treatment. In either case, the current review indicates that the impact of placebo in trials of MDD, even if it reflects no more than the natural course of untreated MDD, has changed in the last 2 decades. In addition, most studies permit the use of sedatives and anxiolytics, so that what is reported as response to placebo typically encompasses the impact of such agents.

Our review found that the response rates to both placebo and active medication in trials of MDD have increased significantly in the last 20 years and that only year of publication is a significant predictor of placebo response when other potentially relevant factors are taken into account. Some factor or factors associated with the level of placebo response must therefore have changed significantly during this period. Unfortunately, we were not able to determine the identity of these factors. The only characteristic of MDD available across studies was the prorated average initial HRSD score, which, despite an increase in the minimum HRSD score required for study entry, did not change significantly in the last 20 years. Because the HRSD was the sole measure of illness severity, was used in 1 of 3 different forms, and can be difficult to compare across studies,¹⁰⁶ it is possible that the HRSD was not sufficiently sensitive to capture relevant changes in the characteristics of MDD.

In recent years, as effective treatments for depression have become more widely available and socially acceptable, there has been a marked increase in the proportion of the population receiving treatment for depression.¹⁰⁷ The factors responsible for this trend may also have contributed to the participation in clinical trials of individuals with milder, briefer, and more responsive forms of depression. In addition, the methods by which patients are recruited into therapeutic trials have changed. Twenty years ago, most patients were referred by other clinicians or learned of research studies while trying to obtain treatment from established centers. In more recent years, it has become commonplace for investigators to advertise the availability of research trials to the public. It is likely that clinically important characteristics of patients participating in treatment studies have changed as a result of these practices¹⁸ and have contributed to changes in placebo and medication response rates.

The variability of the response of patients with MDD to placebo across studies and the fact that the average rate of response has changed over time strongly argue against the use of a historical standard as a valid benchmark against which to measure the efficacy of new antidepressants. Since the response to placebo is variable, often substantial, and increasing, it is not surprising that in many randomized controlled trials the response associated with placebo is similar to that associated with an established antidepressant.^18,100,108 In addition, the association between medication response rate and year of publication was not as robust as that of placebo. These facts, coupled with the inability to predict accurately the response rate to placebo or medication on the basis of study characteristics, raise serious doubts about the scientific validity of trials of new medications in which there is no placebo group and the only criterion for antidepressant efficacy is a response statistically indistinguishable from that obtained with an accepted antidepressant.⁵

Among trials that ultimately detected a difference between the active medication group and the placebo group, a statistically significant difference in mean HRSD score was apparent soon, usually by 3 weeks and almost always by 4 weeks after randomization. This observation was true even for trials of SSRIs, which may require longer treatment to be fully effective.¹⁰⁹ Thus, although the magnitude of the difference in response between medication and placebo generally increases with length of study,^60,103 the current review found, as have others,¹¹⁰ that a statistical difference in HRSD score was detectable early. Because an early drug-placebo difference in the HRSD score may only reflect improvement in depressive symptoms, such as sleep disturbance and agitation, a statistically significant difference in mean HRSD scores is likely to be well short of a clinically significant difference. In addition, sufficient data were not generally available to assess the change in effect size during a study, and none of these studies was specifically designed to determine first onset of antidepressant activity.¹¹¹ Nonetheless, the observation that a difference between the effects of antidepressant medication and placebo can be detected in less than 6 weeks suggests that it may be possible to limit the length of placebo-controlled trials of new antidepressant medications. Such changes would both reduce the duration of time patients receive placebo and help minimize the risks of such treatment.

Our study has several limitations. We examined only published data. For a number of reasons, including the fact that most placebo-controlled trials of antidepressant medications are supported by the pharmaceutical industry, studies in which a significant drug-placebo difference is detected are presumably much more likely to be published than ones in which the responses to placebo and active medication are statistically indistinguishable. Therefore, it seems probable that the placebo response rates in the published studies we examined are lower, and the medication response rates higher, than those in unpublished studies. Several authors^{100- 101,112} have recently begun to access data submitted to the FDA, much of which is unpublished. Until these and other unpublished studies are examined, uncertainty will persist about changes in the response to placebo and to medication over time. In addition, the variable delays in publication of results could bias the results regarding date of publication. Another limitation is that the results presented herein are based on somewhat arbitrary criteria for clinical response. We used a 50% decrease in HRSD score and/or a CGI improvement score of 1 or 2 as the criteria for response because they are routinely reported and generally accepted, for example, by the FDA. It would be of interest to know whether the results we describe are reflected in other measures of clinical response. In addition, because the focus of this review is on changes in the characteristics of studies of MDD during the last 2 decades, the unit of analysis is the study, not the patient. As a consequence, we have very limited ability to examine the relationship between patient characteristics and the likelihood of response. For example, we could examine only the relationship between average patient age and average response rate across studies, not the relationship between an individual's age and the likelihood that he or she would respond.

In summary, our review of recent placebo-controlled trials of medication for MDD indicates that the response to placebo is variable but often substantial and has increased in recent years. These data support the view^5,8,15,112 that the inclusion of a placebo group has major scientific importance in trials of new antidepressant medications and indicate that efforts should continue to minimize the risks of such studies so that they may be conducted in an ethically acceptable manner.