Effectiveness of a Quality Improvement Intervention for Adolescent Depression in Primary Care Clinics:  A Randomized Controlled Trial FREE

Lifetime prevalence for major depression in adolescence is estimated at 15% to 20%,¹ current prevalence is estimated as high as 6%,² and 28.3% of adolescents report periods during the past year of depressive symptoms leading to impairment.³ Untreated depression is associated with suicide, a leading cause of death for youth aged 15 to 24 years,^4- 5 and with other negative outcomes including school dropout, pregnancy, substance abuse, and adult depression.^2,5- 9

The treatment literature supports efficacy for cognitive-behavior therapy (CBT),^10- 14 interpersonal psychotherapy,^14- 16 and some selective serotonin reuptake inhibitors,^17- 21 with recent data indicating an advantage of combined CBT and medication for the treatment of adolescent major depression.²¹ Practice parameters have been developed and algorithms tested to guide pharmacotherapy.^22- 24 However, due to uncertainty regarding the safety and efficacy of selective serotonin reuptake inhibitors in youth,^25- 26 the US Food and Drug Administration recently conducted hearings regarding treatment of adolescent depression and directed a black box warning in the labeling for certain antidepressants to encourage close observation for worsening depression, suicidality, or both.²⁷

These advances have had limited impact on community care, with current data indicating high unmet need^28- 30 and poorer quality and outcomes for community treatment compared with efficacy studies.^31- 32 We address these gaps by evaluating a quality improvement intervention aimed at improving access to evidence-based treatments for depression (particularly CBT and antidepressant medication) in primary care settings. We chose primary care settings for this study because they are major points of health service contact³³ and provide valuable opportunities for effective care for depression but are characterized by low detection and treatment rates for depression among youth.²⁸ We focus on youth with depressive disorders and youth with subsyndromal depressive symptoms. The latter group was included because youth with subsyndromal depression show impairments comparable to those seen in depressive disorders and have increased risk of depressive disorder onset, and because cognitive-behavioral interventions have been shown to be effective in preventing depressive disorder onset.^34- 35

We hypothesized that the intervention would improve use of evidence-based treatments, depression outcomes, mental health–related quality of life, and satisfaction with mental health care after the 6-month intervention period. The quality improvement intervention was compared with usual care.

The Youth Partners-in-Care (YPIC) study is a multisite randomized effectiveness trial comparing the quality improvement intervention with usual care. The study protocol was approved by the institutional review boards from all participating organizations. All participants and parents or legal guardians for youth younger than 18 years provided written informed consent or assent, as appropriate.

Six study sites were selected that represented 5 health care organizations, purposively selected to include public sector (2 sites), managed care (2 sites from 1 organization), and academic health programs (2 sites). Participants were recruited through screening consecutive patients. Screening procedures and results are described in detail elsewhere.³⁶ Following common adolescent medicine practices,³⁷ we defined adolescence broadly. Inclusion criteria for screening were age 13 through 21 years and presenting at clinic for primary care visit. Exclusion criteria included having previously completed screening, not English-speaking, clinician not in the study, and sibling already in the study. Across sites, 4750 youth were eligible for screening during the recruitment period (Figure 1).

Patients completed brief self-administered screening questionnaires in the clinics. Enrollment eligibility was based on youth meeting either of 2 criteria: (1) endorsed “stem items” for major depression or dysthymia from the 12-month Composite International Diagnostic Interview (CIDI-12 [Core Version 2.1])³⁸ modified slightly to conform to diagnostic criteria for adolescents,³⁹ 1 week or more of past-month depressive symptoms, and a total Center for Epidemiological Studies-Depression Scale (CES-D)⁴⁰ score of 16 or greater (range of possible scores, 0-60); or (2) a CES-D score of 24 or greater. The screening questionnaire did not ask about suicidality.

Of 4750 youth eligible for screening, 4149 (87%) began screening, and 4002 (84%) completed screening. Roughly a quarter (1034/4002 [26%]) met enrollment eligibility criteria. Among those, 418 (40%) enrolled in the study, completed the baseline assessment, and were randomized. Among remaining eligible youth, 259 could not be contacted, 123 actively refused the study, and 234 passively refused by not providing consent (166) or baseline assessments (68).

After completing the baseline assessments, participants were randomly assigned to receive the quality improvement intervention or the usual care condition using a computerized random number generator. To improve balance across conditions in terms of clinician mix and patient sequence, we stratified participants by site and clinician and blocked participants recruited from the same clinician in pairs according to the time of their enrollment (98% [409/418] of patients had primary care clinicians [n = 52] with patients in both conditions). Screening/enrollment staff were masked to randomization status and sequence and were different from assessment staff. There was also a time delay between screening and randomization (median, 21 days). These design features prevented protocol subversion due to selection bias in enrollment that might occur with blocked randomization⁴¹ ; we also applied the Berger-Exner test⁴² to confirm this expectation.

Among the 418 youth enrolled, 344 (82%) completed the 6-month follow-up assessment. Follow-up rates did not differ significantly across conditions (81% in quality improvement vs 84% in usual care; P = .36).

The usual care condition was enhanced by providing primary care clinicians with training and educational materials (manuals, pocket cards) on depression evaluation and treatment.⁴³ Patients receiving usual care had access to usual treatment at the site but not to the specific mental health providers trained in the CBT and care management services used in the study. Throughout all phases of the study (including screening), all patients were reminded that the clinics/clinicians were participating in this project because they were interested in how the youths were feeling and that it was important for them to talk to their physicians or nurses about any difficulties, including problems with stress or depression. Serious concerns were communicated to clinicians, and procedures were established to address emergency situations and facilitate care for patients seeking care or information.

The quality improvement intervention included (1) expert leader teams at each site that adapted and implemented the intervention; (2) care managers who supported primary care clinicians with patient evaluation, education, medication and psychosocial treatment, and linkage with specialty mental health services; (3) training of care managers in manualized CBT for depression; and (4) patient and clinician choice of treatment modalities (CBT, medication, combined CBT and medication, care manager follow-up, or referral). The study informed primary care clinicians regarding patient participation only in the quality improvement condition.

Care managers were psychotherapists with master’s or PhD degrees in a mental health field or nursing. The study provided a 1-day training workshop on the study CBT and the study evaluation and treatment model, detailed manuals, and regular consultation to support fidelity to the treatment model and provide case-specific training in CBT and patient outreach/engagement strategies.

Quality improvement patients and their parents (when appropriate) were offered a free clinic visit with the care manager (Figure 2). This visit emphasized evaluation of patient and family needs, education regarding treatment options, and clarification of preferences for different treatment options. A treatment plan was developed, finalized with the primary care clinician, and modified as needed (eg, if a patient started on CBT showed only a partial response, the care manager encouraged another primary care clinician visit to consider medication). Care managers followed up with patients during the 6-month intervention period, coordinated care with the primary care clinician, assisted the clinician in patient management, delivered the CBT, and incorporated CBT components into briefer follow-up contacts. The study CBT was based on the Adolescent Coping With Depression Course,⁴⁴ developed for individual or group sessions and adapted to enhance feasibility within primary care practice settings. This manualized CBT⁴⁵ included a session introducing the treatment model, three 4-session modules emphasizing different CBT components (activities/social skills, cognition, and communication/problem-solving), and a final session emphasizing relapse prevention and follow-up care. Sessions were designed to be 50-minute weekly sessions. The Texas Medication Algorithms for Major Depressive Disorder²³ guided medication treatment and emphasized selective serotonin reuptake inhibitors as the first-stage medication choice. Additional description of the intervention is provided elsewhere.^46- 47

Youth baseline and 6-month follow-up assessments were conducted by interviewers from the Battelle Survey Research Institute who were masked to intervention assignment and used computer-assisted telephone interviews. Interviewers continued attempts to contact participants until an active refusal was obtained or it became clear that the participant could not be contacted. Interviewers were trained and supervised by senior staff with official CIDI and Diagnostic Interview Schedule training and more than 10 years of experience in conducting CIDIs and the Diagnostic Interview Schedule. Interview quality was rated on 10% of interviews for accuracy in presenting questions, probing, and coding; ratings indicated good quality (3-point scale, 1 = highest rating; mean, 1.02 [SD, 0.06]). Emergency procedures were developed with each site, and clinicians were available to address any emergencies or issues of serious concern (eg, report of current suicidality, danger to self or others). Assessments concluded with a reminder to patients that their physicians or nurses wanted them to call if they had any problems or difficulties, and contact or referral information was provided as needed.

Youth baseline and follow-up interviews assessed mental health–related quality of life using the Mental Health Summary Score (MCS-12) (range of possible scores, 0-100),^48- 49 overall mental health using the Mental Health Inventory 5 (MHI-5) (range of possible scores, 5-30),⁵⁰ service use during the previous 6 months using the Service Assessment for Children and Adolescents⁵¹ adapted to incorporate items assessing mental health treatment by primary care clinicians,⁵² and satisfaction with mental health care using a 5-point scale ranging from very dissatisfied (1) to very satisfied (5).⁵³ CIDI diagnoses of major depression and dysthymia were evaluated at baseline and follow-up. To capture a broad range of youth depression, depressive disorder was diagnosed regardless of history of manic symptoms. The CES-D was administered at follow-up. Sociodemographic characteristics were assessed at baseline. Ethnicity and race were self-classified to clarify minority representation in the sample.

The primary outcome variable was CES-D total score. To clarify clinical significance, we also examined the proportion of youth scoring in the severe range (CES-D score ≥24). Secondary outcomes were MCS-12 scores and satisfaction with mental health care. Process-of-care measures included rates of mental health care, psychotherapy/counseling, and medication for mental health problems. Because the CIDI-12 asked about the interval between baseline and 6-month assessments, changes in depression diagnosis were not predicted.

We examined the demographic and baseline clinical characteristics of the enrolled sample, and compared the quality improvement and usual care groups to assess the balance across experimental groups at baseline using t tests for numerical variables and χ² tests for categorical variables (Table 1). We also conducted the Berger-Exner test⁴² for selection bias not captured by observed baseline characteristics.

To evaluate the effectiveness of the intervention, we conducted intent-to-treat analyses with the intent-to-treat population for follow-up outcome measures. Patients were analyzed according to the experimental group they were assigned to, irrespective of whether they received treatment or used study resources such as care management. We fitted analysis of covariance models for continuous outcomes, and logistic regression models for dichotomous outcomes, with intervention status as the main independent variable and the baseline measure for the same outcome as the covariate. However, for follow-up CES-D score, we used baseline MHI-5 score as the covariate, because CES-D score was not measured at baseline. (CES-D and MHI-5 scores were highly correlated at follow-up [r = 0.78, P<.001]; therefore, baseline MHI-5 score was used here as the proxy measure for baseline CES-D score.) Intervention status and baseline measure were both specified as fixed effects. To show effect sizes, we present unadjusted means and proportions by intervention groups, as well as adjusted differences or odds ratios (ORs) that are adjusted for the baseline measure. We also conducted sensitivity analyses for intervention effects using a design-based nonparametric method, the permutation test, to ascertain whether our findings are sensitive to model assumptions.^54- 56

We used nonresponse weighting^57- 58 to address missing data for the 18% of patients who did not complete 6-month follow-up assessments. The objective of nonresponse weighting is to extrapolate from the observed 6-month sample to the original intent-to-treat sample. Nonresponse weights were constructed by fitting logistic regression models to predict follow-up status from baseline clinical and sociodemographic characteristics. Separate models were fitted for each intervention group. The reciprocal of the predicted follow-up probability is used as the nonresponse weight for each participant. Intent-to-treat analyses for intervention effects, weighted by nonresponse weights, were conducted using survey commands in STATA version 8.⁵⁹ Weighted and unweighted analyses yielded very similar results. We report only results from weighted analyses.

We used 2-sided P values of less than.05 as the criterion for statistically significant differences. We used multivariate analysis of variance to combine the results across primary outcome variables to ascertain the potential for spurious significance due to multiple comparisons.

The enrolled sample was clinically and sociodemographically diverse (Table 1). Most patients were female (78%), ethnic minorities (87%), spoke a language other than English at home (64%), and had at least 1 working parent (89%). The sample included those with depressive disorders (43%, primarily major depression [42%]) and those with subsyndromal depression (57%). Among youth with major depression, 60% had CIDI-defined moderate to severe illness, 29% had recurrent illness, 3% had comorbid dysthymia, and 15% had a history of manic episodes. Dysthymia without another mood disorder was rare (<1% [3/418]), as was bipolar disorder without a past-year depressive episode (1.7% [7/418]). Comorbid mental health symptoms were common: 28% of youth reported significant externalizing symptoms or conduct problems (eg, disobedient, stealing, aggression),⁶⁰ 22% screened positive for posttraumatic stress disorder,⁶¹ 25% endorsed 1 or more indicators of problematic substance use,⁶² 27% reported suicidal ideation,⁶⁰ and 13% reported suicide attempts or deliberate self-harm (defined as some suicidal ideation plus some suicide attempt or deliberate self-harm during the previous 6 months on the Youth Self Report).⁶⁰ About 22% reported specialty mental health care and psychotherapy/counseling in the past 6 months, and 16% reported medication treatment in the past 6 months. Medication treatment was more common in youth with depressive disorders vs those with subsyndromal depression (OR, 4.55; 95% confidence interval [CI], 2.54 to 8.16; P<.001). Depression was detected at the index primary care visit in 19% of youth, based on youth report of depression counseling during this visit.

There were no significant differences between the quality improvement and usual care groups at baseline. Most differences were far from being statistically significant, except for a near-significant trend for MCS-12 score (P = .08). The Berger-Exner test for selection bias was insignificant for all outcome measures (P = .52 for CES-D score, P  = .48 for MCS-12 score, and P  = .35 for satisfaction with mental health care).

At 6-month follow-up, patients receiving the quality improvement intervention reported significantly higher rates of mental health care than did those receiving usual care (32% vs 17%; OR, 2.8; 95% CI, 1.6 to 4.9; P<.001) (Table 2). This was due to increased rates of psychotherapy in the intervention group, as the difference for medication treatment was small and statistically nonsignificant (Table 2). Rates of combined medication and psychotherapy were similar for quality improvement (10%) and usual care (12%) patients. No between-group differences were found in rates of combined treatment vs monotherapy (medication or psychotherapy, OR, 1.4; 95% CI, 0.6 to 3.6; P = .43) or in rates of combined treatment vs no treatment (OR, 1.5; 95% CI, 0.6 to 3.4; P = .40). Quality improvement patients had a higher rate of monotherapy (23% quality improvement vs 14% usual care) vs no treatment (OR, 2.1; 95% CI, 1.1 to 3.8; P = .02), again due to a higher rate of psychotherapy in the quality improvement group. Quality improvement patients reported more psychotherapy sessions than did patients receiving usual care, but relatively few patients reported 12 or more sessions (Table 2). Medication treatment was more common among youth with depressive disorders (OR, 3.1; 95% CI, 1.6 to 6.0; P<.001). Similar rates of mental health care from primary care clinicians at baseline and follow-up indicated that the intervention primarily affected use of a care manager or mental health services (Table 2). These self-report data were consistent with chart-review data indicating higher rates of care manager/CBT contacts vs medication in quality improvement patients (44% vs 13%); 34% of quality improvement patients received in-person CBT based on chart review.

At the 6-month follow-up, quality improvement patients had significantly lower mean (SD) CES-D scores compared with usual care patients (19.0 [11.9] vs 21.4 [13.1], P = .02) (Table 3). This improvement among intervention patients was also reflected in a significantly lower rate of severe depression (CES-D score ≥24) in the quality improvement group (31% vs 42%; OR, 0.6; 95% CI, 0.4 to 0.9; P = .02). Quality improvement patients reported higher mental health–related quality of life (measured as mean [SD] MCS-12 score) compared with usual care patients (44.6 [11.3] vs 42.8 [12.9], P = .03), as well as greater satisfaction with mental health care (3.8 [0.9] vs 3.5 [1.0], P = .004). The P value combining the results for CES-D score, MCS-12 score, and satisfaction, using multivariate analysis of variance, was.004, indicating that these findings are not of spurious significance due to multiple comparisons.

We conducted a number of sensitivity analyses on the specifications for the analytic approach. First, we used a design-based nonparametric method, the permutation test, to ascertain whether our findings are sensitive to model assumptions.^54- 56 Results were similar to those reported above (P = .02 for CES-D score, P = .03 for MCS-12 score, and P = .003 for satisfaction with mental health care). Second, we examined unweighted analyses without incorporating attrition weights; results were similar to the analyses reported above (P = .02 for CES-D score, P = .02 for CES-D severe range, P = .03 for MCS-12 score, and P = .008 for satisfaction with mental health care). Third, we examined weighted analyses that incorporated attrition weights and enrollment weights (based on the probabilities of screening and enrollment) to account for nonresponse that occurred before baseline/randomization. Results were again similar (P = .02 for CES-D score, P = .02 for CES-D severe range, P = .05 for MCS-12 score, and P = .03 for satisfaction with mental health care). Thus, our findings appear robust using parametric and nonparametric analyses and weighted and unweighted analyses.

Due to current questions regarding the impact of treatment on youth suicidality, we conducted exploratory analyses examining intervention effects on youth-reported suicidal ideation and suicide attempts or deliberate self-harm. There were no significant intervention effects on either measure. The number of patients reporting suicide attempts or deliberate self-harm declined from 14.2% at baseline to 6.4% at 6 months in the quality improvement group and from 11.6% to 9.5% in the usual care group. However, the difference between quality improvement vs usual care at 6 months is statistically nonsignificant (OR, 0.55; 95% CI, 0.23 to 1.34; P = .19). This is an important subject for future studies with larger samples powered specifically to address this question.

This is the first demonstration that depression and quality-of-life outcomes can be improved through a quality improvement intervention for depressed adolescents in primary care settings. Building on prior demonstrations of improved outcomes from quality improvement interventions for adult and late-life depression,^52,63 our results indicate that this approach can be adapted successfully for younger populations with similar outcomes. Both the YPIC study and the adult Partners-in-Care Study⁵² achieved a roughly 10 percentage-point difference in the percentage of patients falling in the clinically significant range on the CES-D as well as achieving clinically meaningful improvements in mental health-related quality of life. Because evidence supporting depression treatments is less established for adolescents than for adults, it is noteworthy that similarly designed quality improvement interventions are effective in youth, adults, and elderly persons.^52,63

Despite increases in youth antidepressant use and primary care clinician prescriptions for antidepressant medications in the past decade,^64- 66 our results indicate that when both psychotherapy and medication were available options within primary care, psychotherapy (the more difficult option) was generally preferred. Unlike adult studies in which medication rates were higher and increased with quality improvement interventions,^52,63 our intervention did not increase medication rates, despite intervention support of medication treatment. Because the study preceded recent warnings regarding use of antidepressant medications among adolescents,²⁷ our findings were not due to this public controversy and suggest a developmental difference. This reinforces the importance of resources to enable evidence-based psychotherapy in quality improvement programs for adolescent depression in primary care settings.

Our intervention replicated key features of routine community practices: specialties usually treating adolescents (pediatrics, family medicine, adolescent medicine), diverse patients seen in clinics, and patient and clinician choice of treatment. Under these naturalistic conditions, patients and clinicians elected relatively low levels of treatment, with most patients receiving care manager follow-up or low doses of CBT. This led to modest reductions in depression, compared with efficacy studies that tested more intensive interventions with restricted patient populations under controlled conditions. However, our effects were similar to those in other quality improvement effectiveness trials.⁵² Intervention effects were also averaged across the entire quality improvement group (including untreated patients), and patients in the usual care group were free to receive “usual care” treatments, likely attenuating intervention effects.

What can be accomplished with a quality improvement demonstration vs a clinical efficacy study in which treatments are assigned? Our quality improvement study asks: what can primary care practices accomplish by making it easier for clinicians and patients to understand and select evidence-based depression treatments? The YPIC study provided resources and information to encourage patients and clinicians to select evidence-based treatments. Using standardized effect sizes, outcome effects are small compared with those from efficacy studies; in absolute magnitude, however, observed differences were similar to 6-month intervention effects in the adult Partners-in-Care Study⁵² and are of public health significance given the prevalence and morbidity of adolescent depression.

A portion of the targeted sample was lost during screening/recruitment/enrollment procedures, compromising the generalizability of study findings. To some extent, incorporating enrollment weights to account for preenrollment sample loss can mitigate this limitation. Our sensitivity analyses that incorporated enrollment weights yielded results similar to those from our primary analyses (both analyses incorporated attrition weights for postrandomization sample loss). To the extent that enrollment weights capture differences between the enrolled sample and nonenrolled eligible youth, this supports the generalizability of our findings to similar primary care practice. However, enrollment weights may not capture all differences; for instance, participation may have been higher for youth with a preference for psychotherapy due to generally more limited access to psychotherapy vs medication.

The study had other limitations. Because primary care clinician training in care of depression was common to all patients, the YPIC study tests the marginal benefit of the full quality improvement intervention vs usual care “enriched” by primary care clinician education. Although prior research suggests minimal impact for clinician education alone,⁶⁷ this provides a conservative test of the intervention. Primary care clinicians may also learn from experiences with their quality improvement patients and carry this learning over to patients in the usual care group, again resulting in a conservative estimate of the intervention effect. We selected sites to represent a range of practice conditions, but sites were not selected at random. Although our sample is diverse and includes large numbers of minority youths, results may not be generalizable across all ethnic groups, geographic locations, and practice settings. Assessments emphasized youth self-report but with established reliable measures.^49- 51,68 Data on longer-term outcomes are needed to clarify the sustainability of intervention effects after discontinuing intervention resources. Despite significant intervention effects, almost a third of quality improvement patients continued to show severe depressive symptoms. The availability of psychotherapy may have led to substitution of psychotherapy for medication, and emphasizing combined psychotherapy and medication might have led to improved outcomes.²¹ Our effectiveness design, which encouraged but did not require treatment fidelity or adherence, likely weakened intervention effects. Because the study supported screening, primary care practices would have to screen patients to implement the intervention independently. The intervention effect included the effect of improved detection, although the literature suggests that detection without additional practice resources to support mental health care has little impact on outcomes.^69- 70

Since our goal was to improve access to care with patients choosing a range of specific treatments, the study does not provide information on the effects of specific treatments (CBT, medication). The fact that our intervention impacted rates of psychotherapy but not medication use suggests that psychosocial interventions contributed to improved patient outcomes. However, it could be that even with low medication rates, allowing patients and clinicians to select preferred treatments contributed to improved matching of patients to treatments that were most likely to be effective for them. This is consistent with our finding that youth with depressive disorder, the group with greatest need, was most likely to receive medication treatment.

In conclusion, the present results demonstrate that quality improvement interventions for adolescent depression are feasible in primary care settings and associated with benefits on measures of depression, quality of life, and satisfaction with mental health treatment. Our quality improvement model and results are consistent with the recommendation of the US Preventive Services Task Force⁷¹ that depression screening in primary care is effective when combined with access to treatments such as those provided in the YPIC trial.