Impact of Disseminating Quality Improvement Programs for Depression in Managed Primary Care:  A Randomized Controlled Trial FREE

Quality of care for depression in managed primary care settings is moderate to poor^1- 2 with resultant poor outcomes.³ Because depression is expected to be the second leading cause of disability worldwide after the turn of the century,⁴ improving care for this prevalent disorder could enhance quality of life and productivity for many primary care patients.

Clinical trials have established the efficacy of antidepressant medications and specific psychotherapies for depressed psychiatric and primary care patients when care is provided by trained research personnel or under standardized protocols.^5- 9 Trying to improve care through routine screening with feedback to clinicians on patients' depression has little effect, and an early study of implementing practice guidelines for depressed primary care patients found no outcome benefits.¹⁰ However, Katon and colleagues^11- 12 demonstrated that collaborative care models involving psychologists (providing short-term psychotherapy) and psychiatrists (consulting on medications) improve treatment rates and clinical outcomes for up to 9 months of follow-up, but lasting practice impact may depend on the continued participation of the research team.¹³ What are the impacts of quality improvement (QI) programs for depression when disseminated to managed primary care settings under naturalistic conditions that include reliance on usual care providers and free choice of treatment by clinicians and patients?

In this study, we examine this question using a randomized controlled trial of QI dissemination vs usual care. The interventions were based on literature suggesting that provider change is more likely when the following occur: (1) social-influence methods, such as local expert leaders, are used; (2) feedback on performance is provided; (3) clinician workload is shifted to ancillary staff; and (4) multiple practice-change strategies are used.^14- 16 The study fielded 2 similar interventions, one with enhanced resources for supporting medication management (QI-meds) and the other with enhanced resources for providing psychotherapy for depression (QI-therapy). Both had the same goal of improving rates of initiation and adherence to either appropriate antidepressant medication or psychotherapy for depression, and both were based on the same QI methods. The study was designed to test the primary hypothesis that dissemination of QI for depression (the combined interventions) under naturalistic conditions will improve quality of care and health-related outcomes over a follow-up year. We were less certain that QI would affect physical health and employment status within 1 year and consider analyses of these outcomes as exploratory. The study has a secondary aim of determining whether QI interventions relying on enhanced resources for medication management or for psychotherapy are each effective. Practice guidelines emphasize the effectiveness of both modalities,^5- 6,17 but primary care practitioners are less familiar with psychotherapy and its insurance coverage has become limited.¹⁸ We hypothesized that each intervention is effective and provide a preliminary analysis based on 1 year of data in this article. In addition, we test whether the combined interventions were effective for patients with either depressive disorder or depressive symptoms without disorder, a common condition in practice for which there is limited evidence for treatment efficacy.^{11- 12,15,19- 24}

The data are from Partners in Care, an Agency for Healthcare Research and Quality (formerly the Agency for Health Care Policy and Research) Patient Outcomes Research Team (PORT-II).²⁵ The study is a group-level randomized controlled trial.²⁶

We recruited 6 managed care organizations, selected to be diverse in geography and organization and to oversample Mexican Americans. The organizations included staff and network model multispecialty group practices and rural managed public health clinics (Table 1). Patients had prepaid or managed fee-for-service (including Medicare and Medicaid) coverage. Uninsured patients were included in 1 site. We approached 1 or 2 regions per organization; 1 region refused and 7 agreed. Of the 48 primary care clinics with at least 2 clinicians, 46 participated.

We grouped clinic clusters into matched blocks of 3, based on patient demographics, clinician specialty, and distance to mental health providers. We created 9 blocks, 1 each for 6 regions and 3 for the seventh (stratified by low, intermediate, or high percentage of Mexican American patients). Within blocks, we used a random number table to assign clusters to usual care or QI interventions.

We asked all primary care clinicians to enroll prior to their knowledge of intervention status; 97% (n = 181) agreed, including internists and family practice physicians (87%) and nurse practitioners (13%).

Study staff screened consecutive patient visitors over a 5- to 7-month period between June 1996 and March 1997. Patients were eligible if they were depressed and intended to use the clinic as a source of care for the next 12 months. Patients were ineligible if they were younger than 18 years, had an acute medical emergency, did not speak English or Spanish, or did not have either insurance or a public-pay arrangement that covered care delivered by the mental health specialists in the interventions (this criterion excluded all fee-for-service patients in 1 site and half of them in another). In the pilot month for the first site, patients screening positive for bipolar disorder or alcoholism were excluded, but not for the main study. Patients learned of their intervention status after enrollment.

The patient screener included the "stem" items for major depressive and dysthymic disorders from the 12-month Composite International Diagnostic Interview (CIDI)^1,27 and items assessing depressed symptoms in the past month. We defined patients as having probable disorder if they had 2 weeks or more of depressed mood or loss of interest in pleasurable activities during the last year or persistent depression over the year, plus having at least 1 week of depression in the last 30 days. Based on a sample of 1485 study patients who received the full CIDI affective disorders section, the positive predictive value of this indicator is 55%.²⁵ By definition, sensitivity is 100%, as the CIDI requires a positive stem item to score a diagnosis.

The study approached 44,052 persons, but 10,120 were ineligible, mainly because they were not patients of study providers (Figure 1). Of 27,332 patients completing the screener (85% of those approached and eligible), 3918 were potentially eligible. Many left during enrollment, 2417 were present to confirm insurance eligibility, and 241 had ineligible insurance. Of those who read the informed consent, 1356 enrolled and the remaining 21% either refused or left the clinic. We developed weights that adjust for differential probability of enrollment and include factors predicting enrollment as covariates in analyses.

Enrolled patients completed an interview including the affective disorders section of the CIDI; a telephone interview on anxiety disorder, income, wealth, and employment; and a mail survey. We obtained follow-up surveys every 6 months for 2 years. One-year data are now available for analyses. By 1 year, 59 patients (4.4%) had dropped out and 4 (0.3%) died. Patients remained enrolled regardless of survey response or treatment participation. Baseline factors associated with remaining enrolled were being younger or residing in 4 of the sites; 1 intervention (QI-therapy) had somewhat lower panel retention relative to usual care (P<.05). Response rates were 95% for the telephone interview and 88% for the survey at baseline, and 85% and 83% for the 6- and 12-month surveys. Among those enrolled at 12 months, 12-month survey response rates were higher for patients who were female, older, married, white, or college educated, and in 1 site. We weighted the data for probability of enrollment and survey response and control for predictors of retention and response in analyses.

The study was approved by the institutional review boards of all participating institutions as well as RAND. Informed consent was obtained from all patients and clinicians.

The conditions were usual care, QI-meds, and QI-therapy (combined to test our primary hypothesis). Patients and providers in all conditions had full choice of treatment and, in interventions, over use of study resources. For usual care, clinic medical directors were mailed the Agency for Healthcare Research and Quality depression practice guidelines, with quick reference guides for clinicians. The interventions were designed by the study team but implemented by the organizations with study support. Patient screening for probable depression was conducted by study staff. The intervention goal was to increase the percentage of clinically depressed patients receiving any appropriate care according to national guidelines.²⁸ The interventions use a basic QI model but different supplemental resources (Figure 2). The basic QI model has 4 components. The first is institutional commitment. The practices agreed to reserve in-kind resources amounting to one half the estimated costs of implementing the interventions and their time costs for study participation; the other half ($30,000-$72,000, depending on practice size) was paid by the study. In 1 site, the affiliated managed carve-out behavioral health plan paid all practice costs. The second component is training local leaders, including a primary care clinician, a nursing supervisor, and a mental health specialist, to implement the interventions. The study provided a 2-day workshop that reviewed the study's manualized model of treatment and principles of collaborative care.^29- 30 Prior to patient recruitment, these leaders distributed clinician manuals, initiated monthly lectures (80% of primary care clinicians attended at least 1 lecture), and provided academic detailing as needed (provided to 48% of clinicians). In addition, based on monthly team meetings using intervention staff records, the leaders provided audit with feedback on the clinic or individual clinician level (provided to 60% of clinicians). The third component is study training of local staff. Nurses were trained in a 1-day workshop to provide brief clinical assessments, patient education, and activation based on a written manual and videotape. The local leaders supervised nurses and therapists, and the study reviewed progress with leaders in monthly telephone calls. Supporting materials developed by the study were provided to the local experts. These included lecture slides, manuals, and pocket-reference cards for clinician education; pamphlets and videotapes for patient education; and study charts and forms for nurses and therapists to note contacts and provide feedback to patients and clinicians. The fourth component is patient identification. Intervention clinics were provided with lists of their study patients, while usual care clinic patients were told they could inform their provider, but the study did not notify their clinic.

The nurses were asked to contact intervention patients for an initial visit (assessment, education, and activation) 2 weeks after the screening visit. The primary care clinician was asked to consider this information in formulating a treatment plan with the patient. Nurse visits had no co-payment.

In the QI-meds intervetion, the nurse specialists were trained to provide follow-up assessments and support adherence through monthly contacts for 6 or 12 months (randomized at the patient level). Patients in the QI-meds intervention had access to usual practice therapists, but not the study cognitive behavioral therapy (CBT).

In the QI-therapy intervention, the study trained local psychotherapists to provide manualized individual and group CBT³¹ for 12 to 16 sessions. The organizations reduced the therapy co-payment to the primary care visit co-payment (ie, $0-$10 instead of $20-$30). Patients could access other therapy for the usual co-payment. Brief (4-session) CBT was suggested as an option for patients with minor depression.

Health Outcomes. (1) We assessed depressive symptoms at baseline and follow-up using a 23-item version of the Center for Epidemiologic Studies Depression (CES-D) scale,³² developed by Daniel Ford (written communication, March 1996). This version drops 6 items and adds others to approximate Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) criteria. Item responses were summed. We fielded the expanded and traditional versions in the same survey and then used the Samejima graded Item Response Theory model³³ to determine that a cut point of 20 is equivalent to the standard cut point of 16 for identifying probable depression. We used an indicator for being above or below that cut point. The power calculations for the study were based on expecting a 10% to 15% difference by intervention status on this measure. (2) On each follow-up survey, we repeated the screener measure of probable disorder, referring to the prior 6 months and deleting the dysthymia stem item. (3) Health-related quality of life (HRQOL) was measured in the screener and follow-up by the physical and mental health summary scales from the Short-Form 12 (SF-12).^34- 35 Scores were normalized to a general population mean (SD) of 50 (10), with higher scores indicating better health. We conducted an analysis using a physical limitations score derived from the SF-12 items, with no change in conclusions over the summary scale.

Health Service Utilization. The surveys assessed the total medical visits, medical visits for emotional problems, and mental health specialty visits during the prior 6 months. We developed indicators of each type of visit and counts of visits among users. To reduce skewness, we transformed counts (cubic root for total visits, negative reciprocal of square root for medical visits for emotional problems, and log transformation for specialty visits).

Quality of Care. Each survey assessed prescription medications used in the prior 30 days or for a month or more in the prior 6 months. To assess appropriate use of antidepressant medication in the prior 6 months, we applied dosage criteria from national guidelines to each medication used.^{5- 6,36- 37} We developed an indicator of whether the respondent had individual, group, or family counseling by a specialist in the past 6 months. Since patients might not receive care if they improve, we created an indicator of appropriate care that requires that patients were free of probable disorder or had either specialty counseling or use of appropriate antidepressant medication during a given 6-month period.

Employment. We used items from the screener and the telephone survey to assess whether respondents were employed at baseline. The follow-up surveys assessed current employment.

Covariates. From the patient screener, we measured age (and when appropriate, age squared), sex, education (less than high school, completed high school, some college, completed college or more), and a count of having 0, 1, 2, or more than 2 chronic medical conditions, from a total of 19. We used data from the screener and CIDI to categorize patients as having 30-day symptoms plus either 12-month (major depressive or dysthymic) disorder, lifetime disorder only, or no disorder (ie, symptoms only). We identified probable lifetime disorder from a supplemental CIDI item. Using items modeled after the Health and Retirement Study,³⁸ we developed a household-wealth variable, summing the net value of home and other assets.

We conducted patient-level, intent-to-treat analyses. To test the primary hypotheses for each dependent variable, we estimated multivariate regression models with combined intervention status as the main independent variable, with the covariates listed above plus global HRQOL at the screener. For analyses using the baseline survey, we also included an indicator of whether that survey was completed within 30 days of the screener. For dichotomous outcomes, we estimated logistic regression models. For levels of visits among users, we estimated linear models using transformed counts of visits. For continuous HRQOL outcomes, we conducted ordinary least squares regression on untransformed scores and analyzed end status and change scores; results were comparable, so we present end status for ease of interpretation. To test secondary hypotheses, we interacted the combined intervention status with disorder status (specified as 12-month or lifetime disorder vs symptoms only) and separate indicators for QI-meds and QI-therapy, each relative to usual care. The patients are clustered within providers and clinics, but the intraclass correlations are close to zero, so we report results unadjusted for cluster effects.

We present standardized predictions of the intervention effects for each outcome. We used the regression parameters and each individual's actual value for all covariates, other than intervention status, to generate a predicted value for each individual, first as an intervention subject and then as a control subject. We then averaged the intervention and control predictions. Data are weighted for the probability of enrollment, attrition, and wave response. We used multiple imputation for missing data at the item level. We imputed 5 data sets, averaged predictions, and adjusted SEs for uncertainty due to imputation.^39- 41 Because we report many results in the same direction as hypothesized, a formal Bonferroni⁴² correction for multiple statistical comparisons is too conservative. We report actual P values and interpret the results with multiple comparisons in mind, but apply a less strict standard for secondary hypotheses.

The enrolled sample included 443 usual care and 913 intervention patients. The sample was clinically and sociodemographically diverse (Table 2). About half had 12-month depressive disorder, 71% were women, and the mean (SD) age was 44 (15) years; 57% of the sample was non-Hispanic white, and 30% was Hispanic. Intervention patients were somewhat older and more likely to have a college education. While controls were less likely to have current disorder, they were more likely to have lifetime disorder; patients did not differ by intervention status in the percentage with symptoms only vs lifetime or current disorder (t = 0.19, P = .85).

Intervention and control patients did not differ at baseline in the probability of having any type of visit (Table 3). Visit counts among users did not differ by intervention status for any time period. Intervention patients were 20% more likely to have a medical visit for emotional problems in the first 6 months, but not in the second period. Intervention patients were 30% to 40% more likely to have a mental health specialty visit over each follow-up period. There were no significant differences in probability of any medical visit.

The interventions increased by 30% to 40% the rates of counseling and of appropriate use of antidepressant medication in the first 6 months and by 20% to 30% in the second period (Table 4). The effects on counseling and on medications in the first 6 months were largely due to new treatment among those not treated at baseline. In the second period, the interventions increased medication use among those initially medicated, ie, continuation therapy. The interventions increased the probability of appropriate care in each follow-up period by about 10 percentage points (each P<.001).

Intervention patients were less likely to have probable depression at 6- and 12-month follow-up by 7 to 10 percentage points. Intervention patients improved more on mental HRQOL by 1 to 2 points at each time point, but not global physical health (Table 5) or physical limitations (not shown).

Among patients initially employed, 89.7% of intervention patients and 84.7% of control patients worked at 12 months (t = 1.96, P = .05). Among those initially not working, 16.4% of intervention patients and 11.4% of control patients were working at 6 months (P>.10); by 12 months, 17% to 18% of intervention and control patients started working.

The interventions reduced the likelihood of probable disorder in the first 6 months by 10 percentage points, relative to usual care, among those with baseline 12-month or lifetime disorder (t = 3.02, P = .003); in the second 6 months, the likelihood was reduced by 19 percentage points among those with symptoms only at baseline (t = 2.60, P = .009). The interaction of intervention by disorder status was2 borderline at 12 months (t = 1.66, P = .096), with a stronger QI benefit for patients with symptoms only. In the first 6 months, QI increased appropriate care among those with 12-month or lifetime disorder at baseline (a 17 percentage-point increase over usual care, t = 4.64, P<.001), but not among patients with symptoms only (3 percentage-point decrease). The interaction was significant (t = 2.26, P = .02). In the second period, QI increased appropriate care similarly (by 9-10 percentage points relative to usual care) among those with disorder or symptoms only.

QI-therapy and QI-meds patients, each compared with controls, were 8 to 10 percentage points less likely to have probable disorder at 6 and 12 months (lowest t = 2.15, P = .03). At 6 months, rates of appropriate care were greater by 14 percentage points for QI-meds patients than controls (t = 3.81, P<.001) and by 8 percentage points for QI-therapy patients than controls (t = 2.37, P = .002). At 12 months, QI-meds patients had higher rates of appropriate care than did controls (t = 3.60, P<.001) or QI-therapy patients (t = 2.31, P = .02), who did not differ significantly from controls. At each follow-up, QI-meds patients had higher rates than controls of appropriate medication use (by 10-14 percentage points, lowest t = 3.25, P = .001) and any specialty counseling (by 10-12 percentage points, lowest t = 3.00, P = .003). At 6 months, relative to controls, QI-therapy patients were 13 percentage points more likely to have any specialty counseling (t = 3.71, P<.001) and were 5 percentage points more likely to use appropriate antidepressant medication (t = 1.68, P = .09). The increases in appropriate medication use were greater for QI-meds than for QI-therapy patients at both follow-ups (lowest t = 2.07, P = .04), and QI-meds patients were more likely than QI-therapy patients to have any specialty counseling in the second follow-up (t = 2.07, P = .04).

We found that when diverse nonacademic managed primary care practices implemented QI programs, quality of care, clinical outcome, mental health–related functioning, and retention in employment of depressed patients improved over a year. The investment required by practices to implement the interventions was modest and typical for QI projects. The study supported identification of patients with probable depression so practices would have to screen patients to implement the intervention independently. The changes in process we observed were modest, but some benefits, such as a 20% to 30% improvement in rates of probable depression, were substantial. The improvement in HRQOL, however, is small, but could reflect substantial societal impact when accumulated nationally. The persistence of benefits is longer than that observed in most treatment-efficacy studies.^5- 6 The potential for policy impact is suggested by a 5 percentage-point increase in employment retention, which may be important for employers given the high prevalence of depressive disorders and symptoms in community samples.⁴³ To our knowledge, no QI study has demonstrated improved employment, although perceived interpersonal work functioning improves with efficacious treatment for major depression.⁴⁴

The interventions improved outcomes for persons with 12-month or lifetime disorder, as well as those with symptoms only. We found early benefit for those with disorder and delayed benefit for those with symptoms only. Prior studies suggest modest benefits from treating minor depression.^15,19- 24 The interventions also increased rates of appropriate care for the disorder group in the first 6 months and for those with disorder or symptoms only in the second 6 months. Thus, the delayed outcome benefit for the symptoms-only group could be due to treatment in the second 6 months as symptoms continued or a disorder developed. While nonspecific intervention aspects could explain this effect, we think this is an unlikely explanation for prolonged benefits.

The interventions only transiently increased the probability of having a medical visit for mental health or emotional care without increasing total medical visits, and with a sustained increase in use of specialists. Thus, additional primary care visits may not be necessary to improve depression care. Instead, a collaborative partnership, such as has been found to improve quality and outcomes of care in other studies, may have developed.^11- 12 We doubt that the interventions resulted in lasting practice change, as Lin and her colleagues found little enduring practice effect from more intensive interventions.¹³ Nevertheless, we observed relatively long-term impacts on patients.

In an era of declining insurance benefits for psychotherapy, it is noteworthy that interventions improving resources for either medication management or psychotherapy were beneficial for primary care depressed patients across diverse practices using internal or carve-out specialty groups. The medication-resource intervention had more pronounced and sustained effects on our process indicators than did the psychotherapy-resource intervention, which could lead to outcome differences in subsequent years. While the relatively greater increase in appropriate medication use for patients in the medication-resource intervention compared with the therapy-resource intervention is not surprising, the greater rate of specialty counseling in the second follow-up may be, but this could simply reflect ongoing medication management by specialists. The type and content of therapy may well differ for the 2 interventions due to the CBT training that was limited to the therapy-resource intervention.

This dissemination trial is similar to a social experiment, in that it replicates naturalistic practice conditions, including usual care providers and full choice of treatments.⁴⁵ The diversity of organizations and patients in our study suggests broad potential applicability, which we cannot formally address with a limited number of organizations. Whether practices will want to use these interventions may depend on their cost-effectiveness relative to usual care. Our study has other important limitations. There was sample loss during enrollment, but only somewhat more so than in an observational study.¹ The direction of bias is for worse outcomes for the intervention subjects, so we could understate the interventions' benefits. We rely on self-report, but mostly for measures that are validated or easy to report, like employment.

Overall, our findings suggest that provider groups and managed care organizations may benefit patients, employers, and society through modest efforts to improve the quality of care for depression while retaining patient and provider treatment choice.