Improvement in Process of Care and Outcome After a Multicenter Severe Sepsis Educational Program in Spain FREE

Sepsis is one of the most prevalent diseases and one of the main causes of death among hospitalized patients.¹ Severe sepsis accounts for 1 in 5 admissions to intensive care units (ICUs) and is a leading cause of death in noncardiac ICUs.^2- 3 In Spain, the incidence of severe sepsis is 104 cases per 100 000 adult residents per year with a hospital mortality of 20.7%, and the incidence of septic shock is 31 cases per 100 000 adult residents per year with a mortality of 45.7%.⁴ In the United States, the incidence of severe sepsis is higher (300 cases per 100 000 population) as is the mortality rate (28.6%), which represents 215 000 deaths annually.¹

Early appropriate antibiotic therapy,^5- 7 early goal-directed therapy (EGDT),⁸ corticosteroids,⁹ recombinant human activated protein C or drotrecogin alfa (activated),¹⁰ and lung protective strategies¹¹ have all been associated with survival benefits. These and other therapeutic advances led to the development of the Surviving Sepsis Campaign (SSC) guidelines¹² as part of a plan to reduce severe sepsis mortality by 25% by 2009. For improving sepsis care, the SSC and the Institute for Healthcare Improvement recommend implementing a 6-hour resuscitation bundle including lactate determination, early cultures and antibiotics, and EGDT, as well as a first 24-hour management bundle including optimization of glycemic control, respiratory inspiratory plateau pressure, and determination of the need for corticosteroids or drotrecogin alfa (activated).

Although some of the recommendations are controversial, several single-center studies^13- 17 suggested quality-improvement efforts based on the SSC guidelines were associated with better outcome.

The aim of the present study was to determine whether a national educational program based on the SSC guidelines could improve compliance with recommended processes of care in severe sepsis in Spanish ICUs. We hypothesized that better adherence to guidelines would result in improved patient outcomes.

The steering committee of the Edusepsis study defined the project's purpose, timeline, interventions, and design. Through the Spanish Society of Intensive Care, all Spanish ICUs for adults (280 units) were invited to participate. No fees were provided for participation. The study was launched in July 2005. The ICUs were not asked to provide reasons for not participating. Seventy-seven medical-surgical Spanish ICUs homogeneously distributed around the country were included in the study. All ICUs were closed units with a critical care specialist on hand 24 hours per day, 365 days per year.

The general coordinating center was located at the Critical Care Center of the Hospital of Sabadell, Barcelona. Each ICU belonged to a geographical area coordinated by an area coordinator and at least 1 physician was designated principal investigator in each center.

A before and after study design was used. The before period (preintervention) consisted of all consecutive patients with severe sepsis who were admitted to the participating ICUs 2 months before the educational program began (November-December 2005). The intervention was introduced over a 2-month period (January-February 2006), during which no patient data were collected. The postintervention period consisted of all consecutive patients with severe sepsis admitted to the participating ICUs during a 4-month period (March-June 2006). In addition, to determine the longevity of the effects of the educational program, a third observation period, composed of all consecutive patients admitted to a subset of the participating ICUs during a 2-month period (May-June 2007) 1 year later was included.

Each participating centers' research and ethical review boards approved the study and patients remained anonymous. The need for informed consent was waived in view of the observational and anonymous nature of the study.

All ICU admissions from the emergency department or medical and surgical wards and all ICU patients were actively screened daily for the presence of severe sepsis or septic shock using a screening tool, which included definitions of sepsis and organ dysfunction. When the onset of severe sepsis could not be determined, patients were not included in the study.

Severe sepsis was defined as sepsis associated with acute organ dysfunction: (1) respiratory dysfunction, bilateral pulmonary infiltrates with a ratio of PaO₂ to FIO₂ of less than 300 mm Hg; (2) renal dysfunction, urine output of less than 0.5 mL/kg per hour for at least 2 hours or creatinine level of greater than 2.0 mg/dL (to convert to μmol/L, multiply by 88.4); (3) coagulation abnormalities, international normalized ratio greater than 1.5 or a partial thromboplastin time greater than 60 seconds; (4) thrombocytopenia, platelet count of less than 100 × 10³/μL; (5) hyperbilirubinemia, total plasma bilirubin level of greater than 2.0 mg/dL (to convert to μmol/L, multiply by 17.104); (6) hypoperfusion, lactate level greater than 18 mg/dL (to convert to mmol/L, multipy by 0.111); or (7) hypotension, systolic blood pressure of less than 90 mm Hg, mean arterial pressure of less than 65 mm Hg, or a reduction in systolic blood pressure of greater than 40 mm Hg from baseline measurements. Septic shock was defined as acute circulatory failure (systolic blood pressure <90 mm Hg, mean arterial pressure <65 mm Hg, or a reduction in systolic blood pressure >40 mm Hg from baseline) despite adequate volume resuscitation.

To standardize the educational program, the general coordinating center organized meetings with the area coordinators and the principal investigator of the participating centers before and after each study period. Before the implementation of the educational program, the importance of the study was explained to the hospital manager at each participating center to ensure full institutional support. The general coordinating center provided specific material for this meeting with information regarding the epidemiology, morbidity, mortality, and costs of severe sepsis.

The principal investigator acted as local champion, charged with creation of a local multidisciplinary team with representatives of all pertinent stakeholders, including physicians and nurses from the ICU and infectious diseases, emergency, and internal medicine departments. The local multidisciplinary team reviewed the preintervention performance data and shared ideas about the process improvement goals and strategies.

Between January and February 2006, the local multidisciplinary team at each hospital implemented a homogeneous predefined multifaceted educational program based on the SSC guidelines.¹² The educational program consisted of training physicians and nursing staff in the definitions of severe sepsis and septic shock, their early recognition, and the treatments included in the guidelines. The educational program was implemented in the emergency department, medical and surgical wards, and the ICU. Each center was provided with specific training and educational material including: (1) a PowerPoint (Microsoft, Redmond, Washington) presentation regarding sepsis definitions, early recognition and treatment including decision-making algorithms; (2) a Spanish translation of the SSC guidelines in poster and pocket format; (3) posters focused on the early recognition of sepsis with definitions of systemic inflammatory response syndrome and sepsis, and information about its morbidity and mortality; and (4) feedback from the general coordinating center with the baseline period data of each center compared with global baseline period data (audit and feedback). Posters were displayed in prominent places in the emergency departments, medical and surgical wards, and ICUs. Pocket versions were distributed to all participants in the educational program sessions. All teaching material also was available from the SEMICYUC's (Sociedad Española de Medicina Intensiva, Critica y Unidades Coronarias) Web site (http://www.semicyuc.org).

The general coordinating center maintained continuous contact with the principal investigator of each center through a mailing list (edu-sepsis@semicyuc.org). Moreover, after the educational program, a survey was distributed to all principal investigators to check that all participating centers had completed the educational program, to know the number and duration of lectures, and to know the subjective evaluation of the principal investigator of the main end points of the educational program, which were institutional support, creation of a multidisciplinary team, improvement in knowledge, and improvement in hospital processes.

The clinical and demographic characteristics of all patients, including age, sex, Acute Physiology and Chronic Health Evaluation II (APACHE II) score, diagnosis at admission, origin of infection, and organ dysfunction at sepsis presentation were recorded. The onset of sepsis (time zero) was determined according to the patient's location within the hospital when sepsis was diagnosed. In patients diagnosed with sepsis in the emergency department, time zero was defined as the time of triage. For patients admitted to the ICU from the medical and surgical wards or other non–emergency department units, time zero was determined by searching the clinical documentation for the time of diagnosis of severe sepsis. This might include, for example, a physician's note or timed and dated orders, a timed and dated note of a nurse's discussion of severe sepsis with a physician, or timed records initiating referral to the ICU for severe sepsis. If no time and date could be found by searching the chart, the default time of presentation was the time of admission to the ICU. Lastly, for patients who developed severe sepsis after admission to the ICU, the time of presentation was again determined on the basis of the clinical documentation.

The primary outcome measure was hospital mortality. Secondary outcome measures included other outcome measurements (ICU mortality, 28-day mortality, hospital length of stay, and ICU length of stay) and process-of-care variables. Process-of-care variables included 10 tasks grouped in the sepsis resuscitation bundle (comprising 6 tasks that should begin immediately and be accomplished within the first 6 hours of presentation) and the sepsis management bundle (consisting of 4 management tasks that should begin immediately and be completed within 24 hours of presentation). Time (0 to 12 hours) also was recorded from sepsis presentation to the process-of-care variables of serum lactate measurement, blood culture extraction, the administration of broad spectrum antibiotics, achievement of central venous pressure of 8 mm Hg or greater, and central venous oxygen saturation of 70% or greater.

Data were collected prospectively daily using preprinted case report forms. Detailed instructions explaining the aim of the study, instructions for the data collection, and definitions for various items were made available to all investigators before data collection started. Completed data forms were mailed to the coordinating center and registered in the database by a research nurse with experience in sepsis trials. Errors or blank fields generated queries that were returned to each center for correction.

For quality assurance purposes, data were checked for completeness, accuracy, and uniformity. Also, a random sample of 10% of patients was reevaluated. Inter-rater reliability for the variables was assessed in this sample. κ coefficients were calculated as appropriate. A reliability of 96.5% of all variables per case report form was observed.

The study was designed with a power of 80% and a type I error of 5% to detect a 15% relative reduction in septic shock mortality, which was estimated at 45%,⁴ and for logistic reasons the preintervention to postintervention ratio was 1:2. Therefore, a total of 1875 patients were needed. To reach this number, we calculated that we needed 2 months for the baseline period and 4 months for the postinterventional period with the participation of at least 55 ICUs. Descriptive statistics included frequencies and percentages for categorical variables and means, standard deviations, confidence intervals (CIs), medians, and interquartile ranges for continuous variables. To compare continuous variables during the 2 study periods, the t test or the Mann-Whitney test was used when appropriate. To analyze categorical variables during the 2 study periods, χ² analysis was used.

Multivariate logistic regression was performed, with hospital mortality as the dependent variable and the educational program, APACHE II score, age, patient location at sepsis diagnosis, and the presence of shock as independent variables. Because the 2 periods of the study took place in different seasons and pneumonia may have a different seasonal incidence, a sensitivity analysis was conducted by fitting the previous multivariate logistic regression model but including pneumonia as another independent variable into the model. Kaplan-Meier curves representing the 28-day mortality stratified according to group assignment were compared using a log-rank test. Because the missing data rate in the study was low (the variable with the highest missing rate was an APACHE II score with a rate of 1.5%), no imputation of missing data was performed. Although statistical tests were 2-tailed and significance was set at a level of .05 for process-of-care variables, corrections for multiple comparisons also were pursued using the Bonferroni method (for the 10 variables, an adjusted significance level cutoff of .005 was used). All analyses were conducted using SPSS version 15.0 (SPSS, Chicago, Illinois).

Eighteen of the 77 participating ICUs were excluded because they failed to complete the postintervention period. One hundred fifty-one patients treated in these units were excluded. Data from 59 ICUs with a total of 965 critical care beds were included in the final analysis. All ICUs were medical-surgical and most (68%) were located in teaching hospitals with residents in training. The excluded units also were mainly medical-surgical units, with residents in training (66%) and had a total of 250 critical care beds. Patients from the excluded ICUs (n = 151) were not different from the preintervention cohort in terms of age, APACHE II score, sex, diagnosis at admission, patient location at sepsis diagnosis, main sources of sepsis, or mortality (44.0% [95% CI, 41%-47%] vs 45.0% [95% CI, 37%-53%]; P = .82). Eight patients were excluded because time 0 could not be determined (5 in the preintervention cohort and 3 in the postintervention cohort).

Educational lectures and dissemination sessions for attending physicians and nurses from the emergency department, medical and surgical wards, and ICU were conducted at all sites. The mean time dedicated to lectures was 10.6 hours at each center, mostly focused in the ICU (3.5 hours) and the emergency department (2.5 hours). The principal investigators considered that they received the institutional support of their centers in 68% of cases, but only succeeded in the creation of a multidisciplinary team in 47.2% of cases. The principal investigators considered that the staff's knowledge improved in 96.2% of centers, while they considered that hospital processes improved in 86.8% of centers.

A total of 2319 patients fulfilled severe sepsis or septic shock criteria during the preintervention and postintervention periods. The mean (SD) age was 62.2 (16.3) years and the mean (SD) APACHE II score was 21.2 (7.7); 60.8% (n = 1411) were male, 79.4% (n = 1842) had septic shock, and hospital mortality was 41.2% (n = 956). An additional 254 patients were included in the long-term follow-up period.

The preintervention cohort included 854 patients and the postintervention cohort included 1465 patients. The characteristics of patients in the preintervention cohort and the postintervention cohort were similar (Table 1), with no statistically significant differences in age, sex, or APACHE II score. Diagnosis on admission was mainly medically or surgically urgent in both cohorts. Patient location at sepsis diagnosis was predominantly in the emergency department and the medical-surgical wards with no significant differences between the 2 periods. The main sources of sepsis were pneumonia and acute abdominal infections in both periods. Pneumonia was more frequent in the preintervention cohort. In the postintervention cohort, patients with urinary tract infection, meningitis, and multiple infection sites were slightly more frequent. There were no differences in organ dysfunction criteria at sepsis presentation, except for respiratory dysfunction, which was more frequent in the preintervention cohort. Hemodynamic dysfunction was present in more than 80% of cases in both periods.

At baseline (Table 2), compliance with the SSC recommendations was only 5.3% (95% CI, 4%-7%) for the sepsis resuscitation bundle and 10.9% (95% CI, 9%-13%) for the sepsis management bundle. Regarding the sepsis resuscitation bundle, the only 2 process-of-care measurements that showed compliance higher than 50% were blood cultures before antibiotics (54.4% [95% CI, 51%-58%]) and early administration of broad spectrum antibiotics (66.5% [95% CI, 63%-70%]). The measures related to EGDT, which required the insertion of a central venous catheter, were implemented in only 40.9% (95% CI, 37%-44%) of cases for fluids and vasopressors, 21.4% (95% CI, 19%-24%) for optimization of central venous pressure, and 6.3% (95% CI, 5%-8%) for optimization of central venous oxygen saturation. Regarding the sepsis management bundle, the ventilatory support was adequate in 85.7% (95% CI, 83%-89%) of mechanically ventilated patients; however, glucose control was adequate in only 44.6% (95% CI, 41%-48%) of cases. The possible use of low-dose steroids was contemplated, although not necessarily carried out, in only 45.4% (95% CI, 42%-49%) of septic shock patients, while in the rest of patients it was not even considered. Regarding drotrecogin alfa (activated), the possible use was contemplated, although not necessarily carried out, in only 44.3% (95% CI, 41%-48%) of patients.

Process-of-care measurements improved after the educational program (Table 2). All elements of the sepsis resuscitation bundle improved significantly after the educational program, except early administration of broad-spectrum antibiotics (66.5% [95% CI, 63%-70%] to 68.9% [95% CI, 67%-71%]; P = .24). Likewise, all elements of the sepsis management bundle improved significantly after the educational program, except control of plateau pressure (85.7% [95% CI, 83%-89%] to 82.7% [95% CI, 80%-85%]; P = .15). The percentage of patients in whom care complied with all resuscitation and all management measures improved significantly after the educational program. Following correction for multiple comparisons, lactate measurement, blood cultures before antibiotics, central venous oxygen saturation of 70% or greater, and steroids and drotrecogin alfa according to ICU policy maintained their initial significant differences. Fluids plus vasopressors (central venous pressure ≥8 mm Hg) and glucose control lost their significance. While the recommendation to consider steroids in septic shock and drotrecogin alfa (activated) was better implemented after the intervention (steroids, 45.4% [95% CI, 42%-49%] vs 54.7% [95% CI, 51%-57%]; P < .001 and drotrecogin alfa, 44.3% [95% CI, 41%-48%] vs 51.9% [95% CI, 49%-54%]; P < .001), the percentage of cases in which low-dose steroids were administered increased from 36.4% (95% CI, 33%-40%) to 41.7% (95% CI, 39%-44%) (P < .001) after the educational program, but the percentage of cases in which drotrecogin alfa (activated) was actually administered remained stable.

The time from sepsis presentation to process-of-care variables is reported in Table 2. After the educational program, the mean time to the obtainment of blood cultures and to the administration of broad-spectrum antibiotics was reduced by 20 and 26 minutes, respectively. The educational program did not shorten the time elapsed before lactate determination (central venous pressure ≥8 mm Hg achievement or central venous oxygen saturation of ≥70% achievement).

The outcome data are shown in Table 3. Patients in the postintervention cohort had a statistically significant lower risk of hospital mortality (44.0% [95% CI, 41%-47%] vs 39.7% [95% CI, 37%-42%]; P = .04) and 28-day mortality (36.4% [95% CI, 33%-40%] vs 31.1% [95% CI, 29%-33%]; P = .009) compared with the preintervention cohort. A Kaplan-Meier plot of the probability of remaining alive is shown in the Figure. The ICU mortality also was significantly lower in the postintervention cohort. No differences were observed in hospital or ICU stay in the surviving populations before and after the educational program.

Multivariate logistic regression (Table 4) to adjust for possible confounders (APACHE II score, age, patient location at sepsis diagnosis, and the presence of shock) showed that the postintervention cohort was independently associated with lower hospital mortality (odds ratio, 0.81 [95% CI, 0.67-0.98]; P = .03). The sensitivity analysis that included pneumonia into the regression showed highly consistent results with the results presented in Table 4 (odds ratio for the postintervention cohort 0.83 [95% CI, 0.68-1.00); P = .04).

The ICUs were classified in 3 categories by percentiles depending on the process of care at baseline, measured by the number of tasks included in the bundles completed (Table 5). Baseline care of patients treated in category 1 ICUs (20 ICUs) complied with less than 4 tasks; baseline care in category 2 ICUs (19 ICUs) complied with 4 to 5 tasks; and baseline care in category 3 ICUs (20 ICUs) complied with more than 5 tasks. The educational program improved the process of care in all 3 categories of ICUs (relative improvement of tasks completed of 36.0% in category 1, 12.2% in category 2, and 9.3% in category 3) but only reduced mortality in category 1 ICUs (48.0% [95% CI, 41%-55%] vs 39.3% [95% CI, 34%-44%]; P = .046).

One year after the postintervention period, 23 centers participated in a 2-month follow-up study to measure the long-term effects of the educational program (n = 247 patients). No differences in the epidemiological characteristics of patients were observed (Table 6). The percentage of patients in whom care complied with all resuscitation measures returned to baseline in the long-term follow-up period (preintervention, 6.3% [95% CI, 4%-9%]; postintervention, 12.9% [95% CI, 10%-16%]; long-term follow-up, 7.3% [95% CI, 4%-11%]; P = .003). The percentage of patients in whom care complied with all management measures was stable with respect to the postintervention period (preintervention, 9.4% [95% CI, 6%-13%]; postintervention, 19.6% [95% CI, 16%-23%]; long-term follow-up, 26.7% [95% CI, 21%-32%]; P < .001), and mortality remained stable with respect to the postintervention period (preintervention, 42.5% [95% CI, 37%-48%]; postintervention, 38.7% [95% CI, 34%-43%]; long-term follow-up, 38.5% [95% CI, 32%-45%]; P = .50).

Implementing an educational program based on the SSC guidelines and bundles improved the process-of-care variables and reduced mortality in patients with severe sepsis and septic shock. Given the incidence of septic shock of 31 cases per 100 000 adult residents per year and the total adult population of 36 million in Spain, 490 lives may have been saved by this effort in 1 year if the educational program had been extended to all Spanish hospitals. The strengths of this study are its preintervention and postintervention design, the large cohort of patients, and the participation of 59 centers in Spain (21% of all ICUs in the country). The mortality, age, comorbidities, and source of sepsis in our study population is comparable with those found in several epidemiological studies on sepsis.^4,18

At baseline, the adherence to SSC recommendations was poor; however, this is not surprising because it has been reported by other investigators in Spain.¹⁹ Gao et al¹⁷ found better compliance rates in 2 teaching hospitals in England, whereas Nguyen et al¹⁶ found better baseline compliance with antibiotic use but results similar to ours for EGDT and steroids. In the study by Micek et al,¹⁴ baseline compliance was slightly better for antibiotics but worse for the rest of the medical care processes recommended by the guidelines.

All process-of-care measurements improved after the educational program, except antibiotic use and plateau pressure control. These were the 2 measures implemented best in the preintervention cohort; therefore, it is likely that they did not improve because there was less room for improvement. On the other hand, the time to antibiotic treatment was significantly shorter after the educational program, and this could be considered an improvement in the overall use of antibiotics.

This is the first study to prospectively evaluate the impact of an educational program on guideline compliance and mortality in patients with severe sepsis. This strategy has been tested before in community-acquired pneumonia with similar results.^20- 21 However, other investigators could not reproduce these results.²² Recently, also in community-acquired pneumonia, Barlow et al²³ showed that a multifaceted educational strategy was able to reduce door-to-antibiotic time, a process-of-care variable associated with better outcome.

At long-term follow-up, some of the improvements achieved by the educational program had returned to baseline, especially process-of-care measures in the acute phase of treatment. However, it is well-known that quality-improvement initiatives should be sustained,²⁴ especially in areas like the emergency department in which physician turnover is higher than in other areas of the hospital. Applying the “plan-do-study-act” cycles is probably the best approach to sustain the effect of the educational program.

Previous studies found that better compliance with the SSC guidelines was associated with better outcome, but most of these were single-center trials, involving small numbers of patients and did not prospectively test the ability to change clinical practice with a multifacteted educational initiative.^13- 17 Shapiro et al²⁵ were unable to demonstrate mortality benefits, but they showed that it is feasible to change the quality of care in severe sepsis using protocols based on the SSC guidelines. Our data showed that there is a possible relationship between baseline compliance with the guidelines and the effect of the educational program, suggesting that educational efforts are most effective when applied in ICUs with low baseline compliance with the guidelines.

The decreased mortality observed in our study and other studies^13- 17 might derive from better identification of patients with severe sepsis or from improved compliance with quality indicators, including earlier administration of antibiotics, or both. Kumar et al⁶ showed that a 1-hour decrease in the time to antibiotic administration could improve survival. The design of this study did not enable these hypotheses to be tested separately. It is unlikely that the improvement in mortality in the postintervention cohort was due to differences between the 2 cohorts, although it could be postulated that the decrease in mortality was due to a slightly higher frequency of urinary tract infection in the postintervention cohort, which is associated to lower mortality,⁵ or due to seasonal effect with lower incidence of pneumonia in the postintervention cohort. The sensitivity analysis showed that the effect of the educational program on mortality was independent of the cause of sepsis.

Several areas for improvement have been identified through this process, including the rate of compliance with the EGDT-related variables remained below 50% in the postinterventional period. Additionally, the use of low-dose steroids and drotrecogin alfa (activated) according to a standardized ICU policy improved after the educational program, but remained low. While clinicians considered drotrecogin alfa (activated) more frequently after the educational program, they did not administer the drug more frequently. Further improvement in these areas might help to increase survival even more.

Rubenfeld²⁶ categorized the possible reasons for the gap between evidence (guidelines) and practice into 3 major groups: knowledge barriers, attitude barriers, and behavioral barriers. A recent survey conducted in emergency departments of the 25 most densely populated areas in the United States concluded that the barriers to implementing time-sensitive resuscitation to patients with severe sepsis are shortage of nursing staff, problems in obtaining central venous pressure monitoring, and challenges in identifying patients with severe sepsis.²⁷ The educational program was focused on overcoming knowledge barriers and probably had some impact on attitude and behavioral barriers.

Our study has limitations. First, the design makes it difficult to establish a causal connection between the intervention and the improvement in process-of-care variables and outcome. The use of a control group²⁰ or the use of groups with progressive intensity of the education²¹ would have allowed us to rule out a possible effect of a secular trend on the improvement in sepsis treatment. However, this was not feasible because all the participating units wanted to do the full intervention and ethical constraints proscribed failing to promote the standard of care in some units. On the other hand, the short period between the 2 study periods lends weight to the supposition of a true and strong association between the intervention and the outcome. Second, the duration of the educational program was only 2 months, and a longer educational period might have enabled greater improvements in compliance with the quality indicators and survival. Third, the use of other educational strategies like academic detailing has been useful in other studies²⁸ and should be considered in future quality-improvement projects. Fourth, because the majority of patients had septic shock, we cannot know whether the effects of the educational program would have been the same on patients with severe sepsis who had not developed septic shock. Fifth, because our study was limited to patients admitted to the ICU, we do not know the effects of the educational program on patients with severe sepsis cared for in other areas of the hospital.

The baseline compliance with the SSC guidelines in Spain in 2005 was low. A national educational effort to promote bundles of care for severe sepsis and septic shock was associated with improved guideline compliance and lower hospital mortality. However, compliance rates were still low, and the improvement in the resuscitation bundle lapsed by 1 year.