Effect of Evidence-Based Feeding Guidelines on Mortality of Critically Ill Adults: Title and subTitle BreakA Cluster Randomized Controlled Trial FREE

Early nutritional support, provided within 24 hours of injury or intensive care unit (ICU) admission, is a key component in the treatment of critically ill patients and may reduce mortality by 8% to 13%.^{1 - 4} Nevertheless, practice varies widely between ICUs,⁵ and up to 40% of eligible patients may remain unfed after 48 hours in the ICU.⁶

Evidence-practice gaps are common in clinical practice, with 30% of hospitalized patients receiving care inconsistent with current best evidence.^{7 - 8} Evidence-based guidelines (EBGs) help reduce evidence-practice gaps by promoting awareness of interventions of proven benefit and discouraging ineffective care.^{3 ,9 - 11} However, the ICU is a complex multidisciplinary environment, and reducing evidence-practice gaps through the successful implementation of an EBG in such an environment is difficult.^{1 ,6}

The purpose of this project was to conduct a cluster randomized controlled trial (RCT) to examine the effect on mortality and measures of practice change of implementing EBGs for nutritional support. A cluster RCT is the appropriate design to use when an educational intervention is under study and there is the possibility that knowledge gained through participation in the trial could lead to changes in standard care.¹²

Potential participating hospitals were identified throughout Australia and New Zealand (ANZ) through announcements at clinical trials group meetings, scientific meetings, and professional society e-mail lists. Public or private hospitals with a level III or level II ICU were eligible for participation if they were able to identify an intensive care specialist and dietitian as coinvestigators. The ANZ Joint Faculty of Intensive Care Medicine defines a level III ICU as a tertiary referral unit capable of providing comprehensive critical care, including complex multisystem life support, for an indefinite period. A level II ICU is capable of providing a high standard of general intensive care, including complex multisystem life support. Both level II and III ICUs are closed, and patients must be referred for management to the attending intensive care specialist.¹³ Each hospital contributed only 1 ICU to the study.

Approval to conduct the study was obtained from each site's human research ethics committee, all of which waived the requirement for individual patient-level consent.

The study consisted of a 5-week study run-in and guideline development period followed by a 20-week guideline implementation and evaluation period.

During the study run-in, site investigators refined the application of patient eligibility criteria, became familiar with data collection instruments, and improved study conduct by responding to queries and feedback from the data management center.

Midway through the run-in period, hospitals were randomized to intervention or control groups. At the time of randomization, one author (S.F.) provided a random number used to seed an SAS (SAS Institute Inc, Cary, North Carolina) algorithm (developed by G.S.D.) that generated the specific allocation sequence used in the trial. Allocation concealment was maintained because investigators did not have knowledge of the random seed or resultant sequence until the time of randomization, which occurred 2 weeks after all hospitals had been selected and agreed to participate in the trial. Randomization was stratified by number of ICU beds.

The dietitian and intensivist coinvestigators from intervention ICUs participated in a 2-day guideline development conference held October 27-28, 2003. These investigators were specifically requested not to disseminate study-related information outside their study ICU. Control ICUs collected data but were not invited to participate in the guideline development conference and remained unaware of the contents of the guideline and the supporting practice change strategy. Control ICUs were specifically requested not to undertake any new ICU-level nutritional management changes using guidelines or protocols for the duration of the trial.

Following the guideline development conference, all ICUs received a site monitoring visit from the lead investigators (G.S.D., F.S.) to address study conduct and data quality issues. In guideline ICUs, this included formal educational outreach to promote awareness of the EBG. The 20-week guideline evaluation phase was initiated at each participating ICU after the initial site monitoring visit. Staggered initial monitoring visits resulted in patient recruitment running from November 3, 2003, until May 21, 2004.

Patients were screened for eligibility within 24 hours of ICU admission. Adult patients who were expected to stay in the ICU longer than 2 days were eligible for inclusion. Patients were not eligible if they were tolerating an oral diet or scheduled to return to oral intake within 24 hours. Patients receiving palliative care, those who were moribund and not expected to survive 6 hours, those with brain death or suspected brain death, and patients admitted directly from any other ICU were excluded.

The guideline was developed using the Clinical Practice Guideline Development Cycle, an explicit and transparent process for the development of evidence-based guidelines.¹⁴ The process relies on an extensive literature search and critical appraisal to identify valid clinical trials reporting clinically meaningful outcomes,¹⁵ which are synthesized into a series of systematic reviews.¹⁰ Each systematic review is graded to reflect the level of evidence it contains and is formatted as an evidence-based recommendation (EBR).¹⁶ These EBRs are reviewed at a formal consensus conference and approved (ratified) for inclusion in the guideline.

The 2-day guideline development conference included an educational workshop on the use of the multifaceted change strategy to be used to implement the guideline.¹⁷ The practice-change strategy was composed of 18 interventions grouped under 7 categories.

Identification of Peer-Nominated, Educationally Influential Opinion Leaders. Intensive care unit consultants, nurses, and surgeons who admitted patients to the study ICUs were surveyed to identify educationally influential opinion leaders from each respective discipline¹⁸ using a validated instrument.¹⁹ Peer-nominated opinion leaders received education on the content of the ANZ guideline and were provided with copies of the guideline and other resource material to use as educational aids to support academic detailing.

Educational Outreach Visits/Guideline Site Initiation. Educational outreach visits were conducted by the project chief investigators (G.S.D., F.S.) to initiate the guideline implementation process at each guideline hospital. The visit included a presentation of the expected benefits of implementing the ANZ guideline at a grand rounds type of meeting and was followed by one-on-one meetings (academic detailing) with any member of the staff expressing interest in understanding the guidelines.

Academic Detailing. Site investigators and educationally influential opinion leaders were trained to conduct academic detailing. Academic detailing was explained as a one-on-one conversation with a staff member or clinician reluctant to adopt the ANZ guideline to address ongoing individual concerns and encourage him or her to change behavior through the provision of information or evidence.²⁰ Copies of original scientific articles, single-page critical appraisal summaries of all articles, and a visually attractive resource book that contained systematic reviews of the evidence were provided to support academic detailing.

Detailers identified early adopters of the ANZ guideline to use them as positive examples during these academic detailing sessions to convince “laggards”²¹ to change practice.¹⁸

Active Reminders. Dietitian site investigators reviewed ICU patients twice daily to assess eligibility for, and compliance with, the ANZ guideline. When a patient qualified for care under the guideline, the dietitian investigator attempted to communicate directly with senior staff and clinicians through a short friendly chat, preferably face to face.

Timely Audit and Feedback. Key measures of guideline compliance were recorded at each site and entered into a secure study Web server. Guideline ICU site investigators were able to print control-chart graphs to compare their ICU's current performance with the cumulative current performance of all other guideline ICUs.

Passive Reminders. Brightly colored copies of the ANZ guideline, presented in algorithmic format, were posted in high-traffic areas (A3-sized posters), by the patients' bedside (A4-sized laminated sheets), and next to each ICU computer station (mouse mats).

In-servicing. A series of interactive lecture-style presentations were conducted by site investigators to introduce the EBG to ICU staff and clinicians. The number and timing of in-servicing sessions varied between hospitals based on need and schedules.

Site investigators were trained in study conduct and data collection at a 1-day study start-up meeting. Data were collected prospectively using paper case report forms supported by an extensive data dictionary.

In cluster RCTs, the assumption of independence of outcomes within study centers often fails, and the true sample size required to obtain a prespecified power is larger than required for patient-level randomized trials. The intracluster correlation coefficient is a measure of the reduction in independence, and the design effect is a measure of the increase in the total number of participants required using cluster randomization relative to the number required using simple randomization.^{22 - 23}

Estimates of baseline mortality and intracluster correlation coefficient were obtained from ICU outcome data submitted by 8 participating hospitals. Sample-size formulas specific to cluster RCTs were used.²⁴

Given that a previous cluster RCT evaluating a similar intervention demonstrated an absolute reduction in mortality of 13%,³ a conservative estimate of expected benefit was set at 60% of this effect. Assuming baseline mortality of 28.9% and an intracluster correlation coefficient of 0.01153, 26 hospitals enrolling 693 patients per group would provide 80% power to detect a treatment effect of 8%. It was estimated that 20 weeks would be required to enroll this number of patients.

The primary outcome, all-cause hospital discharge mortality, was analyzed using a negative-binomial model.¹² Secondary outcomes based on count data (eg, length of stay, number of days of clinically significant organ dysfunction²⁵ ) were analyzed using a Poisson model.¹² Where appropriate, an offset term (ICU length of stay) was used to account for time at risk. Baseline balance of proportions was assessed using a χ² test, and continuous variables were assessed using t tests.²⁶ All analyses and 95% confidence intervals (CIs) were appropriately adjusted for the effects of clustering.^{12 ,27} A 2-tailed P < .05 was accepted to indicate statistical significance. A 2-tailed P < .25 was accepted to indicate the presence of potentially important confounding due to imbalance in baseline covariates.²⁸ Confounding due to baseline imbalance was investigated using an appropriately adjusted multivariate model. Analyses were conducted using STATA version 7.0²⁹ or Acluster Version 2.1.³⁰

Thirty-six ICUs from 36 hospitals expressed interest in participating in the project. Nine hospitals failed to meet inclusion/exclusion criteria (Figure 1). Nonparticipating ICUs did not differ from participating ICUs with regard to major ICU-level characteristics. Sixty-six percent (6/9) of nonparticipating sites were metropolitan referral centers, compared with 70% (19/27) of participating sites (P = .85). Sixty-six percent (6/9) of nonparticipating sites were level III ICUs compared with 88% (24/27) of participating sites (P = .30), and the median number of beds for nonparticipating ICUs was 12 (range, 5-21) compared with 12 (range, 5-18) (P = .99).

All 27 participating ICUs successfully completed the entire project, and no patients were lost to follow-up.

The comprehensive literature search, which included primary MEDLINE and EMBASE searches, hand searching of reference lists, and contact with experts and industry, resulted in the retrieval of 465 articles. Detailed review (G.S.D., F.S.) of retrieved articles identified 92 primary nutritional support studies that qualified for inclusion in the guideline development process. These studies were conducted in populations of critically ill patients, reported clinically meaningful outcomes, and did not have major methodological flaws. Detailed results of the search, validity appraisal, systematic review, and meta-analytic processes have been published elsewhere.^{31 - 32}

Formal systematic reviews and meta-analyses were generated in 25 topic areas, all of which were addressed at the guideline development conference. The guideline development conference resulted in the approval of 11 systematic reviews, which were expressed in the form of ratified EBRs (Box). These 11 ratified EBRs formed the basis of the EBG and are presented in algorithmic form in Figure 2.

Grade B+

Grade B

Grade B−

Recommendation Grades

Levels of Evidence^a

Enrollment Rates and Baseline Balance . During the 20-week evaluation period, 1118 patients were identified as eligible and were enrolled (Figure 1). There was no difference in the enrollment rates between guideline and control ICUs (3.9 vs 3.4 patients per ICU bed, respectively; P = .31).

Significantly more patients with a neurological admission diagnosis were enrolled in guideline ICUs (69/561 vs 39/557 patients, P = .04). There was no evidence of any other baseline imbalance (Table 1).

Process Measures of Guideline Uptake: Provision of Nutritional Support . No guideline hospitals failed to implement the ANZ guideline. Significantly more patients in guideline ICUs received nutritional support during their ICU stay (94.3% vs 72.7%; difference, 22.5% [95% CI, 18.1% to 25.0%]; P < .001), and significantly more patients in guideline ICUs were fed within 24 hours of ICU admission (60.8% vs 37.3%; difference, 23.4% [95% CI, 12.9% to 36.2%]). Patients in guideline ICUs were fed significantly earlier (0.75 vs 1.37 mean days to start of enteral nutrition; difference, −0.62 [95% CI, −0.82 to −0.36]; P < .001 and 1.04 vs 1.40 mean days to start of parenteral nutrition; difference, −0.35 [95% CI, −0.61 to −0.01; P = .04) and were fed on a greater proportion of ICU days (8.08 vs 6.90 fed days per 10 patient-days; difference, 1.18 [95% CI, 0.41 to 2.03]; P = .002) than patients in control ICUs. On each of the first 5 days of ICU stay, significantly more patients in guideline ICUs received nutritional support (Figure 3).

The mean energy delivered per patient per day and the mean energy delivered per fed patient per day were not significantly different between groups (1241 vs 1065 kcal/patient-day; difference, 177 [95% CI, −51 to 457]; P = .14 and 1265 vs 1204 kcal/fed patient-day; difference, 61 [95% CI, −147 to 310]; P = .59). Complete measures of nutritional support are presented in Table 2 and Figure 4.

Mortality and Length of Stay . There were no significant differences between guideline and control ICUs with regard to hospital discharge mortality (28.9% vs 27.4%; difference, 1.4% [95% CI, −6.3% to 12.0%]; P = .75), ICU discharge mortality (24.5% vs 21.5%; difference, 3.0% [95% CI, −3.2% to 10.4%]; P = .43), mean hospital length of stay (24.2 vs 24.3 days; difference, −0.08 [95% CI, −3.8 to 4.4]; P = .97) or mean ICU length of stay (9.1 vs 9.9 days; difference, −0.9 [95% CI, −2.6 to 1.3 days]; P = .42). Table 3 presents the intracluster correlation coefficient and design effects for the main study outcomes.

Clinically Significant Organ Dysfunction and Concomitant Therapies . The incidence of clinically significant renal dysfunction was significantly lower in the guideline ICUs (1.54 vs 2.12 renal dysfunction days/10 patient-days; difference, −0.58 [95% CI, −1.0 to −0.04]; P = .04) compared with controls; however, there was no difference in the provision of renal replacement therapy (0.75 vs 0.91 dialysis days/10 patient-days; difference, −0.16 [95% CI, −0.38 to 0.16]; P = .29).

There were no other significant differences between the groups. Table 4 provides a complete list of all other clinically significant organ system dysfunctions, and Table 5 provides a summary of other key concomitant interventions.

Multivariate analysis controlling for the only baseline covariate appearing to be in imbalance (neurological admission diagnosis) did not alter the overall interpretation of the primary outcome, difference in hospital discharge mortality (difference, −1.0% [95% CI, −6.7% to 11.7%]; P = .81).

We used the Clinical Practice Guideline Development Cycle¹⁴ to develop a binational (ANZ) EBG for nutritional support of critically ill patients. Recommendations ratified for inclusion in the guideline were supported by systematic reviews of valid clinical trials demonstrating benefit to clinically meaningful patient-oriented outcomes. The effects of implementing the guideline using a robust, multifaceted practice change strategy were assessed in a cluster RCT.

The simple recommendations incorporated into the ANZ guideline were readily adopted across multiple sites with unique local cultures. Implementation resulted in statistically significant improvements to the provision of nutritional support; however, these practice changes did not translate into improvements in hospital discharge mortality.

To our knowledge, this is the first level I cluster RCT, equivalent to a US Food and Drug Administration phase 3 licensing trial, demonstrating that critical care evidence-practice gaps can be successfully addressed across multiple sites using a multifaceted practice-change strategy.

The practice observed in control hospitals and the magnitude of practice change achieved in the ANZ guideline hospitals was remarkably similar to the control hospital practice and improvements observed in the Algorithms for Critical Care Enteral and Parenteral Therapy (ACCEPT) cluster RCT.³ Because the ACCEPT trial demonstrated a significant reduction in hospital discharge mortality, it is reasonable to expect that the change achieved in the ANZ guidelines trial could have translated to improvements in patient outcomes.

Because a statistically significant result would have been obtained if an 8% reduction in mortality had been observed, post hoc loss of power does not explain our negative findings. Assuming we had found an 8% reduction in mortality, using measures of variability calculated from actual trial data, the 95% CI around the treatment effect would have ranged from 1% to 13.1%, which would have been statistically significant.³⁴

Both the ACCEPT trial and the ANZ guideline trial are consistent with a mortality reduction of up to 6.3%, which would be considered clinically important. The 95% CI observed in the ACCEPT trial ranges from a 21% to a 0.002% reduction in mortality. The lower limit of the ANZ guideline trial 95% CI overlaps the ACCEPT 95% CI in the range from 6.3% to 0.002%. Although the ACCEPT trial reported a statistically significant reduction in mortality and the ANZ guideline trial failed to confirm this benefit, their results are consistent.

While trials with major methodological flaws were excluded from the guideline development process, the overall quality of included clinical trials could be considered poor.³² Only 2 of the 11 ratified EBRs were supported by meta-analyses demonstrating significant benefits to clinically meaningful outcomes (Box).² The remaining 9 EBRs were supported by individual level II RCTs (US Food and Drug Administration phase 2 equivalent). Level II trials and meta-analyses of level II trials are known to overestimate potential treatment benefits.³⁵

It is possible that results of the ANZ guideline trial were negative owing to chance alone; however, it is also possible that benefits attributable to the provision of nutritional support are smaller than estimated by the current evidence. The negative findings of the ANZ guideline trial highlight the pressing need for better evidence evaluating the true effectiveness of each individual intervention recommended in this guideline. A series of well-designed, adequately powered level I (US Food and Drug Administration phase 3 equivalent) RCTs is needed.

Evidence from clinical trials conducted in critically ill patients and from animal models demonstrates that amino acid infusions ameliorate the severity of ischemic insults to the kidney³⁶ and enhance recovery of renal function.^{37 - 38} Although improvements observed in this trial may be a spurious finding (P = .04) and there was no significant reduction in the use of renal replacement therapies, future trials with greater power should be undertaken to investigate this relationship.

The content of the ANZ guideline is remarkably consistent with that of the ACCEPT guideline, the only major difference being a grade B− recommendation supporting the consideration of glutamine in parenteral nutrition in the ANZ guideline.

Both the ACCEPT guideline and the ANZ guideline are composed of a few simple recommendations supported by the best available evidence. For example, both guidelines contain an EBR supporting early enteral nutrition, but because reliable evidence was not available, neither contain recommendations for starting rates or advancement rates.³¹ Other guidelines for ICU nutritional support have ratified expert opinion–based recommendations when reliable evidence was not available (eg, start enteral nutrition at 25 mL/h).^{4 ,39}

Cluster RCTs evaluating mixed evidence- and opinion-based nutrition guidelines have not been successful at achieving meaningful practice change,⁶ perhaps because expert opinion is more prone to local dissent.⁴⁰ The ANZ guideline allowed ICUs to start and advance enteral nutrition at rates with which they were already familiar. By allowing ICUs to retain familiar local practices where evidence is unreliable, the ANZ guideline may have optimized the uptake of EBRs supported by clear evidence of benefit.^{14 ,41 - 42}

The ANZ guideline change strategy was coordinated at each site by dietitian and intensivist coinvestigators supported by peer-identified ICU nursing, surgical, and intensivist opinion leaders. The use of a multidisciplinary change team in a complex environment such as the ICU allows issues arising from complex interprofessional team dynamics to be addressed on a peer-to-peer level^{20 ,43 - 44} and may allow discipline-specific and system-level barriers to be addressed more effectively.⁴⁴

The practice improvements achieved in this study cannot be attributed to any single specific practice-change intervention. After study close-out, a follow-up survey was conducted. Study site investigators reported using different subsets of practice-change interventions to overcome various barriers to change common or unique to their site. The survey did not identify any single intervention that all sites found ineffective such that it could be left out of future practice-change initiatives.⁴⁴

An inherent weakness in the use of study hospital discharge status as an outcome is that patients may be discharged to another hospital, and their discharge status from the second hospital may remain unknown. Only 2% of guideline patients (13/561) and 4% of control patients (22/557) were discharged to another hospital. Simulation studies revealed that this degree of early discharge could not change the overall conclusions of the trial.

It is possible that control hospitals improved aspects of standard care over time due to a Hawthorne effect.⁴⁵ However, we observed no evidence of changes in patient outcomes over the duration of the study; thus, if standard care did improve over time, it is unlikely that these improvements contributed toward the negative findings of the trial.

A 20-week evaluation period may not have been long enough to achieve meaningful improvements in feeding practices. The ACCEPT trial³ provides the best benchmark against which to compare this current trial, and ACCEPT provides no evidence that greater improvements in feeding practices could have been expected with a prolonged evaluation period.

Recruitment was slower than anticipated, which resulted in a failure to recruit the target sample size. Due to budget constraints, the duration of the trial could not be extended to achieve the target sample size. Post hoc power analysis suggests that this did not compromise the ability of the study to detect an 8% improvement in mortality.

We used the Clinical Practice Guidelines Development Cycle to develop a binational evidence-based guideline for the provision of nutritional support. The guideline was composed of simple recommendations supported by evidence of benefit obtained from systematic reviews.

We achieved significant practice change in the complex environment of the ICU through the use of a multifaceted, multilevel practice-change strategy, leveraged by educational outreach visits. Although the successful implementation of the guideline resulted in significant practice change, it did not result in reduced hospital mortality in critically ill patients.