Inpatient Computer-Based Standing Orders vs Physician Reminders to Increase Influenza and Pneumococcal Vaccination Rates:  A Randomized Trial FREE

Few medical interventions rival influenza and pneumococcal vaccines in their ability to reduce morbidity,^1- 8 save costs,^1,4 and save lives.^1- 3,7- 8 Yet, among individuals older than 65 years, as many as 34% have not received their annual influenza vaccine and 38% have not received their annual pneumococcal vaccine.⁹ To improve these vaccination rates, many organizations recommend the use of both standing orders and computer-based reminder systems.^10- 12

Manual standing order systems have been effective at increasing rates among patients identified as eligible for vaccination.^13- 21 However, these systems require the investment of personnel time to find eligible patients, and consequently tend to have both limited reach and persistence. For example, in one trial, nurses were able to assess fewer than 40% of inpatients for vaccine eligibility at the beginning of the study, with a decline thereafter.²⁰ In another study, the percentage of patients offered vaccines across different inpatient wards varied more than 5-fold.²² Yet another trial within an emergency department achieved a 74% vaccination rate among patients whom nurses screened, but nurses were able to screen only 13% of the emergency department visits.¹³

In a previous study,²³ we demonstrated that computer reminders delivered to physicians increased inpatient ordering rates of influenza and pneumococcal vaccine, but we were limited by physician acceptance of the reminders. We hypothesized that by using the computer system to do the screening for eligibility, we could reach more patients and achieve higher rates of inpatient vaccination than computerized physician reminders. To test this hypothesis, we compared the effects of computerized standing orders for influenza and pneumococcal vaccines in a randomized trial using the computerized physician reminder system as an active control.

We obtained approval for this trial from the institutional review board at the Indiana University Medical Center, Indianapolis. We included all patients discharged from the 6 general medicine wards of Wishard Memorial Hospital from November 1, 1998, through December 31, 1999, a period covering 2 influenza seasons. General medicine wards are covered by 8 physician teams. Each ward is composed of 1 medical staff faculty member, 1 resident, and 2 interns. Individual physicians are assigned to the 8 teams at monthly rotations. Most vaccine ordering is performed by residents and interns. We randomized the physicians to 4 standing order teams and 4 reminder teams. The assignment of physicians to teams has been previously described.^23- 24 However, we did not constrain the random assignment of physicians to the same team status if they returned for more than 1 rotation. Admitted patients were assigned to teams sequentially by the admitting office and assumed the status of the team to which they were assigned.

We used the hospital’s Gopher physician order entry system^25- 26 and Gopher-Care rules²⁷ to identify patients who were eligible for vaccination and to deliver standing orders or physician reminders.

Following national recommendations,^28- 29 the computer system considered a patient eligible for vaccination if: (1) there was no evidence of the vaccine being given during the required time frame; (2) the patient had one of the relevant chronic diseases; or (3) the patient was older than 65 years. The computer interventions for pneumococcal vaccine were active throughout the study. For the influenza vaccine, computer system inteventions were active during the influenza seasons (November 1, 1998, through January 31, 1999, and October 1, 1999, through December 31, 1999).

For eligible patients in the standing order group, the system automatically produced vaccine orders at the time of discharge. Different electronic input forms were required for entering daily orders, admitting orders, and discharge orders. For eligible patients in the physician reminder group, a pop-up message appeared with orders for the required vaccines each time a physician began a daily order-entry session during the first 5 days of hospitalization and when they began a discharge order session at any time. A physician could accept a suggested order with 1 keystroke. However, all order sessions eventually required the physician to save the session with the F8 key and with the user’s password. The computer system stopped sending pop-up reminder messages once it received a vaccine order.

Prior to the study, the executive committee at Wishard Memorial Hospital authorized nurses to administer vaccines in response to computer-generated standing orders and approved the protocol under which they were dispensed. All ward nurses were trained by nursing managers and were given printed protocols, which included questions to ask the patient about egg allergies (for the influenza vaccine), previous vaccination in the relevant time frame, and the patient’s willingness to receive the vaccination. Nurses withheld the vaccination when a patient was unable to answer the screening questions, reported prior vaccinations or relevant allergies, or refused the vaccination. On the basis of the marked benefits and safety of the vaccinations, the institutional review board waived informed consent.

The computer screening rules for finding vaccine-eligible patients and the questions that the nurses asked before dispensing physician-ordered vaccine were identical in the reminder group and in the standing order group. In the physician reminder group, no order was delivered to nurses without the physician’s acceptance. In the standing order group, an order was produced and delivered automatically in the system for all eligible patients.

The order entry system produces a cue to nurses about all new order sessions by displaying a message on workstations located within the nursing station. Nurses must log on to the computer system and click on each order session to signify that they have removed a given set of orders. Vaccination orders were delivered to the nurses via the same mechanisms for both vaccines in the reminder and standing order groups.

We captured information about whether the nurse actually administered each ordered vaccination in both branches of the study via a 2-item questionnaire (Was the vaccination given? If not, why not?). Reminders to complete each questionnaire also appeared on the nursing workstations. Nurses had to log in with their user ID and private password to complete these questionnaires. The computer system required an answer to the first question but not the second. For the influenza vaccine, the nurse had to choose from 5 response options for the second question (already received this year, patient refused, egg allergy, patient unable to answer questions, or other reason). For the pneumococcal vaccine, the response options were the same, except the time frame was “already received in the last 10 years” and the “egg allergy” option was removed.

We obtained detailed information about the reminders delivered during each order entry session, the immunizations ordered, the vaccines dispensed, the reasons ordered vaccines were not dispensed from the Gopher order entry system, the diagnosis related group (DRG), the length of stay and demographic information (including race) from the hospital’s registration and case abstract system, and information about adverse reactions to the 2 vaccines from the pharmacy’s routine adverse event reporting system.

We used SAS statistical software (version 8.2, SAS Institute Inc, Cary, NC) to perform all statistical analyses and considered P ≤.05 to be significant. The first eligible hospitalization was identified for each patient and included for primary analyses. The first eligible hospitalization was restricted to the specific wards in which the study took place. Average ages were compared using t tests; sex, race, and DRG distributions were compared using χ² tests; and average lengths of stays were compared using the Wilcoxon rank sum test.

For each of the vaccinations studied, we tabulated the number of patients who were eligible for vaccination. For patients assigned to the physician reminder group, we also counted the mean number of reminders displayed per hospitalization and tabulated the percentages of eligible patients for whom a physician reminder was responsible for influenza or pneumococcal vaccine orders. Next, we compared the percentage of vaccinations administered among eligible patients in the physician reminder group compared with the standing order group using separate logistic regression models for influenza and for pneumococcal vaccine (both with and without controlling for patient age, sex, race, and physician team). Spontaneous orders were defined as orders entered by physicians during order sessions, which were not associated with a reminder or standing order. In the physician reminder group, admission orders and daily orders after the fifth day of hospitalization were not associated with reminders. Vaccination orders written during such sessions were classified as spontaneous. In the standing order group, physicians never received reminders; so all orders written by physicians for the study vaccines were classified as spontaneous. We credited spontaneous orders to their respective study groups.

We compared the percentage of vaccinations ordered but not administered in both study groups using logistic regression models (with and without controlling for patient age, sex, race, and physician team). Nurses reported the reasons why they did not deliver ordered vaccines in both study groups. We compared the reasons for not administering vaccines using χ² tests.

Because the computer system did not know about all relevant vaccinations given prior to hospitalization, it misidentified some patients as eligible, but not as “truly eligible,” and inflated the denominators for the calculated percentage effects. In the standing order group, we corrected for the computer system’s overinclusion of patients by subtracting the number of patients who reported that they had been vaccinated from the denominator and recalculated all of the relevant percentages. We could not use this same approach in the physician reminder group because nurses only asked the vaccine-related questions in the subset of patients for whom physicians accepted the reminder and ordered the vaccine. Instead, we assumed that the reminder group had the same proportion of previously vaccinated patients as the standing order group and adjusted the denominators by the same proportion. The corrected figures are presented to provide better estimates of the magnitude of the effects compared with other studies that did not depend on the computer system to define their study population. We did not apply any significance tests to the comparison of these adjusted figures between the groups.

Additional statistical analyses were performed that allowed for multiple hospitalizations per patient using appropriate methods for handling correlated data (including generalized estimating equation methods³⁰ for logistic regression). There were no substantive differences in the test of the intervention effect whether we allowed for multiple hospitalizations, controlled for covariates, or adjusted for the clustering within physician team. For the sake of simplicity, we present only the results based on the first eligible hospitalization without adjusting for covariates or physician team, except where noted.

During the 14-month study period, a total of 3777 patients accounted for 5410 discharges from the 6 eligible general medicine wards (Figure). Twenty-four percent of these patients had multiple admissions, with a mean (SD) of 1.40 (1.05) admissions per patient. The mean (SD) age of the patients on these wards was 53 (17) years; 52% were women; and 52% were black. There were neither demographic differences between the 2 study groups nor DRG distribution differences in the 2 patient groups when the 20 most frequently occurring DRGs were used (45% of the total) and the least frequently occurring DRGs were merged into 1 category. The median (range) lengths of stay for the first hospitalization were identical (3.0 [0.5-96.0] days) between the 2 study groups.

A total of 212 physicians participated in the study during an average of 1.9 one-month rotations at the hospital. Thirty-six percent of the physicians (all residents) served on both reminder and standing order rotations. As has been the case in previous reminder studies dating back to 1976,^31- 32 the temporal direction of crossover from physician reminder to nonreminder had no effect on the physician response rate in this study, and we consequently ignored this distinction in the analysis.

During the influenza seasons, a total of 1706 patients accounted for 2369 discharges from general medicine wards (45% of all study patients and 44% of all discharges). There were no significant differences in mean age, ethnicity, or sex between the 2 study groups in the influenza season subset, or between the subset of patients who were admitted during the influenza season and those admitted during the other 8 months of the study. No adverse reactions to the immunizations dispensed during this study were reported.

During the influenza season, the computer system identified 848 patients (50% of all patients hospitalized during these months) as eligible for influenza vaccination. Of these, 385 were associated with standing order teams and 463 with physician reminder teams (Figure). Adjusting for the patients who reported receiving the vaccine prior to hospitalization, 689 patients (313 from standing order group and 376 from physician reminder group; 40%) hospitalized during influenza season were truly eligible for influenza vaccination.

In the physician reminder group, physicians received a mean (SE) of 5.3 (0.2) flu shot reminders per patient hospitalization; 4.0 (0.2) reminders in those cases when the physician eventually ordered the vaccine and 6.9 (0.4) reminders when the vaccine was not ordered. Physicians tended to wait until late in the hospital stay or until the discharge order to accept the reminders. Physicians in the reminder group wrote influenza vaccination orders for 269 (58%) computer-eligible patients, 12 of which were spontaneous orders. In the standing order group, by design, the computer system generated an influenza vaccination order for 371 patients. In addition, 14 orders were spontaneous.

Because vaccines only provide benefit when administered, we used the administration rates as our primary outcome measure. A total of 163 (42%) eligible patients treated by physicians on the standing order team received an influenza vaccine compared with 137 (30%) patients treated by physicians on the reminder team (P <.001). Neither age, race, nor sex was significant in the covariate-adjusted model. After correcting the denominator for 19% of patients who reported to a nurse that they had already received the proposed vaccine prior to hospitalization, the administration rate became 52% of patients in the standing order group and 36% of patients in the reminder group.

The computer system identified 829 patients (22% of patients hospitalized during the 14-month study) as eligible for pneumococcal vaccination. Of these, 406 of those patients were associated with the standing order group and 423 with the reminder group (Figure). Adjusting for the patients who had received the vaccine prior to hospitalization, 766 patients (375 were associated with the standing order group and 391 were associated with the reminder group) (20%) hospitalized during the study were truly eligible for pneumococcal vaccination.

The computer system displayed a pneumococcal vaccine reminder to physicians a mean (SE) of 5.3 (0.2) times per hospitalization —3.6 (0.3) times in cases when the physician eventually accepted the order and 6.8 (0.4) times when he/she did not. Physicians assigned to the reminder group wrote pneumococcal vaccine orders for 208 (49%) of eligible patients, 21 of which were spontaneous. All patients in the standing order group received vaccination orders, 29 of which were spontaneous and 377 of which were generated by standing orders.

Patients in the standing order group received pneumococcal vaccines more often (209; 51%) than patients in the physician reminder group (132; 31%) (P <.001; Table). Age, race, and sex were not significant in the covariate-adjusted model. Correcting for the 7% of patients who reported they had already received the pneumococcal vaccine, the comparison between groups is 56% and 34%, respectively, which is similar to the corresponding rates among patients truly eligible for influenza vaccine.

Nurses withheld vaccines in a large percentage of cases based on the patient’s response to the preadministration questionnaire: 222 (58%) of influenza vaccine-eligible patients in the standing order group and 132 (49%) patients for whom the physician had ordered an influenza vaccine. This effect was significant (P = .03) without adjusting for physician group, but tipped to P = .07 when adjusting for physician group. Nurses withheld vaccines in 197 (49%) of the pneumococcal vaccine–eligible standing order patients and in 76 (37%) patients for whom the physician had ordered pneumococcal vaccine (the effect was P = .004 and after adjusting for physician team was P = .007).

The nurses reported reasons for nonadministration in 98% of influenza vaccine orders and pneumococcal vaccine orders. There was no significant difference between the study groups in the distribution of these reasons for withholding the vaccine. In the standing order group, a total of 32% of the nonadministered influenza vaccines and 16% for the nonadministered pneumococcal vaccines were due to the patient stating he/she had already received the vaccine prior to hospitalization. We used these figures to adjust for the computer system’s incomplete data about vaccine administration in our corrected estimates of the effects.

In the standing order group, patient refusal accounted for 42% of the nonadministered influenza vaccines and 49% of the nonadministered pneumococcal vaccines. Correcting for the patients who had already received the vaccine unbeknownst to the computer system, those percentages become 63% of the nonadministered influenza vaccines and 58% of the nonadministered pneumococcal vaccines. Five percent of nonadministration of the influenza vaccine was explained by the patient’s inability to respond to questions from the nurse, 2% by an egg allergy, and another 18% by other reasons not further distinguished. The pattern was similar for pneumococcal vaccine. Eight percent of patients could not respond to questions from the nurse and another 28% provided other reasons not further distinguished.

As argued previously,^22,33 hospitalization represents an important opportunity to deliver pneumococcal and influenza vaccines to persons particularly likely to benefit from them. We found that such opportunities for inhospital vaccination arise frequently. Adjusting for those who had already received the vaccine, 40% of general medicine ward patients hospitalized during the influenza seasons during our study were truly eligible for the influenza vaccine and 20% hospitalized during our study were truly eligible for the pneumococcal vaccine. The percentage eligible for influenza vaccine would be even greater given the recent change in eligibility criterion from age 65 years to age 50 years.¹⁰ We found that 19% of the patients who were eligible according to the computer system for influenza vaccine and 7% of those eligible for pneumococcal vaccine had previously received the vaccine. Because institutional medical record systems will typically have such gaps in information about the patient’s vaccination history, additional data collection such as nurse preadministration questions is required.

We have previously demonstrated that computerized physician reminders can dramatically increase hospital vaccine ordering nearly 50-fold.²³ In this study, we used a stricter measure of success: the rates at which these vaccinations were actually administered. Standing orders produced a higher rate of vaccine administration than physician reminders. Of the patients eligible for influenza vaccine per the computer system’s criteria, 42% received the vaccine in the standing order group compared with 30% in the reminder group (P<.001); for pneumococcal vaccine, the values were 51% and 31%, respectively (P<.001). Correcting for patients who received vaccines prior to hospitalization not recorded in the computer system, 52% of the patients eligible to receive the influenza vaccine actually received the vaccine in the standing order group compared with 36% of the patients in the reminder group; for pneumococcal vaccine, the values were 56% and 34%, respectively (P<.001).

Our study was performed at a single institution so we cannot be certain that the results will be transferable to other institutions. However, owing to the success of this study, the Gopher order entry system now generates standing orders at discharge for eligible adult patients on all but the intensive care wards. While our data are 5 years old, the fact that standing orders outperform reminders could lead to a universal, contemporary way to address many quality-of-care deficiencies.

It should not be surprising that standing orders were more effective than physician reminders because we used identical computer processes to identify eligible patients and nurses asked the patients identical preadministration questions in both groups for both vaccines. The difference in the physician reminder group was that the physician had to accept the computer system’s suggestion before the vaccine could be offered to the patient; physicians did not accept 42% of the reminders for influenza vaccine and 51% of the reminders for pneumococcal vaccine.

The self-reported rates of influenza vaccine and pneumococcal vaccine among those aged 65 years or older in the United States are approaching 66% and 62%, respectively.⁹ However, these national rates count any influenza vaccine given during the previous year and any pneumococcal vaccine given during the patient’s lifetime. Among patients who had not previously been vaccinated, our standing order intervention achieved rates of 52% for the influenza vaccine and 56% for the pneumococcal vaccine within a single hospital stay. In contrast to the results from other studies,^34- 35 we did not find any significant racial disparities in the immunization rate in either vaccine or either branch of the study.

Although the physician’s compliance with immunization reminders increased compared with our previous study,²³ compliance was far from complete. Given that the patient acceptance of ordered vaccines tended to be higher when immunization was stimulated by a physician order than by a nurse standing order, some of this noncompliance was likely due to a physician’s appreciation of patient factors or preferences. Based on informal discussion with physicians, some of it was likely due to physician hesitation to do anything that might delay the patient’s hospital discharge.

In the standing order group, 7% of the nonadministration of influenza vaccine was due to egg allergies or to a patient’s inability (due to mental state or other impairment) to agree to the vaccine. In these groups, the nonadministration was appropriate. Information reported by the nurses explained 18% (influenza vaccine) and 28% (pneumococcal vaccine) of the nonadministration among truly eligible patients by “other reason” without further detail. However, patient refusal accounted for the majority of vaccine nondelivery among 63% of the truly eligible patients for influenza vaccine and 58% for pneumococcal vaccine, which is likely due to the widespread negative attitudes and misconceptions about vaccinations. In a 17 000 patient survey, nearly one third of the patients thought the influenza vaccine caused influenza and/or caused adverse events.³⁵ To come close to the Healthy People 2010 targets³⁶ of 90% influenza and pneumococcal vaccination rates for patients aged 65 years or older in inner-city populations, the medical community will have to work hard to change these negative beliefs.

Computer-assisted nurse standing orders improve inpatient immunization rates more than physician reminders, and would be likely to have the same advantage when applied to many kinds of preventive screening (eg, cholesterol, cancer) in many care settings. Furthermore, their adoption faces fewer barriers because they can be delivered as part of normal nursing workflow through the order management systems used widely in health care institutions today. However, physician vaccination reminders require delivery mechanisms (eg, physician order entry systems) that do not yet exist in most institutions. Computer-assisted standing orders could reduce the high rate of omissions documented by McGlynn et al³⁷ for many preventive care interventions.