Clinical Decision Support and Appropriateness of Antimicrobial Prescribing:  A Randomized Trial FREE

Antimicrobial resistance is a serious public threat that is exacerbated by the gradual withdrawal of the pharmaceutical industry from new antimicrobial agent development.¹ Overuse of antimicrobial agents fosters the spread of antimicrobial-resistant organisms.^2- 3 Despite recent trends that demonstrate reduced outpatient use of antimicrobial agents, prescribing continues to significantly exceed prudent levels.^4- 8 Approximately 50% of courses of ambulatory antimicrobial drugs are prescribed for patients with viral respiratory infections and therefore are not clinically indicated.^9- 12 Behavioral facilitators of antimicrobial overuse and barriers to prudent use operate on both clinicians and patients.^13- 16 Patient demand, perceived or actual, creates challenges that distinguish this problem from other quality-of-care concerns.^17- 20 A number of studies have tackled the dual drivers of antimicrobial overuse by coupling academic outreach to clinicians with educational programs directed toward patients.^21- 27

Herein, we describe a 12-community randomized trial in which 2 different strategies to enhance appropriate use of antimicrobials for acute respiratory tract infections were compared. One arm of the randomized trial received a community intervention alone, while the other received the community intervention plus a direct intervention with primary care clinicians. The clinician intervention incorporated stand-alone decision support tools on paper or a handheld personal digital assistant (PDA) for the management of acute respiratory tract infection at the point-of-care. Thus, the major purpose of the study was to measure the added value of a clinical decision support system (CDSS) when coupled with a community intervention.

The community was chosen as the unit of randomization because it was a definable entity and the logical foundation for implementing a population-based intervention. Eligible rural communities were those in Utah and Idaho that contained at least 1 primary care clinic and inpatient facility. Each was located in a nonmetropolitan area based on the definition from the US Office of Management and Budget—an area with a total population of less than 100 000, consisting of cities with populations no greater than 50 000 each. Three rural communities that were involved in other antimicrobial use interventions were excluded. The remaining eligible communities were divided according to state and population-size category (<25 000 and ≥25 000). From each state, 4 smaller (<25 000) communities and 2 larger (≥25 000) communities were randomly selected. Then, one larger community from each state was randomly assigned to the combined community intervention plus CDSS intervention (hereafter referred to as the CDSS arm) and the other to the community intervention–alone arm. Next, 2 smaller communities from each state were randomly assigned to the CDSS arm and the other 2 to the community intervention–alone arm. Altogether, each intervention arm comprised 6 communities; the 6 communities from each state were evenly divided between the CDSS and community intervention–alone arms. Six other communities eligible for participation, but not selected, served as a randomly chosen nonstudy reference group for the analysis of retail pharmacy data.

The characteristics of the 12 study communities and 6 nonstudy communities are shown in Table 1; community-level data on race and ethnicity were obtained from the 2000 Census and are included for descriptive purposes. The distribution of clinician types across the 3 groups of communities was similar. The communities were dispersed widely through the 2-state region, minimizing potential for contamination. Other types of interventions, such as change in pharmacy detailing, were not implemented in the study communities during the period of the study.

Approval for the study was granted from both the University of Utah and the Western Institutional Review Boards. Both approved a waiver of informed consent since deidentified existing data were used. Furthermore, both institutional review boards determined that the use of the decision support tools did not require informed consent, since clinicians were not required to follow the guidelines advocated by the tools.

The intervention was disseminated in 2 waves with the first beginning in January 2002 and the second beginning in November 2002.

The community intervention was explicitly designed to map to constructs within behavior change models.^28- 30 The initial launch of the community intervention on the topic of antimicrobial resistance consisted of introductory meetings with community leaders, news releases in the print media, distribution of educational materials at pharmacies and physician offices, and a mailing to parents of children aged younger than 6 years. The educational materials included examination room posters and brochures about appropriate antimicrobial use and were printed in Spanish and English. The mailing consisted of a 5 × 7 postcard, with a refrigerator magnet attached to the upper right-hand corner. The key message was, “Do not treat viral infections with antibiotics.”

The second wave of the community intervention focused on 2 specific types of patient behaviors: (a) how to self-manage common respiratory tract infections; and (b) how to improve communication with your clinician. Individuals seeking care for acute respiratory tract infection were advised to ask their clinician the following 4 questions: (1) Is my illness caused by a virus or bacteria? (2) When should I expect to feel better? (3) When should I call your office or come back if I’m not feeling better? (4) What can I do at home to feel better? A spiral bound flip chart affixed to a magnet was developed as a guide to self-management of respiratory tract infections. This self-care guide was distributed by clinics at health fairs and special events, and through one-on-one interactions with community residents. In addition, a series of articles containing focused behavioral change messages were distributed to community newspapers.

Three decision support tools were developed to cover a wide variety of acute respiratory tract infections including pharyngitis, otitis media, bronchitis, upper respiratory tract infection, sinusitis, pneumonia, croup, and influenza. Two versions were paper-based and 1 was programmed on a PDA.³¹ Three different formats of the tools were made available in order to give clinicians a choice and thereby, enhance their willingness to participate. One paper version was a patient-initiated chart-documentation tool on which the patient circled answers to questions about specific symptoms. The layout facilitated rapid interpretation of responses by the clinician and guided treatment appropriate for the patient’s diagnosis. The other paper version was an easy-to-use graphical flowchart also designed to lead the clinician to the correct diagnosis and treatment options according to precise criteria.

The PDA-based CDSS generated diagnostic and therapeutic recommendations on the basis of patient-specific information that was input about the suspected diagnosis, such as the presence or absence of specific symptoms and signs. Therapeutic recommendations included over-the-counter medications for symptom control as well as prescription antimicrobials. For pediatric patients, the advice was customized to the patient’s weight and age. For cases of pneumonia, the system also calculated the patient’s pneumonia severity index score.³²

The decision support tools were introduced to primary care clinicians through educational lectures, small group meetings, and one-on-one interactions between primary care clinicians and physician members of the study team. Primary care clinicians were defined for this study as emergency department clinicians, family practice physicians, internists, pediatricians, nurse practitioners, and physician’s assistants. The following steps were undertaken in each community: (1) a local clinician contact was identified and solicited to lend support to the project; (2) an outline of the initiative was communicated at the local hospital’s medical staff meeting; (3) all of the primary care clinicians in the community were invited to a continuing medical education session (that covered the threat of antimicrobial resistance,³³ the problem of overuse of antimicrobials, and evidence-based guidelines for the management of acute respiratory tract infections^31,34- 40) that was delivered jointly by an infectious diseases specialist and a primary care physician on the study team; and (4) primary care clinicians in the community were then recruited to participate in the use of case-specific algorithms, with their choice of format.

Clinicians were asked to use the algorithms on at least 200 consecutive patients with acute respiratory tract infections. Completed paper algorithms were retrieved from the clinicians; PDA cases were transmitted electronically. To offset any additional time needed during patient visits, a remuneration of $3 per case was given to participants up to 200 cases per clinician. Alternatively, clinicians were allowed to keep the PDA if that modality was selected.

Follow-up meetings were held with clinicians between November 2002 and December 2002 to give feedback on community-level antimicrobial prescribing data from the first year, and to recruit clinicians who did not previously participate. This session included a presentation that role modeled communication and management of patient expectations during acute respiratory tract infection visits. Data on clinician-specific prescribing patterns were not yet available at the time of these visits.

Retail pharmacy volume and chart review were the 2 sources of information about antimicrobial prescribing that were used. Projected retail pharmacy antimicrobial prescriptions mapped to the 12 study communities and 6 nonstudy communities according to ZIP codes of clinicians were purchased from a company that specializes in providing data resources to pharmaceutical companies (IMS Health Inc; Xponent database).²² These data, which were derived from transaction records, were supplied as the number of new prescriptions per month per community, divided by antimicrobial drug name, formulation, and strength. The mean coverage of pharmacies in the study communities during the duration of the study period was 79%.

Chart review was performed only in the 12 study communities. Medical records in primary care clinician offices were randomly pulled for review, regardless of the clinician’s participation in the use of CDSS. Case-specific use of CDSS tools was not identifiable from chart review.

Using a detailed data dictionary and guidance manual, nurses trained in chart review extracted information about each acute respiratory tract infection visit. Symptoms, signs, diagnoses, test results, and medications associated with these visits were coded by applying precise criteria. Data were directly entered into a computer-based chart abstraction tool, which included error-trapping logic and data entry rules. Inter-rater reliability testing was performed to ensure consistency of coding.

The diagnosis recorded by the primary care clinician and the condition assigned by the reviewer on the basis of documented symptoms, signs, and test results were cataloged. Discrepancies between clinician and reviewer assessment were resolved by selecting the diagnosis most supportive of an indication for antimicrobial therapy. The diagnoses were categorized into 3 groups according to appropriateness of antibiotic use with labels stating: (1) “never indicated” (acute bronchitis and colds/upper respiratory tract infection); (2) “sometimes indicated” (sinusitis and uncharacterized otitis media or pharyngitis); and (3) “always indicated” (streptococcal pharyngitis, acute otitis media, and pneumonia).^31,41- 42 Only 1% of reviewed visits had clinician and reviewer diagnoses that mapped to discordant antibiotic indication groups.

The number of patients with acute respiratory tract infection reviewed per clinician varied according to the number of primary care clinicians practicing in the community (<15, 40 charts each; 15-30, 30 charts each; >30, 20 charts each). This sample size was designed to achieve an 80% statistical power to detect an absolute difference of 6% in prescribing of antimicrobials between the 2 arms of the study, allowing for an intra-cluster correlation of .05.⁴³

The prespecified analytic plan was to use mixed (multilevel) regression models to compare intervention arms, accounting for the clustered study design.^44- 45 Second order penalized quasi-likelihood was used for parameter estimation.⁴⁶ Confidence intervals and P values were calculated from the Wald statistic. P values less than .05 were considered significant. The statistical software programs of MLwiN 2.0 (Centre for Multilevel Modeling, London, England) and Stata 8.0 (Stata Corporation, College Station, Tex) were used.

Multilevel modeling was selected as the primary method of analysis because the target inference was to estimate the expected community-specific change in antimicrobial use.^47- 48 To assess robustness of the results with respect to choice of statistical model, secondary analyses were conducted using generalized estimating equations to generate population-averaged estimates of effect.⁴⁹ As predicted by theory,^50- 51 generalized estimating equations regression methods produced intervention β coefficients that were modestly attenuated compared with the multilevel regression models, although with narrower confidence intervals. None of the principal conclusions about statistical significance were altered. Only the results of the multilevel models are presented here.

Retail pharmacy data were examined by multilevel Poisson regression using the number of antimicrobial prescriptions per month as the first level and the community as the second level. Extra Poisson variation was permitted. The logarithm of the community population size for each year was incorporated as an offset. The community population size included the central community and its adjacent satellite communities within the same ZIP code corresponding to pharmacy locations. The 2001 to 2003 populations were extrapolated from the 2000 US Census Bureau ZIP code–level counts by applying a growth multiplier, derived by linearly interpolating 1990 and 2000 community level census data. A sensitivity analysis to determine whether varying these population assumptions influenced the interpretation of the intervention effect was performed.

The main terms in the model were study group and time period. The 3 groups of communities were CDSS, community intervention alone, and nonstudy. In this analysis, the 3 time intervals were before the intervention (January 2001-September 2001), first year after the intervention (January 2002-September 2002), and the second year after the intervention (January 2003-September 2003) to maintain an equal number of months for each period. The interaction terms study group × time period after the intervention were used to test the null hypothesis that the change in prescribing in CDSS communities equaled the change in prescribing in community intervention–alone communities. The interaction and intercept terms comprised both fixed and random components to accommodate variation across communities. A sine function was included to account for seasonal fluctuation in the rate of antimicrobial prescribing.⁵² This variable was a strong predictor of rate of antimicrobial prescribing but its inclusion or removal did not influence the estimate of the intervention effect. Separate models were fit to assess effects on total antimicrobial prescriptions and antimicrobial class-specific prescriptions.

Chart review data were analyzed using multilevel logistic regression to model the probability of being prescribed an antibiotic. Level 1 was the respiratory tract infection visit, level 2 was the clinician, and level 3 was the community. When an individual patient experienced multiple acute respiratory tract infection visits within a 1-month period, only the first visit was analyzed. The 2 study groups and 2 time periods, before the intervention (January 2001-December 2001) and after the intervention (January 2002-September, 2003), were compared as main effect terms; the 2 periods after the intervention were combined into 1 interval to facilitate model convergence on stable estimates. Intercept and post-intervention terms included both fixed and random components. The interaction term CDSS group × postintervention, was used to statistically test the null hypothesis that the change in probability of prescribing an antibiotic for infections belonging to the diagnostic category antibiotics never indicated, was equal in the 2 groups of communities. Inserting the fixed components of the model into the logistic equation for predicted probabilities, the proportions of patients who received antimicrobial drugs within each study arm and intervention period were estimated. This allowed communication of the results in the more readily understood terms of relative risk and risk difference rather than odds.⁵³

Selection of drug class, given that an antimicrobial was prescribed at the acute respiratory tract infection visit, was also examined. This model comprised acute respiratory tract infection diagnoses belonging to all 3 categories. Antimicrobial agents were grouped into 4 classes: penicillins, macrolides, cephalosporins, and other. Visits for which an antimicrobial was not prescribed were excluded from this analysis. Penicillin-type drugs, as the most commonly prescribed class, represented the referent category. Model terms were similar to those described previously. Market share percentages (the percent of total antimicrobials accounted for by each drug class) were derived as predicted values from this multinomial-type model.

Multilevel logistic regression was also used to assess whether the magnitude of use of the algorithms was associated with decreased probability of prescribing an antibiotic for diagnoses belonging to the never indicated category within the CDSS communities. The change in odds of prescribing an antibiotic in the time period after the intervention according to the number of algorithms completed was estimated from a model containing: (a) the main effect terms of quartile of number of algorithms used and the period after the intervention; and (b) interaction terms of quartile × the period after the intervention. Models were also constructed that included the interaction term of algorithm use as a continuous variable × the period after the intervention as a test of linear trend.

Community Intervention. A total of 83% to 100% of clinics and pharmacies in each community participated in the distribution of educational materials including posters and brochures (Table 2). Between January 2003 and September 2003, approximately 30 000 self-care guides were distributed through clinics, health fairs, and other special community events.

CDSS Intervention. Forty-five percent (80 of 176) of the primary care clinicians in the CDSS communities attended at least 1 of the annual continuing medical education sessions and 71% used the decision-support tools (Table 2). Fifty three percent of CDSS users began during the first intervention year and the rest initiated participation during the second year. All together, 54% used the PDA version only, 23% used paper algorithms only, and 24% used a mix of PDA and paper algorithms. A total of 20 727 respiratory tract infection cases were completed; 13 424 were PDA, 6354 were paper graphical flowcharts, and 949 were patient-initiated chart documentation forms.

Retail Pharmacy Data. The estimated community-specific rates of antimicrobial prescribing during the year before the intervention ranged from 26 to 198 prescriptions per 100 person-years. Prescribing rates of antimicrobials before the intervention were not significantly different across study arms (Table 3).

Rates of antimicrobial prescribing did not change significantly during the first intervention year. In CDSS and community intervention–alone communities, a nonsignificant decrease of 1% and an increase of 3% from baseline were observed; in nonstudy communities, prescribing rates decreased by 3% compared with baseline. During the second intervention year, prescribing rates in CDSS communities decreased 10% from baseline, whereas in the community intervention–alone communities and nonstudy communities, prescribing rates in 2003 increased by 1% and 6%, respectively (Figure 1). Within CDSS communities, the overall antimicrobial prescribing rate declined by an absolute amount of 0.09 prescriptions per person-year between baseline and the second-intervention year (Table 3). This translated to an expected reduction of 93 antimicrobial prescriptions per month in a rural community with a population size equal to the mean of the CDSS group. The test of whether prescribing decreased more in CDSS than community intervention–alone communities was statistically significant (P  = .03).

In CDSS communities, rates of prescribing of antimicrobials belonging to the macrolide class decreased by 12% (P = .002) during the first intervention year and by 28% (P<.001) during the second intervention year, but remained stable in the other communities (Figure 1). In total, 83% of the macrolide prescriptions were for azithromycin. Prescriptions of other drug classes—cephalosporins and penicillins—declined by 6% and 7%, respectively, in the CDSS communities in 2003 compared with 2001; neither change reached statistical significance.

Chart Review. Overall, charts of 79% of primary care clinicians in the 12 communities were reviewed. A total of 13 081 acute respiratory tract infection visits that took place between January 2001 and September 2003 were abstracted. Fifty-five percent of visits were pediatric and 45% adult. The 5 most common conditions were colds/upper respiratory tract infection (25%), otitis media (18%), pharyngitis (18%), sinusitis (16%), and bronchitis (12%). The distribution of these diagnoses did not change significantly during the 3-year period in either the CDSS or community intervention–alone communities. At baseline, neither the frequency of prescribing of antimicrobial drugs or the distribution of antimicrobial drug classes differed significantly between groups of communities.

Fitting the multilevel model to the data, which was the test of whether antimicrobial prescribing for visits in the antibiotics never indicated category decreased more in the CDSS arm than the community intervention–alone arm and was statistically significant (P  = .03). The estimated relative risk reduction⁵⁴ was 32% in CDSS communities and 5% in community intervention–alone communities (Figure 2). This corresponded with a decrease in antimicrobial prescribing for visits in the never indicated category from 35% to 24% (11% difference) in CDSS communities and from 40% to 38% (2% difference) in community intervention alone–communities. Prescribing of an antimicrobial agent for diagnoses in the sometimes indicated or always indicated groups did not decline in either study group.

Based on an analysis of all acute respiratory tract infections, the proportion of antimicrobial drugs that were macrolides decreased in CDSS communities. In the multinomial model, the reduction in odds of prescribing macrolides relative to penicillins was 50% in CDSS communities (P = .001) (Table 4). This corresponded with a decreased market share for macrolides in CDSS communities from 28% to 17% and increased market share for penicillins from 53% to 63%. In contrast, the odds of macrolide use relative to penicillins increased by 12% in community intervention communities (P>.22), corresponding with a stable market share of penicillins (52%-51%) and a slight increase for macrolides (22%-25%). The fraction of antimicrobial agents represented by cephalosporins did not change significantly in either group of communities.

The magnitude of decrease in antimicrobial prescribing for diagnoses belonging to the category antibiotics never indicated correlated with the number of case-specific algorithms used. Within CDSS communities, clinicians in the highest 2 quartiles of algorithms completed showed a greater decrease in antimicrobial use than clinicians who did not use the tools (linear trend, P = .034) (Table 5).

This trial demonstrated the feasibility, uptake, and benefit of stand-alone, portable CDSS tools for acute respiratory tract infections in rural primary care settings. The CDSS decreased unnecessary use of antimicrobial agents for viral respiratory tract infections and improved antimicrobial agent selection. The macrolide class of drugs, which was not considered first line for any of the common acute upper respiratory tract infections,^37,40 declined the most in use. This experience contrasts with US and European data that have demonstrated a marked upward trend in macrolide prescribing during recent years.^8,11

The aim of recruiting clinicians to use the algorithms for a minimum number of cases was to interfere with automated, inappropriate antimicrobial decision making⁵⁵ and to give practical training in the application of an evidence-based approach to respiratory tract infection management. In an intention-to-treat analysis, it is not possible to define the mechanism by which the intervention worked. However, the analysis of algorithm use supported the inference that its effect was related to the frequency of use. The results, when considered together, strongly suggest that the strategy of repetitive use of a diagnostic and treatment algorithm to ingrain new prescribing habits is a valuable component of a practice change intervention.

The decision-support tools generated case-specific recommendations on the basis of multiple items of individual patient data, a feature that differentiates this CDSS intervention from reference programs or guidelines that are not patient specific and therefore do not meet criteria for CDSS.^56- 60 It was necessary for all patient-specific data to be manually entered into the PDA because the system was not integrated with a practice management system or electronic medical record. This inconvenience was mitigated by the design of the program, in which treatment recommendations were rapidly displayed once minimum criteria for reaching an end node of the decision logic were met.^57- 58 Electronic prescribing programs create the possibility of integrating the respiratory tract infection decision-support logic more seamlessly into clinic workflow, even in the absence of fully computerized medical records.^61- 63

Remote rural communities are settings where CDSS may be particularly beneficial because of limited access to consultation and subspecialty services.⁶⁴ Moreover, the geographic isolation of rural communities is an advantage for community randomized trials because it is easier to define distinct populations. However, the rural setting also poses particular challenges.^65- 67 The large distances between communities limited the opportunity for frequent, direct contacts between rural clinicians and physician members of our study team. Rural clinicians may be more likely than their urban counterparts to have social contacts with patients outside of the therapeutic relationship.^68- 69 Feedback from rural clinicians suggested that perception of patient demand and fear of public censure in the event of a medical mishap may operate at a more intense level in rural communities.

The lack of significant decrease in antimicrobial prescribing in community intervention–alone communities should not necessarily be construed as evidence that this component lacked any effect.⁷⁰ The absence of a CDSS–alone arm precluded determination of whether the community activities enhanced the effect of the clinician intervention. Awareness on the part of clinicians of the messages being delivered to community residents may have facilitated acceptance of the CDSS and willingness to change antimicrobial prescribing practices. When an intervention is directed only at community residents, behavior change may take longer than the period of this study to unfold.

This study had certain limitations. The intervention did not address with equal intensity all of the obstacles to changing antimicrobial drug prescribing.^71- 72 The feedback of information about antimicrobial prescribing given between the first and second year of the intervention did not include clinician-specific data⁷³ because it was not possible to perform chart review at the study mid point. Certain parts of the algorithms, such as the evaluation of severity of symptoms in sinusitis, required subjective interpretation on the part of the diagnostician. Increasing the specificity of rules may be useful to reduce practice variation but conversely may decrease likelihood of adoption.⁷⁴ A shortcoming of the study design was the inclusion of a relatively small number of communities, a reflection of resource restraints. An asset of the study was that the communities were randomly selected from those eligible and not self-selected on the basis of willingness to participate.

Changes in antimicrobial prescribing as assessed by chart review and retail pharmacy data were consistent. The measure based on retail pharmacy prescriptions corresponded most closely with community-level antimicrobial selection pressure. Since acute respiratory tract infections are the dominant indication for outpatient antimicrobial use, this metric was expected to correlate both with the propensity of prescribing antimicrobials for respiratory tract infection and with the volume of visits for respiratory tract infections.⁷⁵ Thus, it had the potential to detect variation in care-seeking behavior for respiratory tract infection symptoms, a possible effect of the community intervention. However, because it was not diagnosis- or individual patient–specific, it was less useful for examining the quality of antimicrobial prescribing and for assessing specific targets of practice change.

An unresolved question is whether the modest decrease in total antimicrobial prescriptions and more substantial reduction in macrolide use induced by the CDSS intervention was sufficient to lessen selection of resistant pneumococci and other bacteria in community populations.^2- 3,76 Decreased prevalence of resistant organisms may not necessarily accompany lowered antimicrobial consumption, in part because resistant organisms have an ability to develop compensatory mutations that ameliorate the fitness costs of resistance.⁷⁷ More potent interventions that sustain greater improvements in antimicrobial use may be needed to adequately control antimicrobial resistance.