Techniques to Improve Physicians' Use of Diagnostic Tests: Title and subTitle BreakA New Conceptual Framework FREE

DIAGNOSTIC TESTING and imaging are essential tools for disease screening, diagnosis, and monitoring. These aspects of medical care represent an enormous expenditure¹ and studies suggest wide variation in test ordering behavior for seemingly similar indications.^{2 - 4} Several factors that may explain this variation include physicians' inability to estimate test performance characteristics, inaccurate interpretation of diagnostic test results,^{5 - 12} and rapid advances in diagnostic technology that make it difficult for clinicians to stay abreast of the most effective testing strategies.

Improving the appropriateness of testing behavior is a major focus of quality improvement, and many interventions have targeted laboratory and radiographic test ordering. Intervention strategies have included educational programs, guideline development and implementation, utilization audits, and economic incentives. Studies of interventions to modify physicians' test ordering behavior suggest that behavior change rarely can be accomplished solely through education, feedback of laboratory utilization data has inconsistent effects, rewarding physicians for improved ordering practices may result in short-term behavior change only, and testing guidelines have a variable effect depending on the implementation of the guidelines.^{13 - 17} Nearly every strategy has had both success and failure, providing limited guidance for designing new or more effective strategies.

Past reviews of this subject^{14 - 18} have not discriminated among tests used for screening, diagnosis, or monitoring—clinical situations that profoundly impact physicians' motivation and willingness to tolerate uncertainty and their test ordering behavior. Interventions aimed at screening generally have used accepted algorithms and attempt to enhance test usage, quite a different challenge than tests used for diagnosis, which are generally overused. No standardized evaluation of the methodology used in these interventions has been applied. Finally, and perhaps most importantly, the literature provides no evidence that the interventions utilized known behavioral theory.

The PRECEDE model of behavior change¹⁸ is a standardized framework that has been used to design successful, large-scale health interventions.^{19 - 20} PRECEDE incorporates 3 types of factors that precipitate or inhibit behavior change—predisposing, enabling, and reinforcing—and can be described through a clinical example. A 45-year-old man complains of chronic lower back pain and sciatica. His physician believes that this patient is at risk for metastatic cancer, leading him to order an imaging study. This physician's perception of his patient's risk for cancer exemplifies a predisposing factor, which includes other cognitive attributes such as attitudes or knowledge, underlying testing behavior. Enabling factors are skills, resources, or structural barriers that facilitate or prevent behavior. Extending the above example, if the physician had access to a specialist in his practice whom he could consult, he might learn that sciatica rarely represents metastatic cancer in patients younger than 50 years.²¹ Hence, proximity to an expert opinion might enable the physician to take a "wait and see" approach instead of ordering imaging. Reinforcing factors reward a specific behavior through feedback. In this example, on follow-up the patient's back and leg pain remits, reinforcing the importance of examining for sciatica and that a wait and see approach is often appropriate in lower back pain.

Predisposing, enabling, and reinforcing factors can facilitate or hinder behaviors. They are not mutually exclusive nor are their effects the same in all settings. For instance, a diagnostic guideline outlining indications for imaging studies printed on a radiology ordering form may guide physicians who do not know current recommendations, thus changing attitudes or predisposing factors, but reinforce the ordering practices of others who are more knowledgeable. These guidelines may also act as a memory aid for other clinicians who have heard about the recommendations, but do not remember them. Thus, the guideline-based order form can act to enable appropriate ordering.

Analysis of behavior change in terms of predisposing, reinforcing, and enabling factors has proven robust in behavior change planning for a variety of behaviors at many levels, including individual, institutional, and community.¹⁸ These groupings of variables encapsulate much of what has been learned empirically in the fields of psychology, sociology, and planned change. Because most behaviors are determined by multiple factors, it follows that interventions that address several key factors are more likely to be successful than unidimensional interventions. This observation was made by Oxman et al,²² who have developed a typology of interventions to change physician behavior. We elected to use the PRECEDE framework to categorize studies in our review since we believe it has greater explanatory power.

We reviewed the literature on interventions to modify physician testing behavior for methodologic rigor and whether known and effective behavioral principles were utilized. We hypothesized that multidimensional interventions that target more than 1 behavioral factor would be most effective.

We searched the MEDLINE, EMBASE, and HEALTHStar databases for articles published between 1966 and January 1, 1998. Paired MeSH (medical subject heading) terms used for key and text word searching included physician behavior, physician practice patterns, laboratory use, test ordering, decision making, practice guidelines, appropriateness, and education. In addition, we contacted researchers in health services, health behavior, and behavior modification for proprietary and other nonpublished references. In addition, references of all articles obtained were scanned to identify others of potential interest.

Few randomized controlled trials of interventions to modify diagnostic test behavior exist; thus, we evaluated randomized and nonrandomized studies. Studies reviewed met the following criteria: (1) the behaviors of an intervention and a comparison group were examined; (2) the test(s) was used for diagnostic purposes and not solely for screening; and (3) the study was published in English.

We built on prior methodologic standards for studies of therapeutics and supplemented them with behavioral criteria and developed an ad hoc methodologic scoring schema (Table 1).^{23 - 24} In our schema, more points corresponded to higher quality, with a maximum score of 38. The features we examined in these criteria included the patient and physician populations, the health care system, the intervention, and analysis including measurement of the outcomes.

Patient characteristics that may impact on the decision to test, such as demographic and clinical characteristics, must be balanced or controlled for in the analysis. We examined what methods were used for allocating physicians to control or intervention groups. Quasi-experimental study designs were categorized into those with prospectively allocated intervention and control groups vs those with only a historical control group. To assess generalizability of results, we looked for a description of the demographic characteristics, specialties, and levels of experience of the ordering physicians. These physician attributes had to be balanced between groups and/or controlled for in the analysis.

We examined whether authors described important environmental factors that might influence the use of diagnostic tests, such as patients' health insurance status. This criterion was not applied to interventions from countries with nationalized health insurance systems.

If physicians do not believe that the diagnostic testing intervention will improve care based on clinical evidence, behavior change may not occur. Hence, we examined the strength of evidence supporting a particular intervention. Interventions based on published evidence in peer-reviewed publications were rated highest and those based on consensus were rated lowest. Other aspects of the intervention that are important from a methodologic standpoint include whether it can be replicated based on the description and whether it is generalizable to other settings, for example, from hospitalized patients to ambulatory settings.

We suggest that interventions attempting to modify physicians' testing practices must be based on sound behavioral science; thus, we examined whether a behavioral framework was mentioned explicitly and whether attitudes that may mediate the intervention's effect were measured. Interventions were categorized according to which behavioral factor(s) was primarily targeted in the intervention: educational interventions were considered to target predisposing factors; utilization or charge audits primarily act to provide feedback and thus were included under reinforcing factors; and changes in the administrative structure of test ordering (such as order form revisions) were analyzed as enabling interventions.

Outcomes, when not resource utilization, should be measured with validated instruments in a blinded fashion. We examined whether appropriate end points were measured and whether the effects were desirable. Educational and behavioral interventions are often short-lived; thus, the durability of the results needs to be studied beyond the intervention period. Unintended changes in patient management should be examined. For example, an intervention to reduce imaging of the back might actually increase patient anxiety and prompt an increase in physician visits. Finally, we looked for appropriate power calculations to see if the results were statistically robust.

Two of us (D.H.S. and H.H.), blinded to the authors and institution studied, independently applied these criteria and the readers' agreement was examined by calculating an intraclass correlation.²⁵ A mean methodologic quality score was calculated for each study and these were aggregated appropriately. Studies that reported reduced test volume or charges during the intervention period were considered successful. The Student t test was used for analysis of comparative differences between study types.

One hundred two studies were identified through our search methods and 49 (48%) met all inclusion criteria and were reviewed. Of the other 53 excluded papers, 41 (40%) had no clear description of an intervention group and 12 (12%) examined interventions to change the use of tests primarily for screening. Agreement between the 2 independent blinded reviewers in the methodologic quality scores was high, with an intraclass correlation of 0.92. The methodologic quality scores were generally low (Table 2). No study was given more than 26 points (of a possible 38), with a mean±SD score of 13±4.4. Demographic characteristics of patients were described in only 17 studies (35%). Physicians' age and sex were outlined explicitly in only 1 study (2%). Eight studies (16%) used randomization, 19 (39%) included intervention and control groups, and the remaining 22 (45%) had only a historical control group. Two studies (4%) mentioned a behavioral framework as the basis of their intervention and 10 investigators (20%) examined physicians' intermediate attitudes. Eight (21%) of the studies performed in countries without nationalized health insurance discussed the insurance status of patients. Finally, 33 studies (67%) targeted physicians in training, 18 (37%) observed test ordering for 12 months or longer, and 22 (45%) examined test ordering after the intervention ended.

Even though the methodologic quality was low, 76% of the interventions reported reduced volume and/or charges for the test(s) being targeted (Table 3 and Table 4). Not all the studies, however, calculated tests of statistical significance. Sixty-two percent (13/21) of interventions aimed at 1 behavioral factor were successful in changing behavior (Table 3), while 86% (24/28) of interventions targeting more than 1 behavioral factor reported success (Table 4) (P=.12). No statistically significant differences between success rates of interventions targeting predisposing, reinforcing, or enabling factors were evident, but there was inadequate power to find small differences.

Three (60%) of the 5 interventions involving exclusively physicians' knowledge or attitudes (predisposing factors) were successful. In this category, successful interventions targeted selected tests and the duration of effect was unclear.^{26 ,28 ,30} Several researchers coupled an educational intervention (predisposing factors) with reinforcements, such as utilization audits, and successfully changed behavior.^{47 - 50 ,52 - 53} However, when utilization audits (reinforcing factors) were used without accompanying educational material (predisposing factors), rates of behavior change were lower, with 4 (36%) of 11 interventions producing no change.^{31 ,33 ,35 ,41}

Changing the system or environment to enable behavior change was effective in selected situations. Three (60%) of 5 interventions targeting enabling factors produced change.^{43 - 45} Utilization audits used in conjunction with interventions that remove barriers to behavior change were successful, with 9 (75%) of 12 reporting positive results.^{54 - 59 ,62 - 64} When paired with educational strategies, enabling factor interventions produced change in every study reviewed.^{66 ,68 - 74}

We also examined whether more recent interventions were more successful at producing behavior change. The success rate for studies published before 1985 (59%) was slightly lower than that of studies published since 1985 (81%); however, this difference did not reach statistical significance (P=.12).

We reviewed the English-language literature concerning changing physicians' test ordering behavior, applied methodologic criteria, and overlaid a behavioral framework. The overall quality of this literature was not optimal, but results were generally positive, perhaps a manifestation of publication bias. Interventions that targeted multiple behavioral factors were more successful at producing change, a finding noted previously by Oxman et al.²² Several intervention types were common and warrant separate discussion.

Exercises to develop consensus among providers or other educational meetings were common but relatively weak interventions. Such interventions can be viewed as targeting attitudes or knowledge (predisposing factors). Often a consensus conference led to development of guidelines for appropriate testing, which were then disseminated as the intervention. Mozes et al⁷¹ found distribution of guidelines ineffective, but when combined with a change in the laboratory ordering form requiring justification of the test to be performed, ordering volume was reduced. Other selected studies also documented that combining a consensus conference with an audit was effective.^{48 ,62 ,69 - 70} Traditional education, aimed at predisposing attitudes or factors, was a weak intervention, but necessary and effective when coupled with strategies to reinforce attitudes or enable the desired behavior.

Utilization audits or presentation of laboratory charges to physicians were variably successful at producing behavior change. Audits can reinforce a desired testing practice but were often implemented without the proper baseline education. Interventions that first educated physicians as to the optimal diagnostic practice, a predisposing factor, and then gave physicians detailed comparative utilization data produced behavior change.^{47 - 50 ,52 - 53} We believe that utilization audits can act to reinforce desired diagnostic practices, but only if delivered to physicians who understand, or are predisposed to, the optimal testing strategy.

Several interventions explicitly used continuous quality improvement programs. This process entails assessment, consensus building, targeted behavior change, and reassessment; it has been recognized as an effective means for quality improvement and behavior change in the health care setting.⁷⁵ For instance, Dowling et al⁵⁰ and Nardella et al⁵⁸ assessed baseline utilization of selected tests, developed and disseminated guidelines for test ordering, and then reassessed utilization. Both interventions reduced ordering and increased appropriateness. In addition, Dowling et al found that frequency of test use returned to baseline after the intervention, underscoring the need for continuous assessment.

Environmental or administrative interventions can be incredibly effective at little expense; however, they must be chosen carefully. For instance, deletion of the coagulation profile from standard admitting orders⁴⁵ and restricting first-year residents to only 8 laboratory tests per patient⁴³ were both successful interventions. On the other hand, Sussman et al⁴⁶ attempted to eliminate standing orders and did not reduce test orders.

Several limitations should be noted. The proposed criteria for evaluating published studies on improving physicians' use of diagnostic tests build on previous work in this area^{23 ,76} and, though somewhat arbitrary, have face validity. Specific criteria regarding the interventions were added to standardize our considerations of this genre of research. How to weight each criterion to maximize predictive validity is a question for further research.

Since the primary data were generally of low quality, we cannot draw strong conclusions. Many of the studies reviewed were not designed as research projects but as quality improvement programs. As the methodology underpinning this literature improves, we expect more interventions will be designed as research projects, allowing for more rigorous evaluation. Additionally, 2 of us who developed the methodologic quality criteria (Table 1) also applied them. Hence, the intraclass correlation was probably higher than it would be for other researchers using the criteria. We look forward to other applications of this scoring system to assess its reliability.

Pressure to use diagnostic tests in a more cost-effective manner continues. Current laboratory and diagnostic imaging practices need to be improved. Many attempts have been made to change physicians' diagnostic practices, with varying success. We reviewed these interventions applying standardized methodologic criteria. Our results suggest that past studies have been of poor quality and that few investigators have applied behavioral science principles.

Future research attempting to change physicians' behavior should incorporate relevant behavior change models, such as PRECEDE, and apply rigorous evaluations over longer periods of observation. Interventions should target multiple behavioral factors since multidimensional interventions appeared more successful than those aimed at 1 level. When utilization data are being supplied to physicians as part of an intervention, it should be coupled with educational material and documentation of clinical evidence to predispose physicians to the intended behavior. Finally, enabling factors that facilitate the preferred diagnostic behavior through blocking improper test orders or defaulting to the intended practice are most potent.