Effect of a Practice-Based Strategy on Test Ordering Performance of Primary Care Physicians: Title and subTitle BreakA Randomized Trial FREE

In many countries, the number of diagnostic tests ordered by primary care physicians is growing, while according to established evidence-based guidelines, many of these tests are seen as unnecessary.^{1 - 3} Possible explanations are test ordering routines that are difficult to change, a more defensive attitude among primary care physicians out of fear of medical errors, or a lack of knowledge about the appropriate use of tests.^{4 - 7} Moreover, patients more actively ask for tests and often attach greater value to test results than is justified by the facts.^{8 - 9} Unfortunately, little is yet known about the negative effects of performing such tests, in terms of, for example, unnecessary exposure to radiation or false-positive results, that may induce fear and anxiety in patients or may result in a cascade of unnecessary further testing.

Given these problems it is challenging to learn how to change test ordering performance effectively and bring it into line with existing evidence or guidelines on optimal testing. Many such attempts have been made with mixed results, showing that successful strategies require a well-balanced combination of interventions.^{10 - 12} We have developed a multifaceted strategy combining personal feedback and guideline dissemination with quality meetings in small groups of primary care physicians. Social interactions were used as an important motivator for change, as physicians learned how colleagues were handling test ordering problems and as they obtained information about the consequences of medical decision making in daily practice.^{13 - 14} The aim of this strategy was to achieve sustained improvements in test ordering, for example, working in line with the national, evidence-based guidelines. The present article describes the changes in test ordering performance resulting from this innovative strategy in a large population of primary care physicians.

Our study was conducted in 5 regions in the Netherlands, each of which made use of the services of a diagnostic center. A diagnostic center is an institute, usually associated with a hospital, where primary care physicians can order tests without referring patients to the hospital. Thirty-seven local groups of primary care physicians linked to 1 of these 5 diagnostic centers were eligible for the study. These groups are a common feature of Dutch general practice, involving teams of primary care physicians collaborating in a specific region. These teams share patient care outside office hours and many of them also engage in continuing medical education. From May until September 1998 the coordinators of the 5 diagnostic centers recruited local groups in their regions to participate.

The strategy consisted of the following elements: personalized graphical feedback, including a comparison of each physician's own data with those of colleagues; dissemination of national, evidence-based guidelines; and regular meetings on quality improvement in small groups. The strategy focused on specific clinical problems and the diagnostic tests used for these problems (Table 1). These tests covered about 90% of all tests a primary care physician can order in a diagnostic center. For the tests used in the trial, national guidelines for optimal test ordering had to be available.

During the first 6 months of 1999, each of the recruited physicians received by mail 3 consecutive feedback reports on 3 different clinical problems, together with concise information on the 3 evidence-based clinical guidelines for these problems, developed by the Dutch College of Primary Care Physicians.

Each postal contact was followed by a 90-minute standardized small group quality improvement meeting about 2 weeks later, supervised by the medical coordinator of the diagnostic center. At the 3 meetings, physicians were asked to discuss and compare their feedback reports with colleagues and to relate them to the national guidelines. They also discussed Bayesian decision rules to help them understand the probability of false-positive results in low-prevalence disorders. Another important topic of debate was how to deal with the frequent requests by patients to have inappropriate tests performed. This discussion of the guidelines was followed by a thorough discussion of the difficulties of achieving changes at the individual primary care physician level, the practice level, or at the patient level. The next step was to try to implement the guidelines in their own practice, and at the end of each session, plans were drawn up for change, both at individual and group level. Subsequent meetings were used to evaluate whether targets had been met.

The effect of the intervention was evaluated in a multicenter, randomized controlled trial that was conducted in the first 6 months of 1999 with a balanced, incomplete block design, consisting of 2 arms, with the local group of primary care physicians as the unit of randomization. (Figure 1) One group of local groups (arm A) underwent the strategy with respect to tests associated with the 3 clinical problems allocated to arm A (Table 1), while the other group of local groups (arm B) underwent the strategy with respect to tests associated with the 3 problems allocated to arm B (Table 1). The groups in arm A acted as blind controls for the groups undergoing the arm B intervention, and vice versa. This rigorous design was used to balance the influence of nonspecific effects on the test ordering performance between the 2 arms and to neutralize the Hawthorne effect, that is, the effect that physicians might change their test ordering because they were aware of taking part in a trial.^{15 - 16} After stratification for region and group size, randomization was performed centrally with Duploran, a random numbers program.

The physicians gave informed consent for the retrieval of anonymous data on the numbers and results of all tests ordered. To avoid seasonal influences, the numbers of tests for effect evaluation were assessed during the last 6 months of 1998 (the baseline period) and the last 6 months of 1999 (the follow-up period).

Characteristics of primary care physicians and local groups were collected by means of a written questionnaire. Two effect measures were used to evaluate intervention effects:

1. A decrease in the total numbers of requested tests per 6 months per physician: since most of the recommendations in the national, evidence-based guidelines advise ordering fewer tests, a decrease in the total numbers of tests ordered was regarded as an improvement in patient care. Separate analyses were performed for the 6 different clinical problems.

2. A decrease in the numbers of inappropriate tests as defined in the guidelines (Table 1 and Article ): these tests were regarded as inappropriate for the associated clinical problems for various reasons, for example, because the results of these tests seldomly have an influence on the treatment, because the high likelihood of false-positive results can occur, because better alternatives are available, or because adverse effects to some tests can occur (eg, radiology tests).

Upper Abdominal Complaints

General Malaise, Fatigue, and Vague Complaints

Degenerative Joint Complaints

Do not order radiographs of the joints since the results of these tests have no influence on the treatment

Differences in individual characteristics of the primary care physician were tested for significance with Pearson χ² test. In the evaluation of intervention effects, the unit had to be the local group of primary care physicians because this unit was also the unit of randomization. To account for clustering within local groups, a 3-level model was used with the local group as level 3, individual physicians as level 2, and numbers of tests as level 1. The analysis was carried out with SAS PROC MIXED, release 8.2 (SAS Institute, Cary, NC). Power calculations based on the baseline data showed that each arm needed approximately 85 physicians to detect a 10% difference in mean total numbers of tests with 80% power, and a risk of type 1 error of .05. All effects were analyzed with analysis of covariance using the numbers of tests during the follow-up period as the dependent variable and the numbers of tests at baseline and the region, which appeared to be an important determinant, as independent variables.

One hundred seventy-four primary care physicians, belonging to 26 local groups, expressed their willingness to participate on first request, so no further recruitment was necessary. After randomization, both arms included 13 local groups (Figure 1). No differences were found among the characteristics of our individual study primary care physicians (Table 2). Likewise, no differences were found in the characteristics of the local primary care physician groups (data not shown). The mean size of the local groups and experience with continuing medical education in small groups of colleagues did not differ between the 2 arms, nor was there any statistically significant difference between the 2 arms in the mean numbers of tests during the baseline period (data not shown). In multilevel analyses, the point estimation and SD were about the same as in the analysis of covariance at individual physician level and therefore no correction for local groups was needed, even though the intraclass correlation coefficient for block A tests was .12 and that for block B tests was .10.

All the changes in the intervention group were in agreement with the national evidence-based guidelines (Table 3), that is, the represented reductions in the numbers of tests ordered. The number of tests ordered were always larger in the intervention arm than in the control arm. The primary care physicians in arm A decreased the total mean numbers of tests relating to problems allocated to arm A by 12% between baseline and follow-up, while no change in the numbers of these tests occurred for primary care physicians in arm B (blind control arm). The decrease for physicians in arm A was 67 tests more per physician compared with the decrease for the physicians in arm B (P = .01). The physicians in arm B achieved a decrease of 8% in total number of tests ordered for the problems allocated to arm B between baseline and follow-up, while a 3% decrease was achieved in the numbers of these tests by physicians in arm A (blind control arm). These results correspond with an additional decrease in the total numbers of tests for problems allocated to arm B of 28 compared with the physicians of arm A (P = .22).

The results per clinical problem also are shown in Table 3. The mean change in numbers of tests ordered for the 3 clinical problems allocated to arm A was statistically significant (cardiovascular, P = .01; upper abdominal, P = .01; lower abdominal, P = .02), while the change in the numbers of tests ordered for the 3 clinical problems allocated to arm B was in agreement with the recommendations in the national guidelines, although each failed to reach statistical significance.

The reduction in the total numbers of inappropriate tests is shown in Table 4. After the intervention, significantly fewer total inappropriate tests for the problems allocated to arm A were ordered by the primary care physicians in this arm (P = .01). The total numbers of inappropriate tests for the problems allocated to arm B ordered by the primary care physicians in arm B also tended to decrease, which was in agreement with the recommendations in the guidelines, but the reduction failed to reach statistical significance (P = .11). A significant reduction in the numbers of tests ordered, compared with the control group, was found for 4 of the tests for upper abdominal complaints: amylase, bilirubin, lactic dehydrogenase, and alkaline phosphatase.

A new strategy to influence test ordering performance was evaluated in a trial with a large group of primary care physicians in 5 diagnostic center regions in the Netherlands. The relatively short intervention period resulted already in a substantial reduction in the total numbers of tests ordered and in the number of inappropriate tests ordered. Although the effects may seem not very large, it is important to realize primary care physicians in the Netherlands already order fewer tests than their colleagues in other countries.¹ This further reduction can be regarded as quality improvement in terms of test ordering because these changes were in agreement with the recommendations in national evidence-based guidelines.

There are some methodological considerations. We have no reason to believe that the large study population differs from the Dutch primary care physician population. Items relevant for the determinants of test ordering performance of primary care physicians were distributed equally over both arms.¹⁷ However, maybe only motivated, well-functioning groups of physicians participated, and it is therefore questionable if the strategy will work for all groups. Secondly, our study only evaluated effects on volume of tests, because patient data were not available from the diagnostic centers. However, available empirical evidence shows that a general reduction in test ordering in primary care does not lead to more referrals or substitution of care.¹⁸ Furthermore, despite that the guidelines state that a reduction in total test ordering equals quality improvement, this does not implicate that each separate test should always decrease. Finally, the duration of the study is too short to determine long-term effects on test ordering.

Our study underlines that multifaceted interventions are superior to single interventions.^{19 - 20} Significant changes in numbers of tests were not found for all clinical problems included, so conclusions about the effectiveness of our strategy are not straightforward. Some clinical problems may require additional strategies, for example, electronic reminders may be necessary to achieve further improvement.²¹ Nevertheless, our strategy would seem to be a powerful effective and tailor-made strategy, which fits in well with routine primary care physician practice in many western countries, is linked to the every day general practice routine, and gives primary care physicians the opportunity to discuss their test ordering performance with colleagues on the basis of actual performance data, making discussions less noncommittal. Discussing feedback reports and guidelines provides physicians the opportunity to change their performance by learning from each other and by learning to implement new strategies. Thus, social influence by peer interaction can be an important motivator for change.¹⁴ Our strategy could also be used for in-hospital teams or other groups of collaborating physicians, as well as for other topics, such as prescription or referral behavior.