0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Original Investigation |

Training Physicians to Provide High-Value, Cost-Conscious Care A Systematic Review FREE

Lorette A. Stammen, MD1; Renée E. Stalmeijer, MSc, PhD1; Emma Paternotte, MD2; Andrea Oudkerk Pool, MSc1; Erik W. Driessen, MSc, PhD1; Fedde Scheele, MD, PhD2,3,4; Laurents P. S. Stassen, MD, PhD5
[+] Author Affiliations
1Department of Educational Development and Research, Faculty of Health, Medicine, and Life Sciences, Maastricht University, Maastricht, the Netherlands
2Department of Medical Education, OLVG Hospital, Amsterdam, the Netherlands
3Department of Medical Education, VU University Medical Centre, Amsterdam, the Netherlands
4Athena Institute for Trans-Disciplinary Research, VU University, Amsterdam, the Netherlands
5Department of Surgery, Maastricht University Medical Centre, Maastricht, the Netherlands
JAMA. 2015;314(22):2384-2400. doi:10.1001/jama.2015.16353.
Text Size: A A A
Published online

Importance  Increasing health care expenditures are taxing the sustainability of the health care system. Physicians should be prepared to deliver high-value, cost-conscious care.

Objective  To understand the circumstances in which the delivery of high-value, cost-conscious care is learned, with a goal of informing development of effective educational interventions.

Data Sources  PubMed, EMBASE, ERIC, and Cochrane databases were searched from inception until September 5, 2015, to identify learners and cost-related topics.

Study Selection  Studies were included on the basis of topic relevance, implementation of intervention, evaluation of intervention, educational components in intervention, and appropriate target group. There was no restriction on study design.

Data Extraction and Synthesis  Data extraction was guided by a merged and modified version of a Best Evidence in Medical Education abstraction form and a Cochrane data coding sheet. Articles were analyzed using the realist review method, a narrative review technique that focuses on understanding the underlying mechanisms in interventions. Recurrent patterns were identified in the data through thematic analyses. Resulting themes were discussed within the research team until consensus was reached.

Main Outcomes and Measures  Main outcomes were factors that promote education in delivering high-value, cost-conscious care.

Findings  The initial search identified 2650 articles; 79 met the inclusion criteria, of which 14 were randomized clinical trials. The majority of the studies were conducted in North America (78.5%) using a pre-post interventional design (58.2%; at least 1619 participants); they focused on practicing physicians (36.7%; at least 3448 participants), resident physicians (6.3%; n = 516), and medical students (15.2%; n = 275). Among the 14 randomized clinical trials, 12 addressed knowledge transmission, 7 reflective practice, and 1 supportive environment; 10 (71%) concluded that the intervention was effective. The data analysis suggested that 3 factors aid successful learning: (1) effective transmission of knowledge, related, for example, to general health economics and prices of health services, to scientific evidence regarding guidelines and the benefits and harms of health care, and to patient preferences and personal values (67 articles); (2) facilitation of reflective practice, such as providing feedback or asking reflective questions regarding decisions related to laboratory ordering or prescribing to give trainees insight into their past and current behavior (56 articles); and (3) creation of a supportive environment in which the organization of the health care system, the presence of role models of delivering high-value, cost-conscious care, and a culture of high-value, cost-conscious care reinforce the desired training goals (27 articles).

Conclusions and Relevance  Research on educating physicians to deliver high-value, cost-conscious care suggests that learning by practicing physicians, resident physicians, and medical students is promoted by combining specific knowledge transmission, reflective practice, and a supportive environment. These factors should be considered when educational interventions are being developed.

Increasing costs of health care are a cause of concern to patients, governments, health economists, and the medical profession around the world.13 The United States has the highest health care expenses, with health care expenditures in 2015 approaching 18% of gross domestic product.4 Leading physician associations, such as the American College of Physicians, the Alliance for Academic Internal Medicine, and the American Board of Internal Medicine, offer educational programs on providing high-value, cost-conscious care.5,6

High-value, cost-conscious care refers to care that aims to assess the benefits, harms, and costs of interventions and consequently to provide care that adds value.7 Although in recent years the harms of increasing health care costs have been acknowledged, bending the cost curve has proved difficult.8 Besides the increase in health care costs and its associated complications such as the accessibility and sustainability of health care, quality cannot always be established.2,9 Interventions targeting physicians and their medical expertise are proposed as a means to reduce health care waste (care that is not beneficial to patients) while maintaining the quality of care.2,7

This review was conducted to gain understanding of how and under what circumstances educational interventions may help practicing physicians, resident physicians, and medical students deliver high-value, cost-conscious care. Insight into this learning process is necessary to develop programs that train physicians in providing such care.

This review was conducted and reported in accordance with the RAMESES publication standards for realist reviews.10 Realist review is a systematic, theory-driven, interpretative, narrative technique developed to analyze heterogeneous evidence (qualitative and quantitative data and/or different outcome measures) to understand the underlying mechanisms of an intervention. Realist review gives priority to understanding what does or does not make an intervention effective over the outcome of the intervention. Included articles were analyzed by searching for information on contexts or settings in which the study was conducted, working mechanisms of the interventions, outcomes of the interventions, and program theories (ie, theories of how and why an intervention causes effect) that could explain effectiveness of the described interventions. The realist review method has a particular strength when designs like randomized clinical trials have produced inconsistent estimates of efficacy and there is no consensus on when, how, and with whom to use these interventions.10

Literature Search

The research team designed a strategy to search the PubMed, EMBASE, Education Resources Information center (ERIC), and Cochrane databases. Search terms were used to identify learners (physicians, residents, medical students, medical education) and topic (cost awareness, cost consciousness, unnecessary procedures). These were combined and searched with and without factors associated with cost containment (cost-effectiveness, risk assessment, value-based care, shared decision-making, practice variation, stewardship of resources). The search was conducted from the inception of each database until September 5, 2015.

Study Selection

After an initial search, titles and abstracts were screened by 3 independent researchers (L.A.S., A.O.P., and Inge Verheijen, Maastricht University) on the basis of inclusion criteria: abstract available, relevant topic, implementation of intervention, evaluation of intervention, educational components in intervention, and appropriate target group. Articles without an abstract were excluded from review. When title or abstract alone did not provide sufficient information, full-text review was done. Articles were assessed using an eligibility form based on the Best Evidence in Medical Education (BEME) abstraction form and a Cochrane data coding sheet (eAppendix 1 in the Supplement).11,12 Articles that were found to be appropriate underwent further evaluation to determine if they met the final inclusion criteria (Box). If during full-text review articles did not meet all 5 inclusion criteria, they were excluded from further review. For a realist review, both qualitative and quantitative articles can be included and analyzed to create a broad range of potential empirical mechanisms of the interventions.10

Box Section Ref ID

Box.
Final Inclusion Criteriaa
  • Article is about at least 1 of the following topics

    • Reducing volume of health care services

    • Reducing health care expenditures

    • Improving quality of care

    • Improving knowledge and/or attitude regarding costs of care

  • Article describes an intervention implementation

  • Intervention is evaluated

  • Intervention contains educational component(s)

  • Target group contains physicians, residents, and/or medical students

aTo be included, articles had to meet all criteria described above.

Data Extraction and Synthesis

To abstract data from the selected articles in order to identify factors that may influence learning by practicing physicians, resident physicians, and medical students, 3 researchers used a merged and modified version of a Best Evidence in Medical Education (BEME) abstraction form and a Cochrane data coding sheet (eAppendix 2 in the Supplement).11,12 Disagreements in the final phase of study selection were resolved by inviting other members of the research team to read articles in full. Data synthesis was guided by realist methodology: for each article, we identified program theories, context, mechanisms, and outcomes through a thematic analysis.13 Recurrent patterns of context and outcome in the data were identified. Next, we sought to explain these patterns by their mechanism to gain understanding of how contexts, mechanisms, and outcomes are related. Both successes and failures can provide valuable information regarding context and mechanism.10 The themes were discussed until consensus was reached within the research team. Particular attention was paid to discrepant examples to ensure that the analysis could account for their occurrence. Quality of the included articles was assessed based on the relevance and rigor of the articles.10 Rigor was defined as to what extent the method used to generate that particular piece of data was credible (internal validity) and trustworthy (reliability) (high or low). Relevance was determined by assessing whether the article contributed to answering the research question (high or low). Assessments of rigor and relevance were conducted by 2 independent researchers (L.A.S. and A.O.P.) and were combined into 1 score: high (rigor high/relevance high), medium (rigor high/ relevance low or rigor low/relevance high), or low (rigor low/relevance low). The reviewers were not blinded to any portion of articles. Disagreement about study selection, data extraction, and data synthesis were resolved through consensus within the research team.

Search Results and Selected Articles

The results of the review process are summarized in the eFigure in the Supplement. A total of 2650 articles were considered for review; 102 articles were appropriate for further selection, of which 79 met the inclusion criteria. The majority of the studies were conducted in North America (78.5%) using pre-post interventional design (58.2%; at least 1619 participants); they focused on practicing physicians (36.7%; at least 3448 participants), resident physicians (6.3%; n = 516), and medical students (15.2%; n = 275). Twenty-eight articles did not report the number of participating practicing physicians, resident physicians, or medical students. More detail regarding the characteristics of the selected articles is shown in Table 1. Of the articles, 87% concluded that their interventions were effective in delivering appropriate care and reducing costs, volume, or unnecessary procedures. Among the 14 randomized clinical trials, 12 addressed knowledge transmission, 7 reflective practice, and 1 supportive environment; 10 (71%) concluded that the intervention was effective. Summaries of articles are provided in Table 2, with more details provided in the eTable in the Supplement.

Table Graphic Jump LocationTable 1.  Characteristics of Included Studiesa
Table Graphic Jump LocationTable 2.  Overview of Included Articles

The data analysis of the included articles concluded that the success of educational interventions preparing future and practicing physicians for the delivery of high-value, cost-conscious care may depend on 3 factors: knowledge transmission, reflective practice, and a supportive environment. These 3 factors were the result of an analytic process in which we first collected potential program theories, contexts, mechanisms, and outcomes as presented in the included articles. Second, we identified recurrent patterns of contexts and outcomes that could be explained by mechanisms found in the included articles. These patterns were extensively discussed within the entire research team to identify overarching factors that influence the learning of practicing physicians, resident physicians, and medical students.

Knowledge Transmission

Increasing knowledge about high-value, cost-conscious care behavior among practicing physicians, resident physicians, and medical students has been associated with reduction in unnecessary or inappropriate health care delivery. Such knowledge interventions specifically focused on 3 subject areas (prices of services and general health economics, scientific evidence, and patient preferences) and were represented in 67 articles (87%).

Prices and General Health Economics

The majority of interventions focused on raising awareness of prices of medical services and on teaching the basics of health economics.1430 The first was done by presenting prices on modified order forms, on pocket cards, or in web-based tools or by visualizing price indicators.14,1618,2124,27 A high-quality study by Ellemdin et al18 used a pocket-sized brochure to inform internal medicine physicians (n = 434) about the costs of laboratory testing and asked them to write these costs on the order form. Over the 4-month follow-up period, there was a 27% to 36% decrease in laboratory expenditures. Teaching physicians the basics of general health economics15,19,20,25,26,28 was done by lectures and interactive sessions informing physicians about the competitive market forces and general health care economic principles, such as the role of insurance companies in price setting.19,26 Although the weekly lectures for medical interns were associated with a nonsignificant reduction in the number of tests ordered during patient admissions,19 the interactive sessions significantly improved physicians’ knowledge of economics and rational prescribing.26

Scientific Evidence

A frequent approach to teaching efficiency was to provide the background evidence underlying indications and guidelines related to medical decision making.14,15,19,21,2426,3163 Before implementing or discussing new,26,32,42,47,53,55,62 modified,44,50,64 or established3234,39,41,43,49,54 guidelines or clinical indications,28,48,56,65 trainees and physicians were informed of the scientific evidence that supported these recommendations. A high-quality study by Lee et al61 implemented a teaching session focusing on guidelines on antibiotic prescriptions among 42 practicing physicians, resident physicians, and medical students. These teaching sessions were accompanied by a checklist addressing dose, route, duration, and appropriateness of prescriptions. Their intervention resulted in an annual saving of $69.42 (baseline, $149.74, vs postintervention, $80.32). Effects associated with such teaching depended on several factors, such as expert involvement, complementary lectures, and timing of information. For example, when guidelines were developed by expert panels, they were accepted and adhered to more readily, especially when constructed in multidisciplinary teams.32,40,44 Additionally, several interventions had an expert deliver didactic sessions or conduct outreach visits on the clinical topic to which the guideline applied.32,34,35,46,47,54,66 Other interventions introduced guidelines using lectures about the key recommendations42,55 or by email notifications.25

Another technique that was often used with the aim of increasing the likelihood that physicians would implement the guidelines was to provide just-in-time scientific evidence. For example, inserting guidelines or recommendations into charts allowed physicians to successfully translate these guidelines immediately into practice.24,33,45,47,54 Complementing guidelines by suggestions to physicians for treatment of selected patients formed the core of several interventions aimed to improve physicians’ prescribing patterns.33,37,45,49,51 Two of these interventions used forms that were inserted in charts of selected patients, which were associated with a significant reduction in the prescription of the targeted pharmaceuticals.33,37 For example, the change from a second-generation cephalosporin to a less expensive but equally effective first-generation cephalosporin occurred in 28% of cases (medium-quality study).33 Moreover, when physicians were able to immediately accept or decline a suggestion to switch to a different drug and to modify their prescription to the more appropriate evidence-informed suggestion, a prescribing reduction of up to 43% was achieved (medium-quality study).37

Patient Preferences

Four studies were identified that sought to stimulate high-value, cost-conscious care by improving understanding of patient preferences.24,39,57,58 Physicians involved patients in the decision-making process and in devising a plan of action. During consultation, physicians discussed risks and benefits of medical services and explored patient concerns to be able to provide the best possible care.24,57 Knowledge of patients’ preferences was deemed sufficient when patients felt comfortable about the shared decision that was made. Educational interventions such as DECISION+ combined evidence and patient preferences in workshops to optimize the prescription of antibiotics for acute respiratory tract infections by residents and practicing physicians57,58; the number of patients opting for immediate antibiotic therapy in this high-quality study declined 16% (49% [n = 70] in the control group vs 33% [n = 81] in the experimental group), but this was not statistically significant (P = .08).57

Reflective Practice

A second derived factor was to stimulate reflective practice in future and practicing physicians to help them gain insight into their performance, with a goal of influencing future behavior.16,21,22,25,29,31,34,35,37,39,40,49,52,61,65,6774 This was achieved by providing feedback, stimulating reflection, or a combination of both, in 56 articles including more than 2039 participants.

Feedback

Feedback to physicians about their performance was most often based on utilization data such as the number of ordered tests or amount of prescribed antibiotics.14,21,22,2830,34,37,43,49,52,55,61,65,6770,7577 This feedback could focus on various elements, which, in turn, influenced how trainees learned from feedback. Feedback could focus on volume (such as numbers of requested tests),43,48,55,7577 costs (eg, total costs of antibiotic prescriptions),14,21,22,28 or appropriateness (eg, extent of adherence to guidelines).67 Feedback could center on physicians’ individual utilization or their utilization in relation to that of others.34,49,52,67,76 Another element of importance was the frequency of feedback.21,40,67 One high-quality study sent emails to 15 physicians in the upper 50th percentile for expenditures to point out the differences in their spending behavior compared with their peers over a 3-month period.67 This measure did not lead to a significant expenditure reduction compared with the control group (per-physician expenditure difference of $654.45; P = .64), which might have been because of the short amount of time between feedback provision and measurement of changes in spending behavior. In contrast, Stuebing and Miner21 made use of more frequent feedback: in their medium-quality study, attending physicians and residents in a surgical department (number not reported) received weekly notifications about the costs, expressed in terms of dollars per intensive care unit patient per day. Despite small increases in total expenditure that occurred each time residents switched services, there was a significant decrease in total laboratory costs.21 At baseline, laboratory expenditures were $147.73 per patient per day, and a decrease of up to 27% (−$39.62) from baseline in expenditures per day was reached.

Reflective Questions

Peer- or supervisor-facilitated reflection on medical decisions and their influence on the quality and cost-effectiveness of care was the focus of several studies.15,16,22,25,27,34,35,39,40,53,55,56,64,66,69,7174,76,78 Supervisors,39,55,72 peers,69,79 and experts27,37,53,56,64,66,76 in several clinical settings were the initiators of reflective discussions.22,25,34,39 For example, they asked reflective questions during morning rounds or grand rounds and in the presence of other health care professionals. Reflective practice was guided by questions such as “What was the indication for this test?”; “What alternatives were available?”; or “Why do you think this specific test is redundant?”16,39 Reflection on one’s own behavior was further stimulated by means of reminder alerts whenever a laboratory request or prescription was ordered that was identified as redundant.73,80 Although determining what classified as redundancy from the perspective of the research team and the participants was difficult, in a medium-quality study73 the alerts appeared to be effective and resulted in cancellation of 69% of redundant tests by physicians (n = 282) compared with 51% in the control group (n = 4769) (P < .001).

Combining Feedback and Reflection

Eight articles used feedback as a starting point for reflection.31,34,43,49,68,69,77,81 The authors chose this combination to counter the potential risk of creating a judgmental setting, which is not conducive to high-value, cost-conscious behavior.49 von Ferber et al69 introduced peer-to-peer review discussion groups (high-quality study). During these sessions, the motives that influence prescribing behavior and attitudes of physicians (n = 79), together with underlying causes of practice variation, were discussed in an open and tolerant environment. This approach was associated with significant decreases in prescription costs, from $853 to $527 for high prescribers and from $469 to $352 for low prescribers (P < .001 for both), and with cost-effective prescription of selected drugs.69

Supportive Learning Environment

The data from 27 articles including more than 521 participants highlighted the essential role of the environment in which educational interventions are implemented. Support at the macro level, such as supportive payment systems, management policies, and reimbursement systems, was emphasized, together with the presence of clinical role models and teachers and a culture of high-value, cost-conscious care.

Macro-Level Support

There was a repeated emphasis in a number of publications (n = 29; 37%) on the effect of macro-level support for their interventions on practicing physicians, resident physicians, and medical students.16,17,19,22,24,25,28,3133,44,47,48,50,55,66,68,77,82,83 The organization was often perceived to frustrate the training of physicians in the provision of high-value, cost-conscious care.36,44,57,8386 This was the case, for example, when physicians felt inhibited by fear of malpractice suits84,85 and pressured to adhere to guidelines.57 Other elements of the health care system that might either positively or negatively influence the delivery of high-value, cost-conscious care were the availability of resources,15,31,34,87 workload,36,66,84 bureaucratic structure,55,84 and access to and transparency of health care costs.16,25 For example, in a high-quality survey study by Post et al16 of 68 residents with an 83% response rate, only 4 claimed that they had adequate access to the costs of care they provided. Furthermore, 59 of these residents claimed that better knowledge of costs would influence their ordering behavior. The use of incentives (financial or otherwise) to modify practice28,47,55,76,84,86 and payment systems17,19,22,28,66,76,77,82,86 are also examples of how macro-level decisions could influence individual physicians. The influence of the reimbursement system on the success of cost-containment interventions was identified by Lyle et al.86 In this high-quality study, it was noted that the structure for financing hospital care could run counter to hospital economics, since a decrease in utilization is related to a decline in third-party payments.86

Clinical Role Models and Teachers

An environment in which learners have role models who are very much committed to delivering and demonstrating high-value, cost-conscious care was often mentioned as an important factor in preparing physicians for sustainable practice.15,19,20,22,34,35,40,47,49,66,79,84 The lack of such clinical role models was described as the most common barrier to lessons learned.22,40,48,84 Some authors noted that residents seeking to learn how to provide high-value, cost-conscious care received little or no support from supervisors or attending physicians. For example, the study by Post et al16 implemented an intervention among 83 internal medicine residents based on knowledge of costs and reflective discussions regarding the appropriateness of health care services for selected patients. Their evaluation included a survey (n = 68); only 26 residents agreed with the statement “My supervising consultants consistently encourage me to consider costs when making medical decisions.” As a result, residents became disheartened by fear of getting into conflict with supervisors or being overruled by them.16,22,40

The importance of the involvement of local role models and respected teachers in educational programs was acknowledged by several studies.19,20,26,28,34,35,47,49,50,57,66,69,76,77,85 To enhance acceptance of educational interventions, people with established clinical expertise or managerial responsibilities or physicians considered to be opinion leaders were given a prominent place in the program.19,47,49,50,57,66,69,76,81,85 Illustrating the influence of teachers on training effectiveness, 1 study asserted that if workshops were not conducted by a highly motivated principal teacher, the effect of the intervention could not be reproduced.57

Culture of Interprofessional Collaboration in Relation to High-Value, Cost-Conscious Care

Learning within interprofessional collaborations compared with individual training of physicians was the focus of 14 interventions.37,4446,48,51,55,61,65,66,68,70,74,79,83,88,89 Associations with learning in collaborations was demonstrated in a medium-quality study by Collins et al,89 who sent participants identical feedback emails discussing the appropriateness of dipyrimidole prescriptions. A physicians-only control group (n = 342) compared with a physicians-pharmacists intervention group (n = 91) spent $37.01 per patient more on dipyridamole (P < .025), even though specific pharmacist action had not been solicited.

The importance of the values and beliefs of coworkers, whether peers or other health care professionals, was highlighted by several other studies.22,25,27,32,33,36,37,3941,46,47,50,51,55,57,64,77,84,88,9092 By raising cost issues in newsletters,27,32,46,50,51,88,90 in posters,32,33,36,37,50,57,64,91 in grand rounds, or at bedside teaching,25,40,50,91 these studies aimed to create a prosustainability environment.22,36,37,55,88 In a high-quality study,55 feedback of participating physicians after an intervention that aimed to reduce the costs of laboratory ordering described that group discussions among staff increased consensus regarding the general understanding of the need for cost reduction in the emergency department. Additionally, the involvement of the nursing staff was defined by the authors as one of the key success factors of the intervention due to active involvement of the nursing staff as moderators in the discussions regarding appropriateness between residents and attending physicians. Declaring cost-effectiveness as the theme of the week was another tactic used to stimulate cost consciousness.41 Three articles discussed the negative effect of the absence of an interprofessional collaborating culture toward high-value, cost-conscious care.39,84,92

To ensure sustainable health care practices, physicians need to be trained to provide high-value, cost-conscious care. This realist review was conducted to examine how and under what circumstances educational interventions may help trainees and practicing physicians deliver such care. Three important elements emerged that could inform development of interventions aimed to train physicians: knowledge transmission, reflective practice, and a supportive environment.

First, knowledge transmission appeared to be pertinent to the cost of care and to health economics, scientific evidence, and patient preferences. In addition to raising awareness of how physicians contribute to health care costs, the literature suggested teaching trainees and practicing physicians how to judge medical value and gain insight into patients’ personal values. The challenge of teaching physicians to deliver high-value, cost-conscious care seems to be to elicit a general understanding of how their medical decisions relate to value. Second, stimulating reflective practice on these different values through feedback, reflective questions, and group discussions incentivized physicians to think critically about medical decisions. The literature suggested that these elements would be most effective when their training goals are also supported by the environment in which trainees work and learn. Third, such a supportive environment may be necessary for clinical teachers, role models, and other health care professionals’ trainees to appreciate the importance of high-value, cost-conscious care. These 3 factors combined provide a framework for the development and further research of educational programs that teach physicians to deliver high-value, cost-conscious care.

The amount of support within an environment may be critical for the success of efforts to train high-value, cost-conscious care. To be successful, transmission of knowledge and reflective practice may work best in environments that feature role models, attention to health care teams, and an organization that supports sustainable practice. An unsupportive environment might inadvertently negate the intended training effect,93 underlining the importance that trainees witness the delivery of high-value, cost-conscious care in clinical practice. There is a risk of adverse effects of learning from poor role models, and good role models may be scarce, as many physicians who provide clinical education were trained with the assumption that costs have no place in medical decision making. Adding to confusion is that the current generation of physicians is practicing in a health care system that features continuously changing payment systems and accreditation requirements, with a legacy of health care costs lacking in transparency and unstable rules and regulations.

This review should be considered in the light of its limitations. First, because of the variability of definitions given to high-value, cost-conscious care, selection bias cannot be excluded. To minimize this risk, no cutoff date was used in the search and all related publications were screened, the oldest articles dating from 1979. Although the field of medicine has evolved tremendously since then, the elements of training inherent in our identified framework and the inherent barriers have remained stable, which reinforces the strength of the findings. A second limitation is the risk of publication bias, which is suggested by apparent effectiveness in 87% of the studies overall and in 71% of the 14 randomized clinical trials. We tried to reduce this bias as much as possible by searching multiple databases and placing no restrictions on the quality of journals. Nevertheless, the possibility of publication bias should be considered. Third, the assessment of quality of the included articles, ie, rigor and relevance, was derived from the professional judgments of the researchers. Although prone to subjectivity, we tried to counteract this through extensive discussions within the research team. Fourth, the factor of supportive environment was partly derived from qualitative data from quantitative studies, not as strong as the evidence used to derive knowledge transmission and reflective practice. Quantitative findings using randomized trials demonstrating an influence of the supportive environment on learning outcomes would have provided stronger evidence.

Although the reported effectiveness of educational interventions seems to provide scope for medical education to bring about improvement, training physicians to deliver high-value, cost-conscious care remains a complex task. Further research should focus on what makes a good role model of high-value, cost-conscious care and how such attributes can be cultivated by means of medical education. Additionally, there is a need to investigate how formal education can help mold the culture of the learning environment. Although measuring the value of care is extremely complex, outcome measures that focus solely on volume or costs might promote the incorrect assumption that cheaper is better. Therefore, thoughtful consideration of which outcome measures can be used to evaluate the effectiveness of interventions remains important.

Research on educating physicians to deliver high-value, cost-conscious care suggests that learning by practicing physicians, resident physicians, and medical students is promoted by combining specific knowledge transmission, reflective practice, and a supportive environment. These factors should be considered when educational interventions are being developed.

Corresponding Author: Lorette A. Stammen, MD, Department of Educational Development and Research, Faculty of Health, Medicine, and Life Sciences, Maastricht University, PO Box 616, 6200 MD Maastricht, the Netherlands (l.stammen@maastrichtuniversity.nl).

Author Contributions: Dr Stammen had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Stammen, Stalmeijer, Driessen, Scheele, Stassen.

Acquisition, analysis, or interpretation of data: Stammen, Stalmeijer, Paternotte, Oudkerk Pool, Driessen, Scheele, Stassen.

Drafting of the manuscript: Stammen, Stalmeijer, Paternotte, Driessen, Scheele.

Critical revision of the manuscript for important intellectual content: Stammen, Stalmeijer, Oudkerk Pool, Driessen, Scheele, Stassen.

Study supervision: Stalmeijer, Paternotte, Driessen, Scheele, Stassen.

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest and none were reported.

Additional Contributions: Angelique van den Heuvel, MA, Maastricht University, provided assistance with English-language editing of the manuscript as a staff member of the department of educational development and research. Jimmie Leppink, PhD, Maastricht University, provided statistical advice during manuscript preparation as a staff member of the department of educational development and research. Inge Verheijen, a final-year medical student at the Faculty of Health, Medicine and Life Sciences at Maastricht University, was involved in the screening of abstracts as a student assistent of our educational department. None of these individuals received additional compensation for their role in this study.

Reinhardt  UE. Waste vs value in American health care. Economix. September 13, 2013. http://economix.blogs.nytimes.com/2013/09/13/waste-vs-value-in-american-health-care/?_r=0. Accessed January 2014, 2014.
Berwick  DM, Hackbarth  AD.  Eliminating waste in US health care. JAMA. 2012;307(14):1513-1516.
PubMed   |  Link to Article
Blumenthal  D.  Controlling health care expenditures. N Engl J Med. 2001;344(10):766-769.
PubMed   |  Link to Article
Altarum Institute. Health Sector Economic Indicators: Insights From Monthly National Health Spending Data Through July 2015. September 11, 2015. http://altarum.org/sites/default/files/uploaded-related-files/CSHS-Spending-Brief_September_2015.pdf. Accessed September 13, 2015.
Duke  P. The ABIM Foundation’s Choosing Wisely communication module. 2013. http://modules.choosingwisely.org/modules/m_00/. Accessed February 2014.
American College of Physicians. High value care curriculum. https://hvc.acponline.org/curriculum.html. Accessed February 2014.
Owens  DK, Qaseem  A, Chou  R, Shekelle  P; Clinical Guidelines Committee of the American College of Physicians.  High-value, cost-conscious health care: concepts for clinicians to evaluate the benefits, harms, and costs of medical interventions. Ann Intern Med. 2011;154(3):174-180.
PubMed   |  Link to Article
Organisation for Economic Co-operation and Development. Health spending, total % of GDP, 1970-2013. In: OECD Health Data: Health Expenditure and Financing: Health Expenditure Indicators. 2013. https://data.oecd.org/chart/4f22. Accessed February 2015.
Cutler  D. The value equation in health care. Presented at: National Health Care Reform: Policy Options and Imperatives. February 23, 2007; Houston, TX.
Wong  G, Greenhalgh  T, Westhorp  G, Buckingham  J, Pawson  R.  RAMESES publication standards: realist syntheses. BMC Med. 2013;11:21.
PubMed   |  Link to Article
Best Evidence Medical Education Collaboration. BEME data coding sheet. http://download.lww.com/wolterskluwer_vitalstream_com/PermaLink/ACADMED/A/ACADMED_86_11_2011_08_19_REINDERS_202668_SDC2.pdf. Accessed June 2014.
Cochrane Library. Data collection form for RCT and non-RCT. http://www.cochranelibrary.com/cochrane-database-of-systematic-reviews/supplements.html. Accessed June 2014.
Pawson  R, Greenhalgh  T, Harvey  G, Walshe  K. Realist Synthesis: An Introduction. Manchester, England: University of Manchester; 2004.
Sussman  AJ, Chabra  B, Coblyn  J,  et al.  Helping primary care physicians lower pharmaceutical expenses: an academic medical center’s experience. J Clin Outcomes Manag. 2004;11(5):290-295.
Spiegel  CT, Kemp  BA, Newman  MA, Birnbaum  PS, Alter  CL.  Modification of decision-making behavior of third-year medical students. J Med Educ. 1982;57(10 pt 1):769-777.
PubMed
Post  J, Reed  D, Halvorsen  AJ, Huddleston  J, McDonald  F.  Teaching high-value, cost-conscious care: improving residents’ knowledge and attitudes. Am J Med. 2013;126(9):838-842.
PubMed   |  Link to Article
Chandawarkar  RY, Taylor  S, Abrams  P,  et al.  Cost-aware care: critical core competency. Arch Surg. 2007;142(3):222-226.
PubMed   |  Link to Article
Ellemdin  S, Rheeder  P, Soma  P.  Providing clinicians with information on laboratory test costs leads to reduction in hospital expenditure. S Afr Med J. 2011;101(10):746-748.
PubMed
Davidoff  F, Goodspeed  R, Clive  J.  Changing test ordering behavior: a randomized controlled trial comparing probabilistic reasoning with cost-containment education. Med Care. 1989;27(1):45-58.
PubMed   |  Link to Article
Englander  R, Agostinucci  W, Zalneraiti  E, Carraccio  CL.  Teaching residents systems-based practice through a hospital cost-reduction program: a “win-win” situation. Teach Learn Med. 2006;18(2):150-152.
PubMed   |  Link to Article
Stuebing  EA, Miner  TJ.  Surgical vampires and rising health care expenditure: reducing the cost of daily phlebotomy. Arch Surg. 2011;146(5):524-527.
PubMed   |  Link to Article
Sommers  BD, Desai  N, Fiskio  J,  et al.  An educational intervention to improve cost-effective care among medicine housestaff: a randomized controlled trial. Acad Med. 2012;87(6):719-728.
PubMed   |  Link to Article
Hart  J, Salman  H, Bergman  M,  et al.  Do drug costs affect physicians’ prescription decisions? J Intern Med. 1997;241(5):415-420.
PubMed   |  Link to Article
Polinski  JM, Schneeweiss  S, Maclure  M, Marshall  B, Ramsden  S, Dormuth  C.  Time series evaluation of an intervention to increase statin tablet splitting by general practitioners. Clin Ther. 2011;33(2):235-243.
PubMed   |  Link to Article
Willens  HJ, Nelson  K, Hendel  RC.  Appropriate use criteria for stress echocardiography: impact of updated criteria on appropriateness ratings, correlation with pre-authorization guidelines, and effect of temporal trends and an educational initiative on utilization. JACC Cardiovasc Imaging. 2013;6(3):297-309.
PubMed   |  Link to Article
Qureshi  NA, Neyaz  Y, Khoja  T, Magzoub  MA, Haycox  A, Walley  T.  Effectiveness of three interventions on primary care physicians’ medication prescribing in Riyadh City, Saudi Arabia. East Mediterr Health J. 2011;17(2):172-179.
PubMed
Roth  EJ, Plastaras  CT, Mullin  MS, Fillmore  J, Moses  ML.  A simple institutional educational intervention to decrease use of selected expensive medications. Arch Phys Med Rehabil. 2001;82(5):633-636.
PubMed   |  Link to Article
Schroeder  SA, Myers  LP, McPhee  SJ,  et al.  The failure of physician education as a cost containment strategy: report of a prospective controlled trial at a university hospital. JAMA. 1984;252(2):225-230.
PubMed   |  Link to Article
Gitelis  M, Vigneswaran  Y, Ujiki  MB,  et al.  Educating surgeons on intraoperative disposable supply costs during laparoscopic cholecystectomy: a regional health system’s experience. Am J Surg. 2015;209(3):488-492.
PubMed   |  Link to Article
Vigneswaran  Y, Linn  JG, Gitelis  M,  et al.  Educating surgeons may allow for reduced intraoperative costs for inguinal herniorrhaphy. J Am Coll Surg. 2015;220(6):1107-1112.
PubMed   |  Link to Article
Das  AK, Rahman  MS.  Prescribing vitamins at primary health care level: exploration of facts, factors and solution. Bangladesh J Pharmacol. 2010;5(2):92-97.
Link to Article
de Leon  N, Sharpton  S, Burg  C,  et al.  The development and implementation of a bundled quality improvement initiative to reduce inappropriate stress ulcer prophylaxis. ICU Dir. 2013;4(6):322-325.
Link to Article
Whiteside  ME, Lefkowitz  S, Justiniani  FR, Ratzan  K.  Changing prescribing patterns: a program of physician education. Hosp Formul. 1987;22(6):561-563, 566, 568.
PubMed
Cammisa  C, Partridge  G, Ardans  C, Buehrer  K, Chapman  B, Beckman  H.  Engaging physicians in change: results of a safety net quality improvement program to reduce overuse. Am J Med Qual. 2011;26(1):26-33.
PubMed   |  Link to Article
Self  TH, Smith  SL, Boswell  RL, Miller  WA.  Medical education provided by a clinical pharmacist: impact on the use and cost of corticosteroid therapy in chronic obstructive pulmonary disease. Drug Intell Clin Pharm. 1984;18(3):241-244.
PubMed
McKay  RM, Vrbova  L, Fuertes  E,  et al.  Evaluation of the Do Bugs Need Drugs? program in British Columbia: can we curb antibiotic prescribing? Can J Infect Dis Med Microbiol. 2011;22(1):19-24.
PubMed
Barbarello-Andrews  L, Susla  G, Ng  V, St John  D, Lau  C.  Cost-effective medication use in critical care: Capital Health System’s experience in VHA’s MUSIC program. J Clin Outcomes Manag. 2006;13(11):615-622.
Marconi  GP, Nager  AL.  Teaching residents established guidelines and standards of care to strengthen their cost-containment practices. Manag Care. 2010;19(5):46-51.
PubMed
Blackstone  ME, Miller  RS, Hodgson  AJ, Cooper  SS, Blackhurst  DW, Stein  MA.  Lowering hospital charges in the trauma intensive care unit while maintaining quality of care by increasing resident and attending physician awareness. J Trauma. 1995;39(6):1041-1044.
PubMed   |  Link to Article
Elligsen  M, Walker  SA, Pinto  R,  et al.  Audit and feedback to reduce broad-spectrum antibiotic use among intensive care unit patients: a controlled interrupted time series analysis. Infect Control Hosp Epidemiol. 2012;33(4):354-361.
PubMed   |  Link to Article
Dowling  PT, Alfonsi  G, Brown  MI, Culpepper  L.  An education program to reduce unnecessary laboratory tests by residents. Acad Med. 1989;64(7):410-412.
PubMed   |  Link to Article
Fakih  MG, Pena  ME, Shemes  S,  et al.  Effect of establishing guidelines on appropriate urinary catheter placement. Acad Emerg Med. 2010;17(3):337-340.
PubMed   |  Link to Article
Verstappen  WH, van der Weijden  T, Sijbrandij  J,  et al.  Effect of a practice-based strategy on test ordering performance of primary care physicians: a randomized trial. JAMA. 2003;289(18):2407-2412.
PubMed   |  Link to Article
Poppleton  VK, Moynihan  PJ, Hickey  PA.  Clinical practice guidelines: the Boston experience. Prog Pediatr Cardiol. 2003;18(1):75-83.
Link to Article
Sleath  B, Collins  T, Kelly  HW, McCament-Mann  L, Lien  T.  Effect of including both physicians and pharmacists in an asthma drug-use review intervention. Am J Health Syst Pharm. 1997;54(19):2197-2200.
PubMed
Ijo  I, Feyerharm  J.  Pharmacy intervention on antimicrobial management of critically ill patients. Pharm Pract (Granada). 2011;9(2):106-109.
PubMed   |  Link to Article
Bernal-Delgado  E, Galeote-Mayor  M, Pradas-Arnal  F, Peiró-Moreno  S.  Evidence based educational outreach visits: effects on prescriptions of non-steroidal anti-inflammatory drugs. J Epidemiol Community Health. 2002;56(9):653-658.
PubMed   |  Link to Article
Bhatia  RS, Milford  CE, Picard  MH, Weiner  RB.  An educational intervention reduces the rate of inappropriate echocardiograms on an inpatient medical service. JACC Cardiovasc Imaging. 2013;6(5):545-555.
PubMed   |  Link to Article
Hux  JE, Melady  MP, DeBoer  D.  Confidential prescriber feedback and education to improve antibiotic use in primary care: a controlled trial. CMAJ. 1999;161(4):388-392.
PubMed
Larmour  I, Pignataro  S, Barned  KL, Mantas  S, Korman  MG.  A therapeutic equivalence program: evidence-based promotion of more efficient use of medicines. Med J Aust. 2011;194(12):631-634.
PubMed
Phillips  L, Landsberg  KF.  Evaluation of a newsletter in altering physicians’ prescribing patterns. Can J Hosp Pharm. 1986;39(4):102-104, 108.
PubMed
Ziskind  AA, Portelli  J, Rodriguez  S,  et al.  Successful use of education and cost-based feedback strategies to reduce physician utilization of low-osmolality contrast agents in the cardiac catheterization laboratory. Am J Cardiol. 1994;73(16):1219-1221.
PubMed   |  Link to Article
Shane  R, Nishimura  L.  Strategic management of therapeutic advances: experience with colony-stimulating factors. Hosp Pharm. 1994;29(9):824-829, 826-829.
PubMed
Weingarten  SR, Riedinger  MS, Conner  L,  et al.  Practice guidelines and reminders to reduce duration of hospital stay for patients with chest pain: an interventional trial. Ann Intern Med. 1994;120(4):257-263.
PubMed   |  Link to Article
Sucov  A, Bazarian  JJ, deLahunta  EA, Spillane  L.  Test ordering guidelines can alter ordering patterns in an academic emergency department. J Emerg Med. 1999;17(3):391-397.
PubMed   |  Link to Article
Okpara  AU, Maswoswe  JJ, Stewart  K.  Criteria-based antimicrobial IV to oral conversion program. Formulary. 1995;30(6):343-348.
PubMed
Légaré  F, Labrecque  M, LeBlanc  A,  et al.  Training family physicians in shared decision making for the use of antibiotics for acute respiratory infections: a pilot clustered randomized controlled trial. Health Expect. 2011;14(suppl 1):96-110.
PubMed   |  Link to Article
Légaré  F, Labrecque  M, Cauchon  M, Castel  J, Turcotte  S, Grimshaw  J.  Training family physicians in shared decision-making to reduce the overuse of antibiotics in acute respiratory infections: a cluster randomized trial. CMAJ. 2012;184(13):E726-E734.
PubMed   |  Link to Article
Braido  F, Comaschi  M, Valle  I,  et al; ARGA Study Group; EAACI/CME Committee.  Knowledge and health care resource allocation: CME/CPD course guidelines-based efficacy. Eur Ann Allergy Clin Immunol. 2012;44(5):193-199.
PubMed
James  E, Cyriac  J.  Impact of educational interventions on the physicians for early switchover of parenteral drugs to oral therapy. European Journal Hospital Pharmacy. 2014;22:176-178.
Link to Article
Lee  TC, Frenette  C, Jayaraman  D, Green  L, Pilote  L.  Antibiotic self-stewardship: trainee-led structured antibiotic time-outs to improve antimicrobial use. Ann Intern Med. 2014;161(10)(suppl):S53-S58.
PubMed   |  Link to Article
Yang  Z, Zhao  P, Wang  J,  et al.  DRUGS system enhancing adherence of Chinese surgeons to antibiotic use guidelines during perioperative period. PLoS One. 2014;9(8):e102226.
PubMed   |  Link to Article
Fortuna  RJ, Zhang  F, Ross-Degnan  D,  et al.  Reducing the prescribing of heavily marketed medications: a randomized controlled trial. J Gen Intern Med. 2009;24(8):897-903.
PubMed   |  Link to Article
Wein  PJ, Hoffman  RP.  Promoting the cost-effective utilization of cefoxitin with a drug use education program. Hosp Formul. 1987;22(3):299-306.
PubMed
Gregory  KD, Hackmeyer  P, Gold  L, Johnson  AI, Platt  LD.  Using the continuous quality improvement process to safely lower the cesarean section rate. Jt Comm J Qual Improv. 1999;25(12):619-629.
PubMed
Pasquale  TR, Komorny  KM, Letting-Mangira  D, Peshek  S.  A pharmacist-physician antibiotic support team. P&T. 2004;29(1):33-40.
Parrino  TA.  The nonvalue of retrospective peer comparison feedback in containing hospital antibiotic costs. Am J Med. 1989;86(4):442-448.
PubMed   |  Link to Article
McPhee  SJ, Chapman  SA, Myers  LP, Schroeder  SA, Leong  JK.  Lessons for teaching cost containment. J Med Educ. 1984;59(9):722-729.
PubMed
von Ferber  L, Bausch  J, Köster  I, Schubert  I, Ihle  P.  Pharmacotherapeutic circles: results of an 18-month peer-review prescribing-improvement programme for general practitioners. Pharmacoeconomics. 1999;16(3):273-283.
PubMed   |  Link to Article
Miyakis  S, Karamanof  G, Liontos  M, Mountokalakis  TD.  Factors contributing to inappropriate ordering of tests in an academic medical department and the effect of an educational feedback strategy. Postgrad Med J. 2006;82(974):823-829.
PubMed   |  Link to Article
Moriates  C, Soni  K, Lai  A, Ranji  S.  The value in the evidence: teaching residents to “choose wisely”. JAMA Intern Med. 2013;173(4):308-310.
PubMed   |  Link to Article
Attali  M, Barel  Y, Somin  M,  et al.  A cost-effective method for reducing the volume of laboratory tests in a university-associated teaching hospital. Mt Sinai J Med. 2006;73(5):787-794.
PubMed
Bates  DW, Kuperman  GJ, Rittenberg  E,  et al.  A randomized trial of a computer-based intervention to reduce utilization of redundant laboratory tests. Am J Med. 1999;106(2):144-150.
PubMed   |  Link to Article
Thakkar  RN, Kim  D, Knight  AM, Riedel  S, Vaidya  D, Wright  SM.  Impact of an educational intervention on the frequency of daily blood test orders for hospitalized patients. Am J Clin Pathol. 2015;143(3):393-397.
PubMed   |  Link to Article
Zimmerman  DR, Collins  TM, Lipowski  EE, Sainfort  F.  Evaluation of a DUR intervention: a case study of histamine antagonists. Inquiry. 1994;31(1):89-101.
PubMed
Zunker  RJ, Carlson  DL.  Economics of using pharmacists as advisers to physicians in risk-sharing contracts. Am J Health Syst Pharm. 2000;57(8):753-755.
PubMed
Sicotte  C, Pineault  R, Tilquin  C, Contandriopoulos  AP.  The diluting effect of medical work groups on feedback efficacy in changing physician’s practice. J Behav Med. 1996;19(4):367-383.
PubMed   |  Link to Article
Krinsley  JS.  Test-ordering strategy in the intensive care unit. J Intensive Care Med. 2003;18(6):330-339.
PubMed   |  Link to Article
Rudy  DW, Ramsbottom-Lucier  M, Griffith  CH  III, Georgesen  JC, Wilson  JF.  A pilot study assessing the influences of charge data and group process on diagnostic test ordering by residents. Acad Med. 2001;76(6):635-637.
PubMed   |  Link to Article
Rotman  BL, Sullivan  AN, McDonald  TW,  et al.  A randomized controlled trial of a computer-based physician workstation in an outpatient setting: implementation barriers to outcome evaluation. J Am Med Inform Assoc. 1996;3(5):340-348.
PubMed   |  Link to Article
Bornard  L, Dellamonica  J, Hyvernat  H,  et al.  Impact of an assisted reassessment of antibiotic therapies on the quality of prescriptions in an intensive care unit. Med Mal Infect. 2011;41(9):480-485.
PubMed   |  Link to Article
Manheim  LM, Feinglass  J, Hughes  R, Martin  GJ, Conrad  K, Hughes  EF.  Training house officers to be cost conscious: effects of an educational intervention on charges and length of stay. Med Care. 1990;28(1):29-42.
PubMed   |  Link to Article
Niquille  A, Ruggli  M, Buchmann  M, Jordan  D, Bugnon  O.  The nine-year sustained cost-containment impact of Swiss pilot physicians-pharmacists quality circles. Ann Pharmacother. 2010;44(4):650-657.
PubMed   |  Link to Article
Zeleznik  C, Gonnella  JS.  Jefferson Medical College student model utilization review committee. J Med Educ. 1979;54(11):848-851.
PubMed
Gist  D, Llorente  J, Mayer  J.  A clinical algorithm for the management of abnormal mammograms: a community hospital’s experience. West J Med. 1997;166(1):21-28.
PubMed
Lyle  CB  Jr, Bianchi  RF, Harris  JH, Wood  ZL.  Teaching cost containment to house officers at Charlotte Memorial Hospital. J Med Educ. 1979;54(11):856-862.
PubMed
Colbert  CY, Ogden  PE, Lowe  D, Moffitt  MJ.  Students learn systems-based care and facilitate system change as stakeholders in a free clinic experience. Adv Health Sci Educ Theory Pract. 2010;15(4):533-545.
PubMed   |  Link to Article
Mallows  JL.  The effect of a gold coin fine on C-reactive protein test ordering in a tertiary referral emergency department. Med J Aust. 2013;199(11):813-814.
PubMed   |  Link to Article
Collins  TM, Mott  DA, Bigelow  WE, Zimmerman  DR.  A controlled letter intervention to change prescribing behavior: results of a dual-targeted approach. Health Serv Res. 1997;32(4):471-489.
PubMed
Ferris  TG, Shea  T, Jacobson  BC,  et al.  A population-based intervention to improve management of patients on chronic acid suppression: a group-randomized trial. J Clin Outcomes Manag. 2005;12(1):35-43.
Landgren  FT, Harvey  KJ, Mashford  ML, Moulds  RF, Guthrie  B, Hemming  M.  Changing antibiotic prescribing by educational marketing. Med J Aust. 1988;149(11-12):595-599.
PubMed
Smith  SR.  An evaluation of a computerized exercise in teaching cost consciousness. J Med Educ. 1983;58(2):146-148.
PubMed
Glicken  AD, Merenstein  GB.  Addressing the hidden curriculum: understanding educator professionalism. Med Teach. 2007;29(1):54-57.
PubMed   |  Link to Article

Figures

Tables

Table Graphic Jump LocationTable 1.  Characteristics of Included Studiesa
Table Graphic Jump LocationTable 2.  Overview of Included Articles

References

Reinhardt  UE. Waste vs value in American health care. Economix. September 13, 2013. http://economix.blogs.nytimes.com/2013/09/13/waste-vs-value-in-american-health-care/?_r=0. Accessed January 2014, 2014.
Berwick  DM, Hackbarth  AD.  Eliminating waste in US health care. JAMA. 2012;307(14):1513-1516.
PubMed   |  Link to Article
Blumenthal  D.  Controlling health care expenditures. N Engl J Med. 2001;344(10):766-769.
PubMed   |  Link to Article
Altarum Institute. Health Sector Economic Indicators: Insights From Monthly National Health Spending Data Through July 2015. September 11, 2015. http://altarum.org/sites/default/files/uploaded-related-files/CSHS-Spending-Brief_September_2015.pdf. Accessed September 13, 2015.
Duke  P. The ABIM Foundation’s Choosing Wisely communication module. 2013. http://modules.choosingwisely.org/modules/m_00/. Accessed February 2014.
American College of Physicians. High value care curriculum. https://hvc.acponline.org/curriculum.html. Accessed February 2014.
Owens  DK, Qaseem  A, Chou  R, Shekelle  P; Clinical Guidelines Committee of the American College of Physicians.  High-value, cost-conscious health care: concepts for clinicians to evaluate the benefits, harms, and costs of medical interventions. Ann Intern Med. 2011;154(3):174-180.
PubMed   |  Link to Article
Organisation for Economic Co-operation and Development. Health spending, total % of GDP, 1970-2013. In: OECD Health Data: Health Expenditure and Financing: Health Expenditure Indicators. 2013. https://data.oecd.org/chart/4f22. Accessed February 2015.
Cutler  D. The value equation in health care. Presented at: National Health Care Reform: Policy Options and Imperatives. February 23, 2007; Houston, TX.
Wong  G, Greenhalgh  T, Westhorp  G, Buckingham  J, Pawson  R.  RAMESES publication standards: realist syntheses. BMC Med. 2013;11:21.
PubMed   |  Link to Article
Best Evidence Medical Education Collaboration. BEME data coding sheet. http://download.lww.com/wolterskluwer_vitalstream_com/PermaLink/ACADMED/A/ACADMED_86_11_2011_08_19_REINDERS_202668_SDC2.pdf. Accessed June 2014.
Cochrane Library. Data collection form for RCT and non-RCT. http://www.cochranelibrary.com/cochrane-database-of-systematic-reviews/supplements.html. Accessed June 2014.
Pawson  R, Greenhalgh  T, Harvey  G, Walshe  K. Realist Synthesis: An Introduction. Manchester, England: University of Manchester; 2004.
Sussman  AJ, Chabra  B, Coblyn  J,  et al.  Helping primary care physicians lower pharmaceutical expenses: an academic medical center’s experience. J Clin Outcomes Manag. 2004;11(5):290-295.
Spiegel  CT, Kemp  BA, Newman  MA, Birnbaum  PS, Alter  CL.  Modification of decision-making behavior of third-year medical students. J Med Educ. 1982;57(10 pt 1):769-777.
PubMed
Post  J, Reed  D, Halvorsen  AJ, Huddleston  J, McDonald  F.  Teaching high-value, cost-conscious care: improving residents’ knowledge and attitudes. Am J Med. 2013;126(9):838-842.
PubMed   |  Link to Article
Chandawarkar  RY, Taylor  S, Abrams  P,  et al.  Cost-aware care: critical core competency. Arch Surg. 2007;142(3):222-226.
PubMed   |  Link to Article
Ellemdin  S, Rheeder  P, Soma  P.  Providing clinicians with information on laboratory test costs leads to reduction in hospital expenditure. S Afr Med J. 2011;101(10):746-748.
PubMed
Davidoff  F, Goodspeed  R, Clive  J.  Changing test ordering behavior: a randomized controlled trial comparing probabilistic reasoning with cost-containment education. Med Care. 1989;27(1):45-58.
PubMed   |  Link to Article
Englander  R, Agostinucci  W, Zalneraiti  E, Carraccio  CL.  Teaching residents systems-based practice through a hospital cost-reduction program: a “win-win” situation. Teach Learn Med. 2006;18(2):150-152.
PubMed   |  Link to Article
Stuebing  EA, Miner  TJ.  Surgical vampires and rising health care expenditure: reducing the cost of daily phlebotomy. Arch Surg. 2011;146(5):524-527.
PubMed   |  Link to Article
Sommers  BD, Desai  N, Fiskio  J,  et al.  An educational intervention to improve cost-effective care among medicine housestaff: a randomized controlled trial. Acad Med. 2012;87(6):719-728.
PubMed   |  Link to Article
Hart  J, Salman  H, Bergman  M,  et al.  Do drug costs affect physicians’ prescription decisions? J Intern Med. 1997;241(5):415-420.
PubMed   |  Link to Article
Polinski  JM, Schneeweiss  S, Maclure  M, Marshall  B, Ramsden  S, Dormuth  C.  Time series evaluation of an intervention to increase statin tablet splitting by general practitioners. Clin Ther. 2011;33(2):235-243.
PubMed   |  Link to Article
Willens  HJ, Nelson  K, Hendel  RC.  Appropriate use criteria for stress echocardiography: impact of updated criteria on appropriateness ratings, correlation with pre-authorization guidelines, and effect of temporal trends and an educational initiative on utilization. JACC Cardiovasc Imaging. 2013;6(3):297-309.
PubMed   |  Link to Article
Qureshi  NA, Neyaz  Y, Khoja  T, Magzoub  MA, Haycox  A, Walley  T.  Effectiveness of three interventions on primary care physicians’ medication prescribing in Riyadh City, Saudi Arabia. East Mediterr Health J. 2011;17(2):172-179.
PubMed
Roth  EJ, Plastaras  CT, Mullin  MS, Fillmore  J, Moses  ML.  A simple institutional educational intervention to decrease use of selected expensive medications. Arch Phys Med Rehabil. 2001;82(5):633-636.
PubMed   |  Link to Article
Schroeder  SA, Myers  LP, McPhee  SJ,  et al.  The failure of physician education as a cost containment strategy: report of a prospective controlled trial at a university hospital. JAMA. 1984;252(2):225-230.
PubMed   |  Link to Article
Gitelis  M, Vigneswaran  Y, Ujiki  MB,  et al.  Educating surgeons on intraoperative disposable supply costs during laparoscopic cholecystectomy: a regional health system’s experience. Am J Surg. 2015;209(3):488-492.
PubMed   |  Link to Article
Vigneswaran  Y, Linn  JG, Gitelis  M,  et al.  Educating surgeons may allow for reduced intraoperative costs for inguinal herniorrhaphy. J Am Coll Surg. 2015;220(6):1107-1112.
PubMed   |  Link to Article
Das  AK, Rahman  MS.  Prescribing vitamins at primary health care level: exploration of facts, factors and solution. Bangladesh J Pharmacol. 2010;5(2):92-97.
Link to Article
de Leon  N, Sharpton  S, Burg  C,  et al.  The development and implementation of a bundled quality improvement initiative to reduce inappropriate stress ulcer prophylaxis. ICU Dir. 2013;4(6):322-325.
Link to Article
Whiteside  ME, Lefkowitz  S, Justiniani  FR, Ratzan  K.  Changing prescribing patterns: a program of physician education. Hosp Formul. 1987;22(6):561-563, 566, 568.
PubMed
Cammisa  C, Partridge  G, Ardans  C, Buehrer  K, Chapman  B, Beckman  H.  Engaging physicians in change: results of a safety net quality improvement program to reduce overuse. Am J Med Qual. 2011;26(1):26-33.
PubMed   |  Link to Article
Self  TH, Smith  SL, Boswell  RL, Miller  WA.  Medical education provided by a clinical pharmacist: impact on the use and cost of corticosteroid therapy in chronic obstructive pulmonary disease. Drug Intell Clin Pharm. 1984;18(3):241-244.
PubMed
McKay  RM, Vrbova  L, Fuertes  E,  et al.  Evaluation of the Do Bugs Need Drugs? program in British Columbia: can we curb antibiotic prescribing? Can J Infect Dis Med Microbiol. 2011;22(1):19-24.
PubMed
Barbarello-Andrews  L, Susla  G, Ng  V, St John  D, Lau  C.  Cost-effective medication use in critical care: Capital Health System’s experience in VHA’s MUSIC program. J Clin Outcomes Manag. 2006;13(11):615-622.
Marconi  GP, Nager  AL.  Teaching residents established guidelines and standards of care to strengthen their cost-containment practices. Manag Care. 2010;19(5):46-51.
PubMed
Blackstone  ME, Miller  RS, Hodgson  AJ, Cooper  SS, Blackhurst  DW, Stein  MA.  Lowering hospital charges in the trauma intensive care unit while maintaining quality of care by increasing resident and attending physician awareness. J Trauma. 1995;39(6):1041-1044.
PubMed   |  Link to Article
Elligsen  M, Walker  SA, Pinto  R,  et al.  Audit and feedback to reduce broad-spectrum antibiotic use among intensive care unit patients: a controlled interrupted time series analysis. Infect Control Hosp Epidemiol. 2012;33(4):354-361.
PubMed   |  Link to Article
Dowling  PT, Alfonsi  G, Brown  MI, Culpepper  L.  An education program to reduce unnecessary laboratory tests by residents. Acad Med. 1989;64(7):410-412.
PubMed   |  Link to Article
Fakih  MG, Pena  ME, Shemes  S,  et al.  Effect of establishing guidelines on appropriate urinary catheter placement. Acad Emerg Med. 2010;17(3):337-340.
PubMed   |  Link to Article
Verstappen  WH, van der Weijden  T, Sijbrandij  J,  et al.  Effect of a practice-based strategy on test ordering performance of primary care physicians: a randomized trial. JAMA. 2003;289(18):2407-2412.
PubMed   |  Link to Article
Poppleton  VK, Moynihan  PJ, Hickey  PA.  Clinical practice guidelines: the Boston experience. Prog Pediatr Cardiol. 2003;18(1):75-83.
Link to Article
Sleath  B, Collins  T, Kelly  HW, McCament-Mann  L, Lien  T.  Effect of including both physicians and pharmacists in an asthma drug-use review intervention. Am J Health Syst Pharm. 1997;54(19):2197-2200.
PubMed
Ijo  I, Feyerharm  J.  Pharmacy intervention on antimicrobial management of critically ill patients. Pharm Pract (Granada). 2011;9(2):106-109.
PubMed   |  Link to Article
Bernal-Delgado  E, Galeote-Mayor  M, Pradas-Arnal  F, Peiró-Moreno  S.  Evidence based educational outreach visits: effects on prescriptions of non-steroidal anti-inflammatory drugs. J Epidemiol Community Health. 2002;56(9):653-658.
PubMed   |  Link to Article
Bhatia  RS, Milford  CE, Picard  MH, Weiner  RB.  An educational intervention reduces the rate of inappropriate echocardiograms on an inpatient medical service. JACC Cardiovasc Imaging. 2013;6(5):545-555.
PubMed   |  Link to Article
Hux  JE, Melady  MP, DeBoer  D.  Confidential prescriber feedback and education to improve antibiotic use in primary care: a controlled trial. CMAJ. 1999;161(4):388-392.
PubMed
Larmour  I, Pignataro  S, Barned  KL, Mantas  S, Korman  MG.  A therapeutic equivalence program: evidence-based promotion of more efficient use of medicines. Med J Aust. 2011;194(12):631-634.
PubMed
Phillips  L, Landsberg  KF.  Evaluation of a newsletter in altering physicians’ prescribing patterns. Can J Hosp Pharm. 1986;39(4):102-104, 108.
PubMed
Ziskind  AA, Portelli  J, Rodriguez  S,  et al.  Successful use of education and cost-based feedback strategies to reduce physician utilization of low-osmolality contrast agents in the cardiac catheterization laboratory. Am J Cardiol. 1994;73(16):1219-1221.
PubMed   |  Link to Article
Shane  R, Nishimura  L.  Strategic management of therapeutic advances: experience with colony-stimulating factors. Hosp Pharm. 1994;29(9):824-829, 826-829.
PubMed
Weingarten  SR, Riedinger  MS, Conner  L,  et al.  Practice guidelines and reminders to reduce duration of hospital stay for patients with chest pain: an interventional trial. Ann Intern Med. 1994;120(4):257-263.
PubMed   |  Link to Article
Sucov  A, Bazarian  JJ, deLahunta  EA, Spillane  L.  Test ordering guidelines can alter ordering patterns in an academic emergency department. J Emerg Med. 1999;17(3):391-397.
PubMed   |  Link to Article
Okpara  AU, Maswoswe  JJ, Stewart  K.  Criteria-based antimicrobial IV to oral conversion program. Formulary. 1995;30(6):343-348.
PubMed
Légaré  F, Labrecque  M, LeBlanc  A,  et al.  Training family physicians in shared decision making for the use of antibiotics for acute respiratory infections: a pilot clustered randomized controlled trial. Health Expect. 2011;14(suppl 1):96-110.
PubMed   |  Link to Article
Légaré  F, Labrecque  M, Cauchon  M, Castel  J, Turcotte  S, Grimshaw  J.  Training family physicians in shared decision-making to reduce the overuse of antibiotics in acute respiratory infections: a cluster randomized trial. CMAJ. 2012;184(13):E726-E734.
PubMed   |  Link to Article
Braido  F, Comaschi  M, Valle  I,  et al; ARGA Study Group; EAACI/CME Committee.  Knowledge and health care resource allocation: CME/CPD course guidelines-based efficacy. Eur Ann Allergy Clin Immunol. 2012;44(5):193-199.
PubMed
James  E, Cyriac  J.  Impact of educational interventions on the physicians for early switchover of parenteral drugs to oral therapy. European Journal Hospital Pharmacy. 2014;22:176-178.
Link to Article
Lee  TC, Frenette  C, Jayaraman  D, Green  L, Pilote  L.  Antibiotic self-stewardship: trainee-led structured antibiotic time-outs to improve antimicrobial use. Ann Intern Med. 2014;161(10)(suppl):S53-S58.
PubMed   |  Link to Article
Yang  Z, Zhao  P, Wang  J,  et al.  DRUGS system enhancing adherence of Chinese surgeons to antibiotic use guidelines during perioperative period. PLoS One. 2014;9(8):e102226.
PubMed   |  Link to Article
Fortuna  RJ, Zhang  F, Ross-Degnan  D,  et al.  Reducing the prescribing of heavily marketed medications: a randomized controlled trial. J Gen Intern Med. 2009;24(8):897-903.
PubMed   |  Link to Article
Wein  PJ, Hoffman  RP.  Promoting the cost-effective utilization of cefoxitin with a drug use education program. Hosp Formul. 1987;22(3):299-306.
PubMed
Gregory  KD, Hackmeyer  P, Gold  L, Johnson  AI, Platt  LD.  Using the continuous quality improvement process to safely lower the cesarean section rate. Jt Comm J Qual Improv. 1999;25(12):619-629.
PubMed
Pasquale  TR, Komorny  KM, Letting-Mangira  D, Peshek  S.  A pharmacist-physician antibiotic support team. P&T. 2004;29(1):33-40.
Parrino  TA.  The nonvalue of retrospective peer comparison feedback in containing hospital antibiotic costs. Am J Med. 1989;86(4):442-448.
PubMed   |  Link to Article
McPhee  SJ, Chapman  SA, Myers  LP, Schroeder  SA, Leong  JK.  Lessons for teaching cost containment. J Med Educ. 1984;59(9):722-729.
PubMed
von Ferber  L, Bausch  J, Köster  I, Schubert  I, Ihle  P.  Pharmacotherapeutic circles: results of an 18-month peer-review prescribing-improvement programme for general practitioners. Pharmacoeconomics. 1999;16(3):273-283.
PubMed   |  Link to Article
Miyakis  S, Karamanof  G, Liontos  M, Mountokalakis  TD.  Factors contributing to inappropriate ordering of tests in an academic medical department and the effect of an educational feedback strategy. Postgrad Med J. 2006;82(974):823-829.
PubMed   |  Link to Article
Moriates  C, Soni  K, Lai  A, Ranji  S.  The value in the evidence: teaching residents to “choose wisely”. JAMA Intern Med. 2013;173(4):308-310.
PubMed   |  Link to Article
Attali  M, Barel  Y, Somin  M,  et al.  A cost-effective method for reducing the volume of laboratory tests in a university-associated teaching hospital. Mt Sinai J Med. 2006;73(5):787-794.
PubMed
Bates  DW, Kuperman  GJ, Rittenberg  E,  et al.  A randomized trial of a computer-based intervention to reduce utilization of redundant laboratory tests. Am J Med. 1999;106(2):144-150.
PubMed   |  Link to Article
Thakkar  RN, Kim  D, Knight  AM, Riedel  S, Vaidya  D, Wright  SM.  Impact of an educational intervention on the frequency of daily blood test orders for hospitalized patients. Am J Clin Pathol. 2015;143(3):393-397.
PubMed   |  Link to Article
Zimmerman  DR, Collins  TM, Lipowski  EE, Sainfort  F.  Evaluation of a DUR intervention: a case study of histamine antagonists. Inquiry. 1994;31(1):89-101.
PubMed
Zunker  RJ, Carlson  DL.  Economics of using pharmacists as advisers to physicians in risk-sharing contracts. Am J Health Syst Pharm. 2000;57(8):753-755.
PubMed
Sicotte  C, Pineault  R, Tilquin  C, Contandriopoulos  AP.  The diluting effect of medical work groups on feedback efficacy in changing physician’s practice. J Behav Med. 1996;19(4):367-383.
PubMed   |  Link to Article
Krinsley  JS.  Test-ordering strategy in the intensive care unit. J Intensive Care Med. 2003;18(6):330-339.
PubMed   |  Link to Article
Rudy  DW, Ramsbottom-Lucier  M, Griffith  CH  III, Georgesen  JC, Wilson  JF.  A pilot study assessing the influences of charge data and group process on diagnostic test ordering by residents. Acad Med. 2001;76(6):635-637.
PubMed   |  Link to Article
Rotman  BL, Sullivan  AN, McDonald  TW,  et al.  A randomized controlled trial of a computer-based physician workstation in an outpatient setting: implementation barriers to outcome evaluation. J Am Med Inform Assoc. 1996;3(5):340-348.
PubMed   |  Link to Article
Bornard  L, Dellamonica  J, Hyvernat  H,  et al.  Impact of an assisted reassessment of antibiotic therapies on the quality of prescriptions in an intensive care unit. Med Mal Infect. 2011;41(9):480-485.
PubMed   |  Link to Article
Manheim  LM, Feinglass  J, Hughes  R, Martin  GJ, Conrad  K, Hughes  EF.  Training house officers to be cost conscious: effects of an educational intervention on charges and length of stay. Med Care. 1990;28(1):29-42.
PubMed   |  Link to Article
Niquille  A, Ruggli  M, Buchmann  M, Jordan  D, Bugnon  O.  The nine-year sustained cost-containment impact of Swiss pilot physicians-pharmacists quality circles. Ann Pharmacother. 2010;44(4):650-657.
PubMed   |  Link to Article
Zeleznik  C, Gonnella  JS.  Jefferson Medical College student model utilization review committee. J Med Educ. 1979;54(11):848-851.
PubMed
Gist  D, Llorente  J, Mayer  J.  A clinical algorithm for the management of abnormal mammograms: a community hospital’s experience. West J Med. 1997;166(1):21-28.
PubMed
Lyle  CB  Jr, Bianchi  RF, Harris  JH, Wood  ZL.  Teaching cost containment to house officers at Charlotte Memorial Hospital. J Med Educ. 1979;54(11):856-862.
PubMed
Colbert  CY, Ogden  PE, Lowe  D, Moffitt  MJ.  Students learn systems-based care and facilitate system change as stakeholders in a free clinic experience. Adv Health Sci Educ Theory Pract. 2010;15(4):533-545.
PubMed   |  Link to Article
Mallows  JL.  The effect of a gold coin fine on C-reactive protein test ordering in a tertiary referral emergency department. Med J Aust. 2013;199(11):813-814.
PubMed   |  Link to Article
Collins  TM, Mott  DA, Bigelow  WE, Zimmerman  DR.  A controlled letter intervention to change prescribing behavior: results of a dual-targeted approach. Health Serv Res. 1997;32(4):471-489.
PubMed
Ferris  TG, Shea  T, Jacobson  BC,  et al.  A population-based intervention to improve management of patients on chronic acid suppression: a group-randomized trial. J Clin Outcomes Manag. 2005;12(1):35-43.
Landgren  FT, Harvey  KJ, Mashford  ML, Moulds  RF, Guthrie  B, Hemming  M.  Changing antibiotic prescribing by educational marketing. Med J Aust. 1988;149(11-12):595-599.
PubMed
Smith  SR.  An evaluation of a computerized exercise in teaching cost consciousness. J Med Educ. 1983;58(2):146-148.
PubMed
Glicken  AD, Merenstein  GB.  Addressing the hidden curriculum: understanding educator professionalism. Med Teach. 2007;29(1):54-57.
PubMed   |  Link to Article
CME


You need to register in order to view this quiz.

Multimedia

Supplement.

eAppendix 1. Eligibility Form

eAppendix 2. Abstraction Form

eTable. Detailed Summary of Included Studies

eFigure. Summary of the Review Process

eReferences

Supplemental Content

Some tools below are only available to our subscribers or users with an online account.

8,057 Views
3 Citations
×

Related Content

Customize your page view by dragging & repositioning the boxes below.

See Also...
Articles Related By Topic
Related Collections
PubMed Articles
Jobs