0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Review |

Effects of Computer-Based Clinical Decision Support Systems on Physician Performance and Patient Outcomes:  A Systematic Review FREE

Dereck L. Hunt, MD; R. Brian Haynes, MD, PhD; Steven E. Hanna, MA, PhD; Kristina Smith
[+] Author Affiliations

From the Graduate Program in Health Research Methodology (Dr Hunt), the Health Information Research Unit, Department of Clinical Epidemiology and Biostatistics (Drs Hunt, Haynes, and Hanna and Ms Smith), and the Department of Medicine (Drs Hunt and Haynes), McMaster University Faculty of Health Sciences, Hamilton, Ontario.


JAMA. 1998;280(15):1339-1346. doi:10.1001/jama.280.15.1339.
Text Size: A A A
Published online

Context.— Many computer software developers and vendors claim that their systems can directly improve clinical decisions. As for other health care interventions, such claims should be based on careful trials that assess their effects on clinical performance and, preferably, patient outcomes.

Objective.— To systematically review controlled clinical trials assessing the effects of computer-based clinical decision support systems (CDSSs) on physician performance and patient outcomes.

Data Sources.— We updated earlier reviews covering 1974 to 1992 by searching the MEDLINE, EMBASE, INSPEC, SCISEARCH, and the Cochrane Library bibliographic databases from 1992 to March 1998. Reference lists and conference proceedings were reviewed and evaluators of CDSSs were contacted.

Study Selection.— Studies were included if they involved the use of a CDSS in a clinical setting by a health care practitioner and assessed the effects of the system prospectively with a concurrent control.

Data Extraction.— The validity of each relevant study (scored from 0-10) was evaluated in duplicate. Data on setting, subjects, computer systems, and outcomes were abstracted and a power analysis was done on studies with negative findings.

Data Synthesis.— A total of 68 controlled trials met our criteria, 40 of which were published since 1992. Quality scores ranged from 2 to10, with more recent trials rating higher (mean, 7.7) than earlier studies (mean, 6.4) (P<.001). Effects on physician performance were assessed in 65 studies and 43 found a benefit (66%). These included 9 of 15 studies on drug dosing systems, 1 of 5 studies on diagnostic aids, 14 of 19 preventive care systems, and 19 of 26 studies evaluating CDSSs for other medical care. Six of 14 studies assessing patient outcomes found a benefit. Of the remaining 8 studies, only 3 had a power of greater than 80% to detect a clinically important effect.

Conclusions.— Published studies of CDSSs are increasing rapidly, and their quality is improving. The CDSSs can enhance clinical performance for drug dosing, preventive care, and other aspects of medical care, but not convincingly for diagnosis. The effects of CDSSs on patient outcomes have been insufficiently studied.

COMPUTER SYSTEMS have long been promoted for their potential to improve the quality of health care, including their use to support clinical decisions.1 Access from clinical settings to bibliographic databases, such as MEDLINE and electronic versions of medical journals and textbooks, is becoming commonplace. Most of these resources, however, leave to the practitioner the task of matching individual patient characteristics to information. Computer-based decision support systems (CDSSs) provide additional assistance. They can synthesize and integrate patient-specific information, perform complex evaluations, and present the results to clinicians in a timely fashion.2 Examples include CDSSs designed to recommend appropriate drug doses, to provide immunization reminders, or to diagnose the cause of a patient's chest pain.3 As with any innovative health care intervention, however, these systems should be rigorously evaluated before widespread introduction into clinical practice. The various stages in this assessment process have been described by Wyatt and Spiegelhalter,4 including using controlled trials to test the effects on clinical behavior and patient outcomes.

Our group previously reviewed controlled trials of computer-aided quality assurance5 and CDSSs published up to 1992.3 This field is rapidly evolving because of technological advances and increasing access to computer systems in clinical practice. Thus, we updated the previous review to provide a cumulative picture of the state of documentation of the effects CDSSs have on clinician performance and patient outcomes based on the strongest trials in health care settings to date.

Study Identification and Selection

A CDSS was defined as any software designed to directly aid in clinical decision making in which characteristics of individual patients are matched to a computerized knowledge base for the purpose of generating patient-specific assessments or recommendations that are then presented to clinicians for consideration. Study identification for the previous review3 involved searching the US National Library of Medicine MEDLINE electronic bibliographic database, Excerpta Medica (EMBASE), the International Information Service for the Physics and Engineering Communities (INSPEC), and the Science Citation Index (SCISEARCH), from 1974 to February 1992, for studies in any language. Conference proceedings and reference lists of relevant articles were also reviewed and authors were contacted. We used a similar strategy for the update, searching for any additional original articles on CDSSs from February 1992 to March 1998. The MEDLINE search strategy included the Medical Subject Heading (MeSH) terms computer-assisted decision making, artificial intelligence, computer-assisted diagnosis, computer-assisted therapy, and hospital information systems. The complete MEDLINE, EMBASE, and INSPEC search strategies are available on request. We searched SCISEARCH for references to the primary studies from the previous reviews and also searched the Cochrane Library6 (search strategy available on request) for potentially relevant citations. Reference lists from all relevant articles were examined and authors of relevant studies were contacted and asked if they were aware of any additional published or unpublished studies that we had not identified.

All citations, and index terms and abstracts if available, were reviewed and rated as "potentially relevant" or "not relevant." To assess interrater reliability, a random subset of 120 citations from the entire citation list was reviewed in duplicate. The raw agreement was 83% and the level of agreement beyond chance was 58% (κ=58%; 95% confidence interval [CI], 42%-75%).

Full-text articles were retrieved for all titles considered to be "potentially relevant" by either reviewer. All of these articles were reviewed independently in duplicate, and studies were included in the review if (1) the study participants were health professionals in clinical practice or postgraduate training; (2) the intervention was a CDSS evaluated in a clinical setting; (3) the outcomes that were assessed included clinician performance (a measure of the process of care) or patient outcomes (including any aspect of patient well-being); and (4) the study prospectively collected data, with a contemporaneous control group so that patient care with a CDSS was compared with care without one. The agreement between reviewers on the eligibility of studies for the systematic review was 98% and the level of agreement beyond chance was 86% (κ=86%; 95% CI, 76%-97%). All disagreements were resolved by consensus.

Study Evaluation

At least 2 authors assessed all selected studies independently for methodological quality. A 10-point rating scale was used and disagreements were resolved by consensus. The scale assessed 5 potential sources of bias, including the method of allocation to study groups (random vs quasi-random vs selected concurrent controls), the unit of allocation (clinic vs physician vs patient), the presence of baseline differences between groups that were potentially linked to the study outcome (no baseline differences or appropriate statistical adjustments made for differences vs baseline differences present and no statistical adjustment made vs unable to assess), the type of outcome measure (objective outcome or subjective outcome with blinded assessment vs subjective outcome with no blinding but clearly defined and explicit criteria for each outcome vs subjective outcome with no blinding of assessors and no explicit criteria for each outcome), and completeness of follow-up (>90% vs 80%-90% vs <80%).

The unit of allocation was included because of the possibility of contamination in trials in which interventions are applied to clinicians and end points are measured for patients. For example, the lowest score for unit of allocation was given when individual patients were allocated to intervention and control groups because a clinician may treat some patients with and others without the aid of the CDSS. Knowledge gained by the clinician from the CDSS may then be applied to control patients, leading to an underestimate of the system's effect. A similar situation can occur when individual clinicians within a team or group are the units of allocation: the presence of a CDSS or of colleagues who are using a CDSS in the clinical setting may influence the treatment given by clinicians allocated to the comparison group. While there are situations in which randomizing patients or physicians may be preferable to randomizing clinics,7 including cases where clinics differ in many respects and access to a large number of practices is limited, we chose to rate articles using the same scoring system as in the initial review3 for the sake of consistency.

All studies were initially graded for adequacy of follow-up of patients. In some trials, however, only a subset of the randomized patients were assessed for outcomes, for example, only those attending an appointment during the study period as a subset of all those randomized from a patient roster. This was especially common in trials assessing the role of CDSSs in preventive care. For these studies, the completeness of follow-up criterion was inapplicable and the remaining score was prorated so that the maximum possible score remained as 10 points.

A total of 40 new studies were assessed for scientific merit. The level of chance-corrected agreement between reviewers was high with a quadratic-weighted κ statistic of 82% (95% CI, 67%-97%). Disagreements were resolved by discussion.

Data Extraction and Analysis

One of the authors extracted information concerning patients, setting, intervention, and outcomes for each of the studies and this was verified by a second author. Subsequently, all studies that evaluated the effects of using a CDSS on patient outcomes and reported no statistically significant improvement were analyzed to determine the power of each to detect a moderate and clinically important difference in outcomes between the groups.8 An example of a moderate and clinically significant change would be a 25% absolute increase in the proportion of patients at a general medicine clinic who had adequate blood pressure control. The approach described by Cohen8 was used and in each case an α probability of .05 along with a medium effect size were used for the calculations.

The studies were then grouped into 4 major topic areas: drug dosing, diagnosis, preventive care, and studies involving other aspects of medical care. Because the studies differed in many respects, including clinical problems, clinicians, patients, method of intervention, follow-up, and measures of outcomes, no common measure of effect was considered to be justified for meta-analysis of the trial results. Thus, measures of the process of care and clinical outcomes were simply characterized for each study according to whether a statistically significant effect was reported.

In addition to the 28 studies previously reviewed,939 we identified 40 new studies that met our selection criteria and included sufficient results to be able to determine whether using the CDSS had affected health care practitioner behavior or patient outcomes.4079 Several additional trials met our inclusion criteria but included insufficient results to determine the effect of using the CDSS.8084

We also identified 9 review articles that discussed decision support systems. Four of these were narrative, nonsystematic reviews, including 2 that discussed the role of decision support in health care in general1,85 and 2 that focused on specific aspects of decision support, including diagnostic aid systems86 and cancer prevention systems.87 The remaining 5 articles were systematic reviews. These included 2 prior publications by our group3,5 and articles by Austin et al,2 Balas et al,88 and Shea et al.89 Austin et al completed a meta-analysis in 1994 of trials that assessed the effects of computer-based reminder systems on cervical cancer screening and tetanus immunization. This demonstrated a significant beneficial impact for both of these maneuvers. Balas and colleagues88 systematically reviewed the evidence from randomized controlled trials assessing numerous clinical information systems. They also reviewed the area of patient education. Their review demonstrated that reminder systems and drug dosing systems can improve health care practitioner performance. The meta-analysis by Shea and colleagues89 evaluated computer-based reminder systems for preventive care. They found benefits for vaccinations, breast cancer screening, colorectal cancer screening, and cardiovascular risk reduction but not for cervical cancer screening.

Linear regression of the validity scores of all trials in our review against their year of publication showed that study quality increased significantly over time (β=.14; 95% CI, 0.07-0.20). For the 28 studies that were identified in the original review, the scores for scientific merit ranged from 2 to 9 (mean, 6.4), with 7 (25%) of the studies having high scores between 8 and 10. The validity scores for the 40 new studies ranged between 4 to 10 (mean, 7.7), with 21 of the studies (53%) scoring 8 to 10. The majority of the trials were randomized. Only 9 studies (13%) used quasi-random allocation14,16,50,63 or selected concurrent controls.24,39,40,61,69

Of the total of 68 included trials, 15 (22%) tested CDSSs designed to assist with dosing estimations for potentially toxic drugs (Table 1), 5 (7.0%) evaluated diagnostic aids, 19 (28%) assessed systems for preventive care (Table 2), and 29 (43%) tested programs for other aspects of medical care (Table 3). Almost all of these studies evaluated effects on clinician performance, but only 14 assessed patient outcomes. Of the 65 studies that evaluated the effect of using a CDSS on clinician behavior, 43 (66%) found at least some benefit. These included 9 (60%) of 15 studies on drug dosing systems, 1 (20%) of 5 studies evaluating diagnostic aids, 14 (74%) of 19 preventive care systems, and 19 (73%) of 26 studies assessing CDSSs for other medical care. Only 6 (43%) of the 14 studies that evaluated the effects on patient outcomes documented a benefit. However, 5 (62%) of the 8 trials that found no benefit for patient outcomes had a power of less than 80% to detect a moderate and clinically important improvement.

Table Graphic Jump LocationTable 1.—Studies of Computerized Drug Dosing*
Table Graphic Jump LocationTable 2.— Studies of Computerized Reminders for Preventive Care*
Table Graphic Jump LocationTable 3.—Studies of Other Aspects of Medical Care*
Drug Dosing Studies

Fifteen studies assessed systems designed to assist with drug dosing (Table 1), with 11 of these coming from the update. Eight studies9,10,41,4346,50 addressed the dosing of intravenous medications and 6 found improvements with the use of a CDSS. The systems that were tested were designed to assist with achieving or maintaining therapeutic theophylline or lidocaine hydrochloride levels or achieving improved anticoagulation control with heparin. Four of these trials also evaluated patient outcomes and only 1 found a significant benefit compared with usual clinical practice.46

The remaining 7 studies11,12,40,42,4749 evaluated the role of CDSSs in warfarin dosing using a number of different end points. The time to reach a therapeutic international normalized rate (INR) or prothrombin time (PT) when initiating warfarin therapy was assessed by 3 trials. The findings were inconsistent. One study found no effect.48 In another study, the CDSS group reached therapeutic levels faster,11 while the reverse occurred in another trial.40 Using a CDSS decreased the time required to reach a stable INR or PT (in contrast to the first occurrence of a therapeutic INR or PT) in one trial,48 but had no effect in another.42 For maintenance therapy for outpatients, 1 study found that a CDSS was able to keep the INR within the therapeutic range more consistently than if a CDSS was not used,48 but 3 other studies found that using a CDSS did not improve the anticoagulation control.12,47,49 However, 1 of these 3 studies49 compared warfarin sodium dosing by a trainee physician with dosing by a nurse practitioner using a CDSS. Anticoagulation was equivalent in the 2 groups.

Diagnosis

Five studies1317 assessed the role of CDSSs in diagnosis, all from the initial review. These evaluated a variety of systems designed to assist in the management of pediatric patients or patients presenting with chest or abdominal pain. In addition, Chase and colleagues13 tested a system that identified patients at high risk of respiratory tract complications postoperatively so that physiotherapy could be selectively provided. This was the only study to find a benefit with the CDSS. Positive effects were noted for both timely referral of patients for physiotherapy and reduced risk of postoperative complications.

Prevention

With the update, the number of studies of CDSSs providing preventive care reminders increased from 6 to 19 (Table 2). All of the studies evaluated clinician performance and 14 (74%) found a benefit for at least one of the processes of care measured. The majority of the systems in the original review reminded clinicians about a single test, such as blood pressure assessment,18,21 vaccinations,19,22 and Papanicolaou tests.20 In contrast, most of the systems tested in the new studies reminded physicians about several preventive care tests, including vaccinations and cancer screening. Also, the only study to look at prevention reminders for inpatients receiving acute medical care59 found no increase in the implementation of 22 preventive procedures, even though the majority of participating physicians indicated that they felt that acute hospitalizations provided an appropriate opportunity for reviewing and implementing preventive care.

Other Medical Care

Twenty-nine studies, including 16 new trials, assessed the role of CDSSs in other medical care (Table 3). Some of the systems that were tested were disease specific, focusing on hypertension, diabetes, or management of the acquired immunodeficiency syndrome, while others addressed numerous general medical problems (Table 3). Several systems were designed to improve drug prescribing and test ordering and others to decrease the number of admissions to hospital, to hasten hospital discharge, or to improve the functioning of anticoagulation clinics. Systems designed specifically to assist nurses have also been evaluated.

Almost all of these studies assessed the effects of using a CDSS on health care practitioner performance, and 19 (73%) of 26 studies found a benefit. The process of care for hypertension improved in one trial,24 but not in another.27 For diabetes care, compliance with recommendations improved in 2 of 4 studies.28,6870 The systems that provided patient-specific information about laboratory test results37,64 found improvements. Compliance with recommendations for general medical problems also improved in the 5 studies2931,71,78 that addressed this. Safran and colleagues76 found that health care practitioners caring for patients with acquired immunodeficiency syndrome completed tasks more quickly when they received computer-based support. Fihn and colleagues66 noted that a CDSS designed to recommend follow-up periods for outpatients receiving warfarin anticoagulation at a thromboembolism clinic led to significantly longer periods between appointments, without any differences in the level of anticoagulation control between the 2 groups. No improvements were noted in the trials that evaluated drug prescribing or hospital admission practices. Using a CDSS did, however, lead to slightly earlier discharge from the hospital.77 Recently, Dexter and colleagues65 demonstrated that computer-generated reminders are effective at increasing the rate of advance directive discussions and form completion.

A smaller number of trials evaluated patient outcomes as end points; 4 (57%) of 7 found a benefit. Of the studies that assessed blood pressure, no improvements were noted for this end point.25,27,3436 In contrast, beneficial changes in weight and patient quality of life were found in 2 studies.3436,74 Also, fewer people who had been cared for with the assistance of a CDSS for the management of general medical problems subsequently required hospitalization or a chest or respiratory tract assessment.

This update documents considerable progress in the quantity and quality of studies of CDSSs. The number of high-quality, controlled trials has more than doubled since the previous review of studies up to 1992, and stronger conclusions can now be reached. For example, with 19 studies assessing preventive care systems in comparison with the limited set of 6 studies that were previously available and with 74% reporting positive findings, ambulatory care services and clinics should consider opportunities to acquire preventive care reminder systems—a conclusion reached in another recent review.88

Information relating to dosing systems has also increased since the initial review. Six studies, instead of 2, now indicate that potentially toxic, intravenously administered medications can be titrated more effectively using a CDSS than without one if the pharmacokinetics are well understood. Only 2 published studies found no improvement with a CDSS. While larger, confirmatory studies may be warranted (as concluded after the initial review), the consistent results suggest that it would now be reasonable to consider using a CDSS for this purpose in certain circumstances. The same cannot be said for warfarin dosing systems. Despite the fact that 7 studies have now addressed this topic, in comparison with the previous 2, the findings have remained inconsistent.

Perhaps not surprisingly, given the disappointing findings of earlier trials, there have not been any new controlled trials of diagnostic aid systems. Thus, our conclusions regarding these systems remain unchanged. For the time being they should only be deployed in settings in which they are being properly evaluated.

Also, given the rapid rate of development of CDSSs, the inconsistencies in results within classes of CDSSs, and the limited range of clinical settings in which CDSSs have been tested to date, it is important for health care centers to include some form of in-house evaluation when incorporating CDSSs. This is especially important if the system has not been previously tested, if the clinical setting differs substantially from that of prior testing sites, or if specially trained staff were included in the previous trials. The study of Overhage and colleagues59 is enlightening in this respect: despite the success of reminder systems for preventive care in ambulatory settings and the prior belief of clinicians that preventive care reminders would be welcomed for seriously ill, hospitalized patients, no benefits were detected when the reminders were implemented. Thus, as a minimum, adherence rates should be assessed after the implementation of any reminder system and compared with the preimplementation rates. Similarly, systems designed to alter laboratory testing behavior or adherence to disease management recommendations should have postimplementation evaluations to assess their impact.

To improve the chance that a decision support system will have a positive influence, the findings from 2 additional studies are worth considering. Litzelman et al90 demonstrated that requiring physicians to acknowledge reminders improved their adherence to recommendations. Recently, Lobach91 found that providing personalized feedback to physicians was an effective means of improving adherence to recommendations after the implementation of a CDSS.

Few studies of CDSSs to date have assessed patient outcomes and only a small proportion of these have found benefits. However, many of these studies have been too small to detect clinically important improvements in patient outcomes. It could be argued that this is mainly a matter of appropriate timing. Without more substantive effects on process, looking for clinical and economic benefits is premature. The CDSSs are improving so larger trials will be warranted soon. Given the expense and difficulties of doing such trials, this position has merit as long as the final stages of validation are not neglected—once more compelling process effects are shown. Unfortunately, no regulatory or other guarantee exists that such trials will be done, and market forces may drive deployment of such systems before adequate evaluation. Nevertheless, the increasing number and quality of CDSS trials in the past few years and the rapid assimilation of technological information into clinical settings bode well for the future of improving the effectiveness and efficiency of clinical care.

Clayton PD, Hripcsak G. Decision support in health care.  Int J Biomed Comput.1995;39:59-66.
Austin SM, Balas EA, Mitchell JA, Ewigman BG. Effect of physician reminders on preventive care: meta-analysis of randomized clinical trials.  Proc Annu Symp Comput Appl Med Care.1994;:121-124.
Johnston ME, Langton KB, Haynes RB, Mathieu A. Effects of computer-based clinical decision support systems on clinician performance and patient outcome: a critical appraisal of research.  Ann Intern Med.1994;120:135-142.
Wyatt JC, Spiegelhalter DJ. Evaluating medical expert systems: what to test and how?  Med Inf (Lond).1990;15:205-217.
Haynes RB, Walker CJ. Computer-aided quality assurance: a critical appraisal.  Arch Intern Med.1987;147:1297-1301.
The Cochrane Library.  Update Software . Issue 2. Oxford, England: Cochrane Library; 1998.
McDonald CJ, Hui SL, Zhou XH. Analyzing computer-based decision support systems.  Ann Intern Med.1994;121:469.
Cohen J. Statistical Power Analysis for the Behavioural Sciences . Hillsdale, NJ: Lawrence Erlbaum Publisher; 1988.
Gonzalez E, Vanderheyden BA, Ornato JP, Comstock TG. Computer-assisted optimization of aminophylline therapy in the emergency department.  Am J Emerg Med.1989;7:395-401.
Rodman JH, Jeliffe RW, Kolb E.  et al.  Clinical studies with computer-assisted initial lidocaine therapy.  Arch Intern Med.1984;144:703-709.
White RH, Hong R, Venook AP.  et al.  Initiation of warfarin therapy: comparison of physician dosing with computer-assisted dosing.  J Gen Intern Med.1987;2:141-148.
White RH, Mungall D. Outpatient management of warfarin therapy: comparison of computer-predicted dosage adjustment to skilled professional care.  Ther Drug Monit.1991;13:46-50.
Chase CR, Vacek PM, Shinozaki T, Giard AM, Ashikaga T. Medical information management: improving the transfer of research results to presurgical evaluation.  Med Care.1983;21:410-424.
Pozen MW, D'Agostino RB, Selker HP, Sytkowski PA, Hood WB. A predictive instrument to improve coronary-care-unit admission practices in acute ischemic heart disease: a prospective multicenter clinical trial.  N Engl J Med.1984;310:1273-1278.
Wellwood J, Johannessen S, Spiegelhalter DJ. How does computer-aided diagnosis improve the management of acute abdominal pain?  Ann R Coll Surg Engl.1992;74:40-46.
Wexler JR, Swender PT, Tunnessen WW, Oski FA. Impact of a system of computer-assisted diagnosis: initial evaluation of the hospitalized patient.  AJDC.1975;129:203-205.
Wyatt JR. Lessons learnt from the field trial of ACORN, an expert system to advise on chest pain. In: Proceedings of the Sixth World Conference on Medical Informatics; Singapore; 1989:111-115.
Barnett GO, Winickoff RN, Morgan MM, Zielstorff RD. A computer-based monitoring system for follow-up of elevated blood pressure.  Med Care.1983;21:400-409.
McDowell I, Newell C, Rosser W. Comparison of three methods of recalling patients for influenza vaccination.  CMAJ.1986;135:991-997.
McDowell I, Newell C, Rosser W. Computerized reminders to encourage cervical screening in family practice.  J Fam Pract.1989;28:420-424.
McDowell I, Newell C, Rosser W. A randomized trial of computerized reminders for blood pressure screening in primary care.  Med Care.1989;27:297-305.
Rosser WW, Hutchison BG, McDowell I, Newell C. Use of reminders to increase compliance with tetanus booster vaccination.  CMAJ.1992;146:911-917.
Tierney WM, Hui SL, McDonald CJ. Delayed feedback of physician performance versus immediate reminders to perform preventive care: effects on physician compliance.  Med Care.1986;24:659-666.
Brownbridge G, Evans A, Fitter M, Platts M. An interactive computerized protocol for the management of hypertension: effects on the general practitioner's clinical behaviour.  J R Coll Gen Pract.1986;36:198-202.
Coe FL, Norton E, Oparil S, Tatar A, Pullman TN. Treatment of hypertension by computer and physician—a prospective controlled study.  J Chronic Dis.1977;30:81-92.
Keller LS, McDermott S, Alt-White A. Effects of computerized nurse care planning on selected health care effectiveness measures.  Proc Annu Symp Comput Appl Med Care.1991;:38-42.
McAlister NH, Covvy HD, Tong C, Lee A, Wigle ED. Randomized controlled trial of computer assisted management of hypertension in primary care.  Br Med J (Clin Res Ed).1986;293:670-674.
McDonald CJ. Use of a computer to detect and respond to clinical events: its effect on clinician behavior.  Ann Intern Med.1976;84:162-167.
McDonald CJ. Protocol-based computer reminders, the quality of care and the non-perfectability of man.  N Engl J Med.1976;295:1351-1355.
McDonald CJ, Wilson GA, McCabe GP. Physician response to computer reminders.  JAMA.1980;244:1579-1587.
McDonald CJ, Hui SL, Smith DM.  et al.  Reminders to physicians from an introspective computer medical record.  Ann Intern Med.1984;100:130-138.
McDonald CJ, Hui SL, Tierney WM. Effects of computer reminders for influenza vaccination on morbidity during influenza epidemics.  MD Comput.1992;9:304-312.
Petrucci K, Petrucci P, Canfield K.  et al.  Evaluation of UNIS: urological nursing information systems.  Proc Annu Symp Comput Appl Med Care.1991;:43-47.
Rogers JL, Haring OM. The impact of computerized medical record summary system on incidence and length of hospitalization.  Med Care.1979;17:618-630.
Rogers JL, Haring OM, Wortman PM. Medical information systems: assessing impact in the areas of hypertension, obesity and renal disease.  Med Care.1982;20:63-74.
Rogers JL, Haring OM, Goetz JP. Changes in patient attitudes following the implemenation of a medical information system.  Q Rev Bull.1984;10:65-74.
Tierney WM, McDonald CJ, Hui SL, Martin DK. Computer predictions of abnormal test results: effects on outpatient testing.  JAMA.1988;259:1194-1198.
White KS, Lindsay A, Pryor TA, Brown WF, Walsh K. Application of a computerized medical decision-making process to the problem of digoxin intoxication.  J Am Coll Cardiol.1984;4:571-576.
Young DW. Improving the consistency with which investigations are requested.  Med Inf (Lond).1981;6:13-17.
Abbrecht PH, O'Leary TJ, Behrendt DM. Evaluation of a computer-assisted method for individualized anticoagulation: retrospective and prospective studies with a pharmacodynamic model.  Clin Pharmacol Ther.1982;32:129-136.
Begg EJ, Atkinson HC, Jeffery GM, Taylor NW. Individualised aminoglycoside dosage based on pharmacokinetic analysis is superior to dosage based on physician intuition at achieving target plasma drug concentrations.  Br J Clin Pharmacol.1989;28:137-141.
Carter BL, Taylor JW, Becker A. Evaluation of three dosage-prediction methods for initial in-hospital stabilization of warfarin therapy.  Clin Pharm.1987;6:37-45.
Casner PR, Reilly R, Ho H. A randomized controlled trial of computerized pharmacokinetic theophylline dosing versus empiric physician dosing.  Clin Pharmacol Ther.1993;53:684-690.
Hickling K, Begg EJ, Moore ML. A prospective randomised trial comparing individualised pharmacokinetic dosage prediction for amnioglycosides with prediction based on estimated creatinine clearance in critically ill patients.  Intensive Care Med.1989;15:233-237.
Hurley SF, Dziukas LJ, McNeil JJ, Brignell MJ. A randomized controlled clinical trial of pharmacokinetic theophylline dosing.  Am Rev Respir Dis.1986;134:1219-1224.
Mungall D, Anbe D, Forrester PL.  et al.  Clinic trials and therapeutics: a prospective randomized comparison of the accuracy of computer-assisted versus GUSTO nomogram—directed heparin therapy.  Clin Pharmacol Ther.1994;55:591-596.
Poller L, Wright D, Rowlands M. Prospective comparative study of computer programs used for management of warfarin.  J Clin Pathol.1993;46:299-303.
Vadher B, Patterson DLH, Leaning M. Evaluation of a decision support system for initiation and control of oral anticoagulation in a randomised trial.  BMJ.1997;314:1252-1256.
Vadher BD, Patterson DLH, Leaning M. Comparison of oral anticoagulant control by a nurse-practitioner using a computer decision-support system with that by clinicians.  Clin Lab Haematol.1997;19:203-207.
Verner D, Seligmann H, Platt S. Computer assisted design of a theophylline dosing regimen in acute bronchospasm: serum concentrations and clinical outcome.  Eur J Clin Pharmacol.1992;43:29-33.
Burack RC, Gimotty PA, George J.  et al.  Promoting screening mammography in inner-city settings: a randomized controlled trial of computerized reminders as a component of a program to facilitate mammography.  Med Care.1994;32:609-624.
Burack RC, Gimotty PA, George J.  et al.  The effect of patient and physician reminders on use of screening mammography in a health maintenance organization: results of a randomized controlled trial.  Cancer.1996;78:1708-1721.
Burack RC, Gimotty PA. Promoting screening mammography in inner-city settings: the sustained effectiveness of computerized reminders in a randomized controlled trial.  Med Care.1997;35:921-931.
Chambers CV, Balaban DJ, Carlson BL, Grasberger DM. The effect of microcomputer-generated reminders on influenza vaccination rates in a university-based family practice center.  J Am Board Fam Pract.1991;4:19-26.
Frame PS, Zimmer JG, Werth PL, Hall WJ, Eberly SW. Computer based vs manual health maintenence tracking: a controlled trial.  Arch Fam Med.1994;3:581-588.
McPhee SJ, Adair Bird J, Jenkins CN, Fordham D. Promoting cancer screening: a randomized, controlled trial of three interventions.  Arch Intern Med.1989;149:1866-1872.
McPhee SJ, Adair Bird J, Fordham D, Rodnick JE, Osborn EH. Promoting cancer prevention activities by primary care physicians: results of a randomized controlled trial.  JAMA.1991;266:538-544.
Ornstein SM, Garr DR, Jenkins RG, Rust PF, Arnon A. Computer-generated physician and patient reminders: tools to improve population adherence to selected preventive services.  J Fam Pract.1991;32:82-90.
Overhage JM, Tierney WM, McDonald CJ. Computer reminders to implement preventive care guidelines for hospitalized patients.  Arch Intern Med.1996;156:1551-1556.
Rosser WW, McDowell I, Newell C. Use of reminders for preventive procedures in family medicine.  CMAJ.1991;145:807-812.
Tape TG, Campbell JR. Computerized medical records and preventive health care: success depends on many factors.  Am J Med.1993;94:619-625.
Turner RC, Peden JG, O'Brien K. Patient-carried card prompts vs computer-generated prompts to remind private practice physicians to perform health maintenance measures.  Arch Intern Med.1994;154:1957-1960.
Turner BJ, Day SC, Borenstein B. A controlled trial to improve delivery of preventive care: physician or patient reminders.  J Gen Intern Med.1989;4:403-409.
Bates DW, Kuperman GJ, Rittenberg E.  et al.  Reminders for redundant tests: results of a randomized controlled trial [abstract].  Proc Annu Symp Comput Appl Med Care.1995:935.
Dexter PR, Wolinsky FD, Gramelspacher GP.  et al.  Effectiveness of computer-generated reminders for increasing discussions about advance directives and completion of advance directive forms: a randomized, controlled trial.  Ann Intern Med.1998;128:102-110.
Fihn SD, McDonell MB, Vermes D.  et al.  A computerized intervention to improve timing of outpatient follow-up: a multicenter randomized trial in patients treated with warfarin.  J Gen Intern Med.1994;9:131-139.
Hales JW, Gardner RM, Jacobson JT. Factors impacting the success of computerized preadmission screening.  Proc Annu Symp Comput Appl Med Care.1995;:728-732.
Lobach DF, Hammond WE. Development and evaluation of a computer-assisted management protocol (CAMP): improved compliance with care guidelines for diabetes mellitus.  Proc Annu Symp Comput Appl Med Care.1994;:787-791.
Mazzuca SA, Vinicor F, Einterz RM, Norton JA, Kalasinski LA. Effects of the clinical environment on physicians' response to postgraduate medical education.  Am Educ Res J.1990;27:473-488.
Nilasena DS, Lincoln MJ. A computer-generated reminder system improves physician compliance with diabetes preventive care guidelines.  Proc Annu Symp Comput Appl Med Care.1995;:640-645.
Overhage JM, Tierney WM, Zhou XA, McDonald CJ. A randomized trial of "corollary orders" to prevent errors of omission.  J Am Med Inform Assoc.1997;4:364-375.
Rossi RA, Every NR. A computerized intervention to decrease the use of calcium channel blockers in hypertension.  J Gen Intern Med.1997;12:672-678.
Rotman BL, Sullivan AN, McDonald TW.  et al.  A randomized controlled trial of a computer-based physician workstation in an outpatient setting: implementation barriers to outcome evaluation.  J Am Med Inform Assoc.1996;3:340-348.
Rubenstein LV, McCoy JM, Cope DW. Improving patient quality of life with feedback to physicians about functional status.  J Gen Intern Med.1995;10:607-614.
Ryff-de-Leche A, Engler H, Nutzi E, Berger M, Berger W. Clinical application of two computerized diabetes management systems: comparison with the log-book method.  Diabetes Res.1992;19:97-105.
Safran C, Rind DM, Davis R.  et al.  Guidelines for management of HIV infection with computer-based patient's record.  Lancet.1995;346:341-346.
Shea S, Sideli RV, DuMouchel W.  et al.  Computer-generated informational messages directed to physicians: effect on length of hospital stay.  J Am Med Inform Assoc.1995;2:58-64.
Thomas JC, Moore A, Qualls PE. The effect on cost of medical care for patients treated with an automated clinical audit system.  J Med Syst.1983;7:307-313.
Tierney WM, Miller ME, Overhage M, McDonald CJ. Physician inpatient order writing on microcomputer workstations: effects on resource utilization.  JAMA.1993;269:379-383.
Sheiner LB, Halkin H, Peck C, Rosenberg B, Melmon KL. Improved computer-assisted digoxin therapy: a method using feedback of measured serum digoxin concentrations.  Ann Intern Med.1975;82:619-627.
Fitzmaurice DA, Hobbs FDR, Murray ET, Bradley CP, Holder R. Evaluation of computerized decision support for oral anticoagulation management based in primary care.  Br J Gen Pract.1996;46:533-535.
Pringle M, Robins S, Brown G. Computer assisted screening: effects on the patient and his consultation.  BMJ.1990;290:1709-1712.
Hobbs FD, Delaney BC, Carson A, Kenkre JE. A prospective controlled trial of computerized decision support for lipid management in primary care.  Fam Pract.1996;13:133-137.
Destache CJ, Meyer SK, Bittner MJ, Hermann KG. Impact of a clinical pharmacokinetic service on patients treated with aminoglycosides: a cost-benefit analysis.  Ther Drug Monit.1990;12:419-426.
Dasta JF, Greer ML, Speedie SM. Computers in healthcare: overview and bibliography.  Ann Pharmacother.1992;26:109-117.
Miller RA. Medical diagnostic decision support systems—past, present, and future.  J Am Med Inform Assoc.1994;1:8-27.
McPhee SJ, Detmer WM. Office-based interventions to improve delivery of cancer prevention services by primary care physicians.  Cancer.1993;72:1100-1112.
Balas EA, Austin SM, Mitchell JA.  et al.  The clinical value of computerized information services: a review of 98 randomized clinical trials.  Arch Fam Med.1996;5:271-278.
Shea S, DuMouchel W, Bahamonde L. A meta-analysis of 16 randomized controlled trials to evaluate computer-based reminder systems for preventive care in the ambulatory setting.  J Am Med Inform Assoc.1996;3:399-409.
Litzelman DK, Dittus RS, Miller ME, Tierney WM. Requiring physicians to respond to computerized reminders improves their compliance with preventive care protocols.  J Gen Intern Med.1993;8:311-317.
Lobach DF. Electronically distributed, computer-generated, individualized feedback enhances the use of a computerized practice guideline.  Proc AMIA Annu Fall Symp.1996;:493-497.

Figures

Tables

Table Graphic Jump LocationTable 1.—Studies of Computerized Drug Dosing*
Table Graphic Jump LocationTable 2.— Studies of Computerized Reminders for Preventive Care*
Table Graphic Jump LocationTable 3.—Studies of Other Aspects of Medical Care*

References

Clayton PD, Hripcsak G. Decision support in health care.  Int J Biomed Comput.1995;39:59-66.
Austin SM, Balas EA, Mitchell JA, Ewigman BG. Effect of physician reminders on preventive care: meta-analysis of randomized clinical trials.  Proc Annu Symp Comput Appl Med Care.1994;:121-124.
Johnston ME, Langton KB, Haynes RB, Mathieu A. Effects of computer-based clinical decision support systems on clinician performance and patient outcome: a critical appraisal of research.  Ann Intern Med.1994;120:135-142.
Wyatt JC, Spiegelhalter DJ. Evaluating medical expert systems: what to test and how?  Med Inf (Lond).1990;15:205-217.
Haynes RB, Walker CJ. Computer-aided quality assurance: a critical appraisal.  Arch Intern Med.1987;147:1297-1301.
The Cochrane Library.  Update Software . Issue 2. Oxford, England: Cochrane Library; 1998.
McDonald CJ, Hui SL, Zhou XH. Analyzing computer-based decision support systems.  Ann Intern Med.1994;121:469.
Cohen J. Statistical Power Analysis for the Behavioural Sciences . Hillsdale, NJ: Lawrence Erlbaum Publisher; 1988.
Gonzalez E, Vanderheyden BA, Ornato JP, Comstock TG. Computer-assisted optimization of aminophylline therapy in the emergency department.  Am J Emerg Med.1989;7:395-401.
Rodman JH, Jeliffe RW, Kolb E.  et al.  Clinical studies with computer-assisted initial lidocaine therapy.  Arch Intern Med.1984;144:703-709.
White RH, Hong R, Venook AP.  et al.  Initiation of warfarin therapy: comparison of physician dosing with computer-assisted dosing.  J Gen Intern Med.1987;2:141-148.
White RH, Mungall D. Outpatient management of warfarin therapy: comparison of computer-predicted dosage adjustment to skilled professional care.  Ther Drug Monit.1991;13:46-50.
Chase CR, Vacek PM, Shinozaki T, Giard AM, Ashikaga T. Medical information management: improving the transfer of research results to presurgical evaluation.  Med Care.1983;21:410-424.
Pozen MW, D'Agostino RB, Selker HP, Sytkowski PA, Hood WB. A predictive instrument to improve coronary-care-unit admission practices in acute ischemic heart disease: a prospective multicenter clinical trial.  N Engl J Med.1984;310:1273-1278.
Wellwood J, Johannessen S, Spiegelhalter DJ. How does computer-aided diagnosis improve the management of acute abdominal pain?  Ann R Coll Surg Engl.1992;74:40-46.
Wexler JR, Swender PT, Tunnessen WW, Oski FA. Impact of a system of computer-assisted diagnosis: initial evaluation of the hospitalized patient.  AJDC.1975;129:203-205.
Wyatt JR. Lessons learnt from the field trial of ACORN, an expert system to advise on chest pain. In: Proceedings of the Sixth World Conference on Medical Informatics; Singapore; 1989:111-115.
Barnett GO, Winickoff RN, Morgan MM, Zielstorff RD. A computer-based monitoring system for follow-up of elevated blood pressure.  Med Care.1983;21:400-409.
McDowell I, Newell C, Rosser W. Comparison of three methods of recalling patients for influenza vaccination.  CMAJ.1986;135:991-997.
McDowell I, Newell C, Rosser W. Computerized reminders to encourage cervical screening in family practice.  J Fam Pract.1989;28:420-424.
McDowell I, Newell C, Rosser W. A randomized trial of computerized reminders for blood pressure screening in primary care.  Med Care.1989;27:297-305.
Rosser WW, Hutchison BG, McDowell I, Newell C. Use of reminders to increase compliance with tetanus booster vaccination.  CMAJ.1992;146:911-917.
Tierney WM, Hui SL, McDonald CJ. Delayed feedback of physician performance versus immediate reminders to perform preventive care: effects on physician compliance.  Med Care.1986;24:659-666.
Brownbridge G, Evans A, Fitter M, Platts M. An interactive computerized protocol for the management of hypertension: effects on the general practitioner's clinical behaviour.  J R Coll Gen Pract.1986;36:198-202.
Coe FL, Norton E, Oparil S, Tatar A, Pullman TN. Treatment of hypertension by computer and physician—a prospective controlled study.  J Chronic Dis.1977;30:81-92.
Keller LS, McDermott S, Alt-White A. Effects of computerized nurse care planning on selected health care effectiveness measures.  Proc Annu Symp Comput Appl Med Care.1991;:38-42.
McAlister NH, Covvy HD, Tong C, Lee A, Wigle ED. Randomized controlled trial of computer assisted management of hypertension in primary care.  Br Med J (Clin Res Ed).1986;293:670-674.
McDonald CJ. Use of a computer to detect and respond to clinical events: its effect on clinician behavior.  Ann Intern Med.1976;84:162-167.
McDonald CJ. Protocol-based computer reminders, the quality of care and the non-perfectability of man.  N Engl J Med.1976;295:1351-1355.
McDonald CJ, Wilson GA, McCabe GP. Physician response to computer reminders.  JAMA.1980;244:1579-1587.
McDonald CJ, Hui SL, Smith DM.  et al.  Reminders to physicians from an introspective computer medical record.  Ann Intern Med.1984;100:130-138.
McDonald CJ, Hui SL, Tierney WM. Effects of computer reminders for influenza vaccination on morbidity during influenza epidemics.  MD Comput.1992;9:304-312.
Petrucci K, Petrucci P, Canfield K.  et al.  Evaluation of UNIS: urological nursing information systems.  Proc Annu Symp Comput Appl Med Care.1991;:43-47.
Rogers JL, Haring OM. The impact of computerized medical record summary system on incidence and length of hospitalization.  Med Care.1979;17:618-630.
Rogers JL, Haring OM, Wortman PM. Medical information systems: assessing impact in the areas of hypertension, obesity and renal disease.  Med Care.1982;20:63-74.
Rogers JL, Haring OM, Goetz JP. Changes in patient attitudes following the implemenation of a medical information system.  Q Rev Bull.1984;10:65-74.
Tierney WM, McDonald CJ, Hui SL, Martin DK. Computer predictions of abnormal test results: effects on outpatient testing.  JAMA.1988;259:1194-1198.
White KS, Lindsay A, Pryor TA, Brown WF, Walsh K. Application of a computerized medical decision-making process to the problem of digoxin intoxication.  J Am Coll Cardiol.1984;4:571-576.
Young DW. Improving the consistency with which investigations are requested.  Med Inf (Lond).1981;6:13-17.
Abbrecht PH, O'Leary TJ, Behrendt DM. Evaluation of a computer-assisted method for individualized anticoagulation: retrospective and prospective studies with a pharmacodynamic model.  Clin Pharmacol Ther.1982;32:129-136.
Begg EJ, Atkinson HC, Jeffery GM, Taylor NW. Individualised aminoglycoside dosage based on pharmacokinetic analysis is superior to dosage based on physician intuition at achieving target plasma drug concentrations.  Br J Clin Pharmacol.1989;28:137-141.
Carter BL, Taylor JW, Becker A. Evaluation of three dosage-prediction methods for initial in-hospital stabilization of warfarin therapy.  Clin Pharm.1987;6:37-45.
Casner PR, Reilly R, Ho H. A randomized controlled trial of computerized pharmacokinetic theophylline dosing versus empiric physician dosing.  Clin Pharmacol Ther.1993;53:684-690.
Hickling K, Begg EJ, Moore ML. A prospective randomised trial comparing individualised pharmacokinetic dosage prediction for amnioglycosides with prediction based on estimated creatinine clearance in critically ill patients.  Intensive Care Med.1989;15:233-237.
Hurley SF, Dziukas LJ, McNeil JJ, Brignell MJ. A randomized controlled clinical trial of pharmacokinetic theophylline dosing.  Am Rev Respir Dis.1986;134:1219-1224.
Mungall D, Anbe D, Forrester PL.  et al.  Clinic trials and therapeutics: a prospective randomized comparison of the accuracy of computer-assisted versus GUSTO nomogram—directed heparin therapy.  Clin Pharmacol Ther.1994;55:591-596.
Poller L, Wright D, Rowlands M. Prospective comparative study of computer programs used for management of warfarin.  J Clin Pathol.1993;46:299-303.
Vadher B, Patterson DLH, Leaning M. Evaluation of a decision support system for initiation and control of oral anticoagulation in a randomised trial.  BMJ.1997;314:1252-1256.
Vadher BD, Patterson DLH, Leaning M. Comparison of oral anticoagulant control by a nurse-practitioner using a computer decision-support system with that by clinicians.  Clin Lab Haematol.1997;19:203-207.
Verner D, Seligmann H, Platt S. Computer assisted design of a theophylline dosing regimen in acute bronchospasm: serum concentrations and clinical outcome.  Eur J Clin Pharmacol.1992;43:29-33.
Burack RC, Gimotty PA, George J.  et al.  Promoting screening mammography in inner-city settings: a randomized controlled trial of computerized reminders as a component of a program to facilitate mammography.  Med Care.1994;32:609-624.
Burack RC, Gimotty PA, George J.  et al.  The effect of patient and physician reminders on use of screening mammography in a health maintenance organization: results of a randomized controlled trial.  Cancer.1996;78:1708-1721.
Burack RC, Gimotty PA. Promoting screening mammography in inner-city settings: the sustained effectiveness of computerized reminders in a randomized controlled trial.  Med Care.1997;35:921-931.
Chambers CV, Balaban DJ, Carlson BL, Grasberger DM. The effect of microcomputer-generated reminders on influenza vaccination rates in a university-based family practice center.  J Am Board Fam Pract.1991;4:19-26.
Frame PS, Zimmer JG, Werth PL, Hall WJ, Eberly SW. Computer based vs manual health maintenence tracking: a controlled trial.  Arch Fam Med.1994;3:581-588.
McPhee SJ, Adair Bird J, Jenkins CN, Fordham D. Promoting cancer screening: a randomized, controlled trial of three interventions.  Arch Intern Med.1989;149:1866-1872.
McPhee SJ, Adair Bird J, Fordham D, Rodnick JE, Osborn EH. Promoting cancer prevention activities by primary care physicians: results of a randomized controlled trial.  JAMA.1991;266:538-544.
Ornstein SM, Garr DR, Jenkins RG, Rust PF, Arnon A. Computer-generated physician and patient reminders: tools to improve population adherence to selected preventive services.  J Fam Pract.1991;32:82-90.
Overhage JM, Tierney WM, McDonald CJ. Computer reminders to implement preventive care guidelines for hospitalized patients.  Arch Intern Med.1996;156:1551-1556.
Rosser WW, McDowell I, Newell C. Use of reminders for preventive procedures in family medicine.  CMAJ.1991;145:807-812.
Tape TG, Campbell JR. Computerized medical records and preventive health care: success depends on many factors.  Am J Med.1993;94:619-625.
Turner RC, Peden JG, O'Brien K. Patient-carried card prompts vs computer-generated prompts to remind private practice physicians to perform health maintenance measures.  Arch Intern Med.1994;154:1957-1960.
Turner BJ, Day SC, Borenstein B. A controlled trial to improve delivery of preventive care: physician or patient reminders.  J Gen Intern Med.1989;4:403-409.
Bates DW, Kuperman GJ, Rittenberg E.  et al.  Reminders for redundant tests: results of a randomized controlled trial [abstract].  Proc Annu Symp Comput Appl Med Care.1995:935.
Dexter PR, Wolinsky FD, Gramelspacher GP.  et al.  Effectiveness of computer-generated reminders for increasing discussions about advance directives and completion of advance directive forms: a randomized, controlled trial.  Ann Intern Med.1998;128:102-110.
Fihn SD, McDonell MB, Vermes D.  et al.  A computerized intervention to improve timing of outpatient follow-up: a multicenter randomized trial in patients treated with warfarin.  J Gen Intern Med.1994;9:131-139.
Hales JW, Gardner RM, Jacobson JT. Factors impacting the success of computerized preadmission screening.  Proc Annu Symp Comput Appl Med Care.1995;:728-732.
Lobach DF, Hammond WE. Development and evaluation of a computer-assisted management protocol (CAMP): improved compliance with care guidelines for diabetes mellitus.  Proc Annu Symp Comput Appl Med Care.1994;:787-791.
Mazzuca SA, Vinicor F, Einterz RM, Norton JA, Kalasinski LA. Effects of the clinical environment on physicians' response to postgraduate medical education.  Am Educ Res J.1990;27:473-488.
Nilasena DS, Lincoln MJ. A computer-generated reminder system improves physician compliance with diabetes preventive care guidelines.  Proc Annu Symp Comput Appl Med Care.1995;:640-645.
Overhage JM, Tierney WM, Zhou XA, McDonald CJ. A randomized trial of "corollary orders" to prevent errors of omission.  J Am Med Inform Assoc.1997;4:364-375.
Rossi RA, Every NR. A computerized intervention to decrease the use of calcium channel blockers in hypertension.  J Gen Intern Med.1997;12:672-678.
Rotman BL, Sullivan AN, McDonald TW.  et al.  A randomized controlled trial of a computer-based physician workstation in an outpatient setting: implementation barriers to outcome evaluation.  J Am Med Inform Assoc.1996;3:340-348.
Rubenstein LV, McCoy JM, Cope DW. Improving patient quality of life with feedback to physicians about functional status.  J Gen Intern Med.1995;10:607-614.
Ryff-de-Leche A, Engler H, Nutzi E, Berger M, Berger W. Clinical application of two computerized diabetes management systems: comparison with the log-book method.  Diabetes Res.1992;19:97-105.
Safran C, Rind DM, Davis R.  et al.  Guidelines for management of HIV infection with computer-based patient's record.  Lancet.1995;346:341-346.
Shea S, Sideli RV, DuMouchel W.  et al.  Computer-generated informational messages directed to physicians: effect on length of hospital stay.  J Am Med Inform Assoc.1995;2:58-64.
Thomas JC, Moore A, Qualls PE. The effect on cost of medical care for patients treated with an automated clinical audit system.  J Med Syst.1983;7:307-313.
Tierney WM, Miller ME, Overhage M, McDonald CJ. Physician inpatient order writing on microcomputer workstations: effects on resource utilization.  JAMA.1993;269:379-383.
Sheiner LB, Halkin H, Peck C, Rosenberg B, Melmon KL. Improved computer-assisted digoxin therapy: a method using feedback of measured serum digoxin concentrations.  Ann Intern Med.1975;82:619-627.
Fitzmaurice DA, Hobbs FDR, Murray ET, Bradley CP, Holder R. Evaluation of computerized decision support for oral anticoagulation management based in primary care.  Br J Gen Pract.1996;46:533-535.
Pringle M, Robins S, Brown G. Computer assisted screening: effects on the patient and his consultation.  BMJ.1990;290:1709-1712.
Hobbs FD, Delaney BC, Carson A, Kenkre JE. A prospective controlled trial of computerized decision support for lipid management in primary care.  Fam Pract.1996;13:133-137.
Destache CJ, Meyer SK, Bittner MJ, Hermann KG. Impact of a clinical pharmacokinetic service on patients treated with aminoglycosides: a cost-benefit analysis.  Ther Drug Monit.1990;12:419-426.
Dasta JF, Greer ML, Speedie SM. Computers in healthcare: overview and bibliography.  Ann Pharmacother.1992;26:109-117.
Miller RA. Medical diagnostic decision support systems—past, present, and future.  J Am Med Inform Assoc.1994;1:8-27.
McPhee SJ, Detmer WM. Office-based interventions to improve delivery of cancer prevention services by primary care physicians.  Cancer.1993;72:1100-1112.
Balas EA, Austin SM, Mitchell JA.  et al.  The clinical value of computerized information services: a review of 98 randomized clinical trials.  Arch Fam Med.1996;5:271-278.
Shea S, DuMouchel W, Bahamonde L. A meta-analysis of 16 randomized controlled trials to evaluate computer-based reminder systems for preventive care in the ambulatory setting.  J Am Med Inform Assoc.1996;3:399-409.
Litzelman DK, Dittus RS, Miller ME, Tierney WM. Requiring physicians to respond to computerized reminders improves their compliance with preventive care protocols.  J Gen Intern Med.1993;8:311-317.
Lobach DF. Electronically distributed, computer-generated, individualized feedback enhances the use of a computerized practice guideline.  Proc AMIA Annu Fall Symp.1996;:493-497.

Letters

CME
Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.

Multimedia

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Collections
PubMed Articles
JAMAevidence.com

Users' Guides to the Medical Literature
Chapter 9.6. Clinical Decision Support Systems

The Rational Clinical Examination
Evidence Summary and Review 3