We're unable to sign you in at this time. Please try again in a few minutes.
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Special Communication |

Pitfalls of Converting Practice Guidelines Into Quality Measures:  Lessons Learned From a VA Performance Measure

Louise C. Walter, MD; Natalie P. Davidowitz, BA; Paul A. Heineken, MD; Kenneth E. Covinsky, MD, MPH
JAMA. 2004;291(20):2466-2470. doi:10.1001/jama.291.20.2466.
Text Size: A A A
Published online

The Department of Veterans Affairs (VA) manages the largest health care system in the United States, and the Institute of Medicine has recommended that many practices of VA quality measurement be applied to the US health care system as a whole. The VA measures quality of care at all of its sites by assessing adherence rates to performance measures, which generally are derived from evidence-based practice guidelines. Higher adherence rates are used as evidence of better quality of care. However, there are problems with converting practice guidelines, intended to offer guidance to clinicians, into performance measures that are meant to identify poor-quality care. We suggest a more balanced perspective on the use of performance measures to define quality by delineating conceptual problems with the conversion of practice guidelines into quality measures. Focusing on colorectal cancer screening, we use a case study at 1 VA facility to illustrate pitfalls that can cause adherence rates to guideline-based performance measures to be poor indicators of the quality of cancer screening. Pitfalls identified included (1) not properly considering illness severity of the sample population audited for adherence to screening, (2) not distinguishing screening from diagnostic procedures when setting achievable target screening rates, and (3) not accounting for patient preferences or clinician judgment when scoring performance measures. For many patients with severe comorbid illnesses or strong preferences against screening, the risks of colorectal cancer screening outweigh the benefits, and the decision to not screen may reflect good quality of care. Performance measures require more thoughtful specification and interpretation to avoid defining high testing rates as good quality of care regardless of who received the test, why it was performed, or whether the patient wanted it.

Figures in this Article
Sign in to download this free article PDF

Sign in

Create a free personal account to sign up for alerts, share articles, and more.

Purchase Options

• Buy this article
• Subscribe to the journal

First Page Preview

View Large
First page PDF preview


Figure 1. Severity of Comorbid Illness by Age of Patients Audited for Colorectal Cancer Screening
Graphic Jump Location
Severe comorbidity was defined as having at least 1 of the following: severe liver disease, class III or IV congestive heart failure, metastatic cancer, oxygen-dependent chronic obstructive pulmonary disease, severe dementia, or end-stage renal disease requiring dialysis. Moderate comorbidity was defined as having at least 1 of the following: class II congestive heart failure, symptomatic coronary artery disease, stroke with residual deficits, diabetes with end organ damage, moderate dementia, or symptomatic chronic obstructive pulmonary disease. Mild comorbidity was defined as not having any moderate or severe comorbidity.
Figure 2. Flow Diagram of Patients Audited for Adherence to the Colorectal Cancer Screening Performance Measure in 2002 at the SFVAMC
Graphic Jump Location
VA indicates Department of Veterans Affairs; SFVAMC, San Francisco VA Medical Center.
*Patients were counted as tested for colorectal cancer if there was chart documentation of fecal occult blood testing within 1 year, sigmoidoscopy within 5 years, or colonoscopy within 10 years.
†National VA auditors missed documentation of colorectal cancer testing in 20 persons and counted 4 persons as screened for whom we could not find documentation of testing. Therefore, we counted 148 patients as "tested" and 81 as "not tested" whereas VA auditors counted 132 as "tested" and 97 as "not tested."
‡A test was categorized as "diagnostic" rather than "screening" if there was documentation in the medical chart that the test was performed to work up a gastrointestinal complaint or follow up a previously abnormal examination. The indication for a test is routinely documented in all endoscopy reports at the SFVAMC.



Also Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Please click the checkbox indicating that you have read the full article in order to submit your answers.
Your answers have been saved for later.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.


Some tools below are only available to our subscribers or users with an online account.

Web of Science® Times Cited: 117

Sign in to download this free article PDF

Sign in

Create a free personal account to sign up for alerts, share articles, and more.

Purchase Options

• Buy this article
• Subscribe to the journal

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Collections
PubMed Articles