Users' Guides to the Medical Literature: Title and subTitle BreakXX. Integrating Research Evidence With the Care of the Individual Patient

You are the attending physician on an internal medicine service who, one night, admits 2 patients with strokes (patient A, a 65-year-old woman; patient B, a 65-year-old man). On examination, both have mild weakness of the right arm and left carotid bruits. Patient A has a history of hypertension and an admission blood pressure of 200/110 mm Hg; neither patient has other relevant medical history or physical examination findings.

Aware that carotid bruits are not highly specific for identifying carotid artery stenosis, you send both patients for Doppler ultrasonography.¹ Since your radiology department, in a recent audit, demonstrated that their ultrasonographic interpretations are highly correlated with angiographic results,² you feel confident from their findings that both patients have moderate stenoses (50%-69% by North American Symptomatic Carotid Endarterectomy Trial criteria) with no irregularity or ulceration of the plaque surface.³

Aware of the recent flurry of literature concerning surgical vs medical therapy for patients with symptomatic carotid stenoses, you decide to review the literature to guide your management of these patients. You formulate the question: "In a patient with a mild stroke and moderate ipsilateral carotid stenosis, would a carotid endarterectomy (compared with best medical therapy) reduce the likelihood of subsequent severe stroke or death?"

A systematic review of randomized trials comparing carotid endarterectomy with standard medical therapy (aspirin in your practice setting) in patients with recent mild stroke would provide the best evidence to answer your question. Through your hospital library, you have access to Ovid Evidence-Based Medicine Reviews, allowing you to search both Best Evidence (which includes the contents of ACP Journal Club and Evidence-Based Medicine) and the Cochrane Database of Systematic Reviews with a single search engine. Using the search terms stroke and carotid endarterectomy, you don't find any relevant reviews in the Cochrane Database but you retrieve 18 citations from Best Evidence. Scanning these citations you find one that looks relevant to your question⁴ and after reviewing the abstract and commentary from Best Evidence, you link to the full-text article for further details.

Investigators in this trial randomized 2267 patients with moderate carotid stenosis (<70%) and ipsilateral transient ischemic attacks or nondisabling stroke within 180 days to carotid endarterectomy or medical care alone.⁴ After 5 years of follow-up, significantly fewer patients in the carotid endarterectomy arm (vs the medical care arm) had suffered a recurrent disabling stroke (5.3% vs 10.3%; 49% relative risk reduction [RRR]; 95% confidence interval [CI], 14% to 83%]) or death (13% vs 15%; 13% RRR; 95% CI, –18% to 44%). The size of the treatment effect was such that 20 patients (95% CI, 12 to 70) would have to undergo carotid endarterectomy to prevent 1 disabling stroke that would occur with medical therapy alone. Although encouraged by these results, you are concerned about the wide CIs and the potential for perioperative complications (1.4% excess risk of disabling stroke or death within the first month of surgery), and you question how to apply the results to your patients.

While randomized trials provide the most valid estimates of the true effects (both beneficial and harmful) of an intervention, they necessarily report average treatment effects. Whether these results are derived from a homogeneous group of high-risk, highly responsive patients (as in efficacy trials) or a heterogeneous group of "all-comers" (as in effectiveness trials),⁵ clinicians must decide how to extrapolate the results to individual patients. In this article, we will build on previous Users' Guides^{6 - 9} that assessed the validity and applicability of therapeutic studies to outline a framework that clinicians might use to integrate research results (whether from single trials or systematic reviews) with patient values to determine the optimal care for an individual patient.

Previous Users' Guides and other articles have dealt extensively with issues of determining the applicability of evidence to individual patients.^{7 - 10} We will not repeat all of the key principles here, but will emphasize that differences between study participants and patients in real-world practice tend to be quantitative (differences in degree of risk of the outcome or responsiveness to therapy) rather than qualitative (no risk or adverse response to therapy).^{8 ,10} These variations may be unimportant (eg, angiotensin converting enzyme inhibitors appear to exhibit similar beneficial effects in patients with systolic congestive heart failure regardless of cause, severity of symptoms, age, or sex)¹¹ or easily remediable (eg, drug dosages can be adjusted based on individual patient responsiveness).

Restricting efficacious therapies to "ideal patients" may result in significant harm to those excluded. For example, while β-blockers are prescribed to only a minority of patients with acute myocardial infarction, myocardial infarction patients with concomitant conditions that might lead clinicians to withhold treatment (such as peripheral vascular disease, diabetes mellitus, heart failure, or chronic obstructive pulmonary disease) derive substantial survival benefits from β-blocker therapy.¹² This message is a consistent theme emerging from cardiovascular outcomes research.¹³

A key element to consider in extrapolating the results of the carotid endarterectomy trial that you identified is local surgical expertise because the net benefits in the trial were highly sensitive to perioperative complication rates. In fact, the benefits from carotid endarterectomy in this trial (expressed as RRR in disabling stroke) would be reduced by 20% for each 2% absolute increase in the rate of perioperative stroke and death.¹⁴ Moreover, surgical teams whose complication rates and operative volumes would have rendered them ineligible for the trial perform the majority of endarterectomies in North America.¹⁵ Thus, as has been pointed out by others, "caution should be exercised in drawing conclusions about the effectiveness of carotid endarterectomy in the general population on the basis of trials of clinical efficacy conducted at highly selected facilities."¹⁵

The process of individualizing research evidence to the care of a particular patient incorporates 2 components: determining the likelihood that treatment will prevent the target event (at the expense of adverse events) in that patient and incorporating the patient's values. We will now consider both of these steps in some depth.

Although we can summarize the results of randomized trials with binary outcomes in a number of ways, the number of patients that would need to be treated to prevent 1 additional adverse event (number needed to treat [NNT])¹⁶ has gained widespread acceptance as 1 clinically relevant format.^{17 - 18} The NNT is the inverse of the difference in absolute event rates between the experimental and control arms and thus reflects baseline risk as well as treatment effect.¹⁷ For example, the NNT to prevent 1 disabling stroke in patients with moderate carotid artery stenosis is 20, calculated as follows: control event rate (10.3%) minus experimental event rate (5.3%) equals absolute risk reduction (5%). The NNT is the inverse of the absolute risk reduction (1/0.05=20).⁴

Analogous to the NNT, the number needed to harm (NNH) is an expression of the number of patients who would need to receive an intervention to cause 1 additional adverse event. The NNH is the inverse of the absolute difference in adverse event rates between the experimental and control arms. For example, a meta-analysis of 51 studies of carotid endarterectomy in patients with symptomatic carotid stenosis found that the absolute perioperative mortality rate was 1.6% higher with endarterectomy than with medical treatment: this translates into an NNH to cause 1 additional death in the perioperative period with carotid endarterectomy of 63 compared with withholding surgery.¹⁹

While one can easily calculate NNT when investigators report event rates and relative risks (RRs), difficulties arise when investigators report only odds ratios (ORs). Since the OR is not always an accurate estimate of the RR (particularly as disease incidence increases above 10%),²⁰ the clinician must employ standard formulas¹⁸ to derive the NNT or NNH from the OR (Table 1). Alternatively, a nomogram has been developed for converting ORs to RRs.²¹

The average NNT (or NNH) reported in a trial or systematic review may not be directly applicable to an individual patient (because of differences in baseline risk and/or RRR across subgroups), and the clinician is faced with 3 questions in extrapolating to his or her patient: Is my patient's RRR likely to be different from the group average? What is my patient's baseline risk of the target event? What is my patient's risk of harm from the treatment?

Although we often assume that RRRs are constant across the limited range of susceptibilities normally encountered in clinical practice,^{22 - 24} recently published studies have demonstrated that while this is often the case,^{25 - 33} it may not always be.^{31 - 35} Thus, the clinician must carefully scrutinize the reports of trials or systematic reviews for information on the relative treatment effects in different subgroups and should use available criteria for evaluating subgroup analyses.²⁴ In situations where RRR does appear to differ across subgroups, clinicians should employ the RRR from the subgroup most similar to their patient.

Returning to our clinical scenario, the RRR for stroke with carotid endarterectomy does differ by degree of stenosis and presurgical symptom status.¹⁴ Because our patients have symptomatic stenoses of 50%-69%, it would be inappropriate to extrapolate directly the results from either a trial of symptomatic patients with high-grade stenoses (>70%)³⁶ or a trial of asymptomatic patients with moderate stenoses³⁷ to their situation. However, it is possible to extrapolate from the previously identified study⁴ that enrolled symptomatic patients with similar degrees of stenoses as our patients.

We will now outline 2 approaches to addressing the latter 2 questions, our patient's risk of adverse events without treatment and our patient's risk of harm with therapy.²² Both approaches that are described below require time, but with the explosion in the development of electronic evidence resources, this obstacle may be ameliorated in the near future.

Recognizing that patients are rarely identical to the average study patient, clinicians can derive estimates of the patient's baseline risk from various sources. First, if the study reports risk in various subgroups, clinicians can use the baseline risk for the subgroup most like their patient. However, most trials are not large enough to allow generation of precise estimates of baseline risk in various patient subgroups, and the clinician may have to search for systematic reviews (particularly those including individual patient data)³⁸ to glean useful information. For example, the Atrial Fibrillation Investigators pooled the individual patient data from all randomized trials testing antithrombotic therapy in nonvalvular atrial fibrillation and were able to provide estimates of prognosis for patients in clinically important subgroups.²⁵

Second, as an extension of the subgroup approach, one can use clinical prediction guides to quantitate an individual patient's potential for benefit (and harm) from therapy.^{33 ,39 - 40} Returning to our example, a prognostic model that could identify patients with carotid stenosis most likely to benefit from endarterectomy would be useful. Such a model would need to incorporate the risk of stroke without surgery (and thus the potential benefit from surgery) with the risk of stroke or other adverse outcomes from surgery. Using the European Carotid Surgery Trial database,⁴¹ investigators have developed a preliminary version of just such a model.⁴² However, our enthusiasm for applying this clinical prediction guide should be tempered until it has been prospectively validated in a different group of patients (and preferably with different clinicians).³⁹

Third, clinicians could derive an estimate of their patient's baseline risk from published articles (preferably population-based cohort studies)⁴³ that describe the prognosis of similar (untreated) patients. For example, analysis of the Malmo Stroke Registry demonstrated that in the 3 years after a stroke, patients have a 6% risk of recurrent nonfatal stroke and a 43% risk of death; these risks were higher in older patients or those with diabetes mellitus or cardiac disease.⁴⁴

Analogous to the estimation of patient-specific baseline risk, clinicians can use these same sources of information to determine an individual patient's likelihood of harm from treatment. For example, a systematic review of 36 studies relating the risk of perioperative complications from carotid endarterectomy to various preoperative clinical characteristics revealed that women were at higher risk than men (OR, 1.44; 95% CI, 1.14 to 1.83; absolute rate, 5.2%).⁴⁵

The final step in generating a patient-specific NNT (or NNH) involves the formula: NNT=1/(PEER × RRR) (where PEER is the patient's estimated event rate or baseline risk).²¹ Given the 3-year risk of recurrent disabling stroke in diabetic patients from the Malmo Stroke Registry (8.4%)⁴⁴ and the 49% RRR expected with carotid endarterectomy,⁴ the patient-specific NNT in a 65-year-old patient with diabetes, ipsilateral carotid stenosis, and a minor stroke would be calculated as NNT=1/(0.084 × 0.49)=24. Clinicians who know a patient's baseline risk and RRR can also use a nomogram to calculate the NNT.⁴⁶

Alternately, the clinician can use the NNT and NNH directly from a study to generate patient-specific estimates. This method involves only 2 steps and is less time-consuming than the previous method (because, depending on the experience of the clinician, it may not require a detailed literature review).

First, the clinician estimates the patient's risk of the outcome event relative to that of the average control patient in the study and converts this risk to a decimal fraction (labeled f _t , "for treatment").⁴⁷ Patients judged to be at less risk than those in the trials will be assigned an f _t less than 1 and those thought to be at greater risk will be assigned an f _t greater than 1. There are several sources that a clinician can use to obtain a value for f _t . The best estimate would come from a systematic review of all available data about the prognosis of similar patients; individual studies about prognosis would provide the next best estimates. Alternatively, the clinician could use clinical expertise in assigning a value to f _t . While this may appear to be overly subjective, preliminary data suggest that experienced clinicians may be accurate in estimating relative differences in baseline risk (ie, f _t ) between patients (far exceeding our abilities to judge absolute risks).⁴⁸

Second, the clinician calculates the patient-specific NNT by dividing the average NNT by f_t. Thus, if the clinician felt that patient A was at one fifth (f_t=_t2) the risk of the average patient in the trial (based on the reduced baseline risk for women demonstrated in the subgroup analyses reported by the investigators),⁴ her patient-specific NNT for the prevention of 1 disabling stroke would be 100 (20/0.2).

In addition to considering the benefits from therapy, the clinician needs to consider a patient's risk of adverse events from any intervention. Patients A and B need to be informed that carotid endarterectomy does carry with it a risk of perioperative death. To individualize your patient's risk of death, you can use the f method just described (labeled f_h, "for harm"). For example, patient A may be assumed to be at twice the risk (f_h=2) of perioperative death as patients in the control group of the study because of her gender, hypertension, and the fact that she has left-sided carotid artery stenosis.^{4 ,45} You can adjust the NNH using f_h, assuming the RR increase is constant across the spectrum of susceptibilities (an assumption that, as we've noted for RRR, may or may not hold depending on the particular therapy being considered). Thus, patient A's NNH is estimated to be approximately 32 (63/2).

We have determined the risks of benefit and harm for the individual, but we must still incorporate patient values into the decision-making process. As outlined in a previous Users' Guide,⁹ systematically constructed decision analyses and practice guidelines that include an explicit statement of values can be used to integrate the evidence on benefit or harm with patient values to reach treatment recommendations or establish threshold NNTs.^{9 ,49} Although this situation would be ideal, such evidence is often not available (we could not, for instance, identify a relevant decision analysis for our scenario). Moreover, as there is often substantial variation in values between individuals,^{50 - 52} decision analyses that rely on group averages for values may not always be applicable to a particular patient, although close examination of the utility sensitivity analyses of a decision analysis may provide some guidance.^{53 - 55}

While active patient involvement in decision making can improve outcomes and reported quality of life and possibly reduce health care expenditures,^{56 - 62} the initial step in this process is to determine the extent to which your patient wants to be involved in decision making (recognizing that this may vary with each clinical decision).

There are 3 main elements to clinical decision making: the disclosure of information (about the risks and benefits of therapeutic alternatives); the exploration of the patient's values about both the therapy and the potential health outcomes; and the actual decision. Each patient varies in desired level of involvement in these steps, and clinicians may not accurately gauge the degree to which an individual patient wants to be involved.^{63 - 68} Some patients may want all available information provided to them and may want to make the decision themselves, with the clinician's role being that of information provider. Other patients may want all the information provided but may want the clinician to make the final decision. Still others may want to collaborate with their clinician in the entire process. These differences emphasize the need for clinicians to accurately assess patient preferences for information, discussion, and decision making and tailor their approach to the individual.

Regardless of whether the clinician, the patient, or both in partnership will make the decision, clinicians must explore patients' values about the therapy and the potential health outcomes. You can elicit your patient's values in informal ways during exploratory discussions or by more formal (and time-consuming) methods such as the time trade-off, standard gamble, or rating scale techniques.⁶⁹

If your patient's goal is shared decision making, there are several models available for providing shared decision-making support. First, formal clinical decision analysis, incorporating the patient's likelihood of the outcome events with his or her own values for each health state, could be used to guide the decision. Performing a clinical decision analysis for each patient would be too time-consuming for the busy clinician, and this approach therefore currently relies on finding an existing decision analysis. To be able to use the existing decision analysis, either our patient's values must approximate those in the analysis, or the decision analysis must provide information about the impact of variation in patient values on the results of the decision analysis. Computer models available at the bedside may broaden the scope of decision analysis applicability and permit wider use with individual patients.⁷⁰

Second, investigators have developed numerical methods of presenting information to patients that incorporate calculated patient values.^{40 ,71} However, these methods have not been fully tested and are not yet feasible for widespread use. Here too, computer models may be useful in the future. Third, clinicians can use "decision aids" that present descriptive and probabilistic information about the disease, treatment options, and potential outcomes.^{72 - 75} Most commonly, these decision aids present the outcome data in terms of the percentage of people with a certain condition who do well without intervention compared with the percentage who do well with intervention. While each of these methods has considerable merit, they sometimes fall short in terms of comprehensibility, applicability, and efficiency for use in busy clinical services. Making well-validated decision aids available on the Internet could improve their clinical usefulness.

Although the NNT and NNH are useful for clinicians to describe the benefits and harms of therapy, they may be less informative for individual patients who want to know their unique risk of these events. One recently developed method of expressing information to patients that incorporates patient values, can be applied to any clinical decision, and that preliminary evidence suggests may be useful in busy clinical services is the likelihood of being helped vs harmed. (S.E.S., unpublished data, 2000). The first step in this method is the exploration of patient values about receiving the treatment (vs not receiving it) and the severity of adverse events that might be caused by the treatment (vs the severity of the target event that we hope to avoid with the treatment). To answer these questions, patients are provided with brief descriptions of both the target event to be prevented and the potential adverse event from the treatment ( Article ).

Following review of the description of the target event, the clinician presents the patient with a rating scale (anchored at 0 [death] and 1 [full health]) and asks him or her to mark the value of the target event.

During your discussions with patient A, you discover that she is a fiercely independent newspaper journalist who lives alone and previously cared for her father after he suffered a disabling stroke. She believes that a disabling stroke is as bad as immediate death and assigns it a value of 0. Similarly, you give your patient the description of the adverse event that could result from the therapy (death within 30 days of surgery) and ask her to assess this using the rating scale (she assigned a value of 0.25 since death may not necessarily be immediate). Using the 2 ratings, you infer that she believes a disabling stroke to be 1.3 times worse than death within the next month [(1-0)/1-0.25)]. This exercise should be repeated on another occasion to confirm that her values are stable.

A stroke can result in weakness and loss of function in one side of your body. With a disabling stroke, you are admitted to a hospital for initial treatment (which would include some rehabilitation therapy) and then transferred to a rehabilitation hospital for at least 2 months of intense rehabilitation. You regain some movement in your arm and leg but are left with a permanent weakness in that side of your body and require assistance with activities of daily living such as getting dressed, taking a bath, cooking, eating, and using a toilet. You have trouble getting the words out when you speak.

A surgical procedure called carotid endarterectomy can decrease the risk of disabling stroke but can result in death. This surgery involves repairing one of the major blood vessels in your neck that supplies blood to your brain. It must be performed by a surgeon with experience in this procedure. Death is most likely to occur in the first 30 days after this surgical procedure.

In contrast, during your conversation with patient B, you find that he is a former truck driver who recently retired to the country with his wife so that he could be near his daughter and grandson. When you explore his values, he decides that death is 5 times worse than having a disabling stroke.

How can you now incorporate your individual patients' values into the description of therapy? The average patient with a hemispheric stroke and ipsilateral moderate carotid stenosis has a 10.3% chance of having a disabling stroke over 5 years, but this can be decreased to 5.3% with carotid endarterectomy.⁴ The average NNT for such patients is 20. The absolute risk increase for death for patients having carotid endarterectomy is 1.6%,¹⁹ which translates to an average NNH of 63 (1/0.02). You work in a hospital where the vascular surgeons have a perioperative mortality rate of 2%, and therefore you can apply this study NNH to your patients.

To calculate the likelihood of being helped vs harmed (LHH), 1/NNT (absolute risk reduction [ARR]) and 1/NNH (absolute risk increase [ARI]) are combined into an aggregate ratio. (Note that although we use 1/NNT and 1/NNH here, alternatively we could use ARR and ARI in these calculations. In a pilot study, we found that physicians made fewer errors in calculation when using NNT/NNH vs ARR/ARI, and many of the errors were in decimal placement.) For both patients, the first approximation of the LHH is LHH=(1/NNT) : (1/NNH)=(1/20) : (1/63)=3 to 1 in favor of surgery. As a first approximation, both patients can be told that "carotid endarterectomy is 3 times as likely to help you as harm you."

However, this first approximation ignores both patients' individual risks of, and values relating to, stroke and perioperative death. You can particularize the LHH for each patient using the f factors we described previously. As discussed above, women have a lower risk of stroke and the f _t for patient A can be estimated at approximately 0.2.⁴ This study (and a systematic review of other studies⁴⁵ ) found that women, patients with left-sided carotid disease, and patients with a history of hypertension have increased risks of perioperative deaths (RRs, 1.4-2.3). Thus, patient A is at an increased risk of death from surgery (f _h =2). Her risk-adjusted LHH is: LHH_A=[(1/NNT) × f _t : [(1/NNH) × f _h ]=[(1/20) × 0.2] : [(1/63) × 2] = 3 to 1 in favor of medical therapy. Similarly, the LHH for patient B can be individualized for his unique risks. Men had a greater risk of stroke in the trial^{1 ,4} and you can estimate from the reported subgroup analyses that patient B's f _t is approximately 1.25. Patient B also has left-sided carotid disease, suggesting that his risk of perioperative death is increased (f _h =2). His risk-adjusted LHH is: LHH_B=[(1/20) × 1.25] : [(1/63) × 2]=2 to 1 in favor of surgery.

These risk-adjusted LHHs still ignore each patient's values. Patient A ranked a disabling stroke as 1.3 times worse than death, and this number (the s or severity factor) can be used to adjust the LHH as follows: LHH_A =[(1/NNT) × f_t × s]: [(1/NNH) × f _h =[(1/20) × 0.2 × 1.3] : [(1/63) × 2]=2 to 1 in favor of medical therapy. Thus, incorporating patient A's values and unique risks of benefit and harm, she is twice as likely to be helped as harmed by medical therapy. On the other hand, patient B stated that death was 5 times worse than a stroke and incorporating this into his LHH you calculate: LHH_B=[(1/20) × 1.25] : [(1/63) × 2 × 5]=3 to 1 in favor of medical therapy.

These 2 cases illustrate how to incorporate your patient's values into the decision-making process. At present, this process is time-consuming and inexact, and we don't know how much difference it makes to patients or their clinical outcomes. Thus, this approach is best considered as a logical and feasible, but untested, model. Computerized versions of this approach should make it more clinically useful. If you are unsure of your patient's f or if there is some uncertainty around your patient's estimate of values, you could do a sensitivity analysis (inserting different values for these variables into the above equation to see how this is reflected in the LHH). We've described a simple formulation for the LHH (ignoring other outcomes from carotid endarterectomy and the risks of the diagnostic workup),⁷⁶ but this could be modified for more complex situations.

Before making a final decision with your patient, you need to determine what the perioperative complication rate is in your own practice setting. If we assume that local surgical expertise is sufficient to apply the study results and use our patients' individual risks of benefit and harm from surgery, adjusted for their unique values, medical therapy appears to be the favored management strategy for both patients.