Relevance of Cost-effectiveness Analysis to Clinicians and Policy Makers

One of the main objectives of educating clinicians and policy makers is to help them make better decisions. Should I order blood cultures for this patient? Can this patient with chest pain safely go home or should she be admitted to the hospital? Do I recommend implementing a hospital-wide influenza immunization program for all patients and health care workers? Should my hospital buy a new information technology system?

Most clinicians and health care administrators like to think these decisions are based on the rational use of all relevant information. The traditional patient-physician relationship of 50 years ago required that the physician base his or her recommendation on the clinical risks and benefits of each option. Costs were considered only if the patient bore them directly. Health care is now much more expensive, and a variety of insurance schemes pay for much of it. These third-party payers are definitely interested in the costs as well as the benefits and adverse effects of therapies.¹ Thus, it is not surprising that the measurement of both the costs and benefits of new health care technologies, particularly drugs, has increased greatly in the last 3 decades. Reimbursement parties insist on it.

In this Commentary, we look back over the past 30 years of experience with the application of cost-effectiveness measurement to health care technologies and ask 2 questions: How useful is cost-effectiveness information in actual decision making? And is this information used appropriately? In the classic tradition of economics, the answer to the first question is complex, and the answer to the second is both yes and no.

Cost-effectiveness analysis was invented by mathematicians and linear programmers to help those individuals who had to make allocation decisions in the absence of market places with price signals (ie, where the consumers of goods and services are not faced with the true price of those goods and services).² The principle early use of cost-effectiveness analysis was to guide the efficient allocation of resources to government investment projects that appear to be “free” to consumers who use them. The kinds of choices considered in those analyses were investments in roads vs dams vs parks.³ When third-party payers removed the real price signals from the view of health care consumers through insurance (patients no longer faced the real price of a service at the point of delivery), the application of this managerial technique to health was natural.

The crux of a cost-effectiveness analysis is the measurement of the incremental cost and effects that result from choosing one strategic option over another (eg, influenza vaccine for health care workers vs no vaccine).⁴ In essence, it represents the impact of choices on costs and outcomes. The summary measure is known as an incremental cost-effectiveness ratio. The numerator is an estimate of the expected difference in costs between strategies that are relevant to the perspective of the decision maker. The denominator is the difference in relevant outcomes between strategies, with several choices for units of effectiveness: clinical effects (myocardial infarctions avoided, life years extended), utilities (measures of preference for health states), or benefits (where effects are converted to dollars using methods such as willingness to pay for improved outcomes).⁵

The real purpose of a true cost-effectiveness analysis (as opposed to a cost-effectiveness measurement) in health care is to help the decision maker determine how to allocate resources across a defined number of competing needs to maximize health outcomes from a limited budget.⁴ Consider the following hypothetical example. An administrator for a health care region is considering 5 new programs that are vying for funding. The programs are listed in the Table and are rank ordered by the ratio of incremental cost per incremental quality-adjusted life-years (QALYs) shown in the first column. A QALY is a unit that incorporates changes both in length and quality of life. Therefore, the cost per QALY ratios are estimates of the extra dollars required to achieve 1 extra QALY. The second and third columns show the numerators and denominators of those ratios. The fourth column shows the estimated number of patients in the region that will require each therapy. The fifth column indicates the total dollars the region would have to spend, including all of the downstream costs and benefits, to implement the program each year for all patients (the product of columns 2 and 4). The last column is the total number of incremental QALYs that could be expected per year from fully funding the program (the product of columns 3 and 4).

If the societal goal is to maximize the number of QALYs that can be achieved, a rational person would first invest in programs that require the smallest number of dollars to achieve a unit of improved outcome (QALY), and therefore the influenza immunization program would be at the top of the list. However, a cost-effectiveness analysis requires one more piece of information: How much money does the administrator have to put into these programs? If the total budget (in this example) is $1 million, which programs would the administrator fund? Clearly, the first 3 programs have lower incremental cost per QALY ratios than the last 2 programs and they can be fully funded by spending $895 000 (the sum of the data in column 5 for those first 3 programs), resulting in $105 000 left over. The next program, the new chemotherapy for breast cancer, would require $50 million to fully fund, and therefore the $105 000 will be expended after only 2 patients. The total number of incremental QALYs gained from this allocation would be maximized in this way at approximately 37 QALYs (the sum of the numbers in column 6 for the first 3 programs plus 1.67 extra QALYs for funding 2 patients in the fifth program). No other allocation of the resources across the programs listed would yield a higher total number of QALYs gained for the region from a budget of $1 million.

Everyone who has used cost-effectiveness information in the decision making process knows that this kind of analysis never occurs. Instead, individual cost-effectiveness ratios are estimated for a single program, not a group of programs vying for a fixed budget. These single ratios are then compared with arbitrary thresholds below which programs are recommended for funding and above which they are not. This is not a true cost-effectiveness analysis. Using cost-effectiveness ratios in this way may lead to inappropriate decisions, because funding may not be provided for therapies that have more economically attractive ratios but have not been considered when the decision maker looks at 1 program in isolation.^{6 - 7} Simply adding all programs with known ratios below a threshold will inevitably lead to a never-ending increase in the health care budget, because very few new therapies are cost-saving or cost-neutral. In addition, one speculation is that the identification of an arbitrary economically attractive or “cost-effective” ratio (eg, $20 000 per QALY)⁸ may have the effect of encouraging drug companies to charge a price that achieves that ratio, even if they could make a profit at a lower price.

In addition to the problems of looking at cost-effectiveness ratios individually, interpreting those ratios can be difficult for clinicians and decision makers. It is not easy to understand what a QALY is. The measures of patient preferences (utility) are complex, debatable, and can vary widely depending on methodology. What does an average gain of a half a year of life expectancy mean?^{9 - 10} How should the various measures of quality of life be implemented? On the other hand, clinicians and decision makers are more familiar with studies that measure differences in risk (either absolute or proportionate differences), and they know what preventing 1 myocardial infarction or death means.

Clinicians treating individual patients may have different perspectives than those perspectives used in available cost-effectiveness analyses (they may not care about the effects on third-party payers or other patients). In those cases, the reported cost-effectiveness ratio perspective may be irrelevant to them in their role as patient advocate. However, those same clinicians may “wear a different hat” in their professional lives when they perform other functions, such as serving on hospital committtees or public advisory committees that allocate scarce resources. In those cases, the population or group perspective is appropriate to them.

Most cost-effectiveness ratios are estimated by extrapolating data from clinical results via the use of models (for example, estimating the effect of a statin on mortality over a patient's lifetime based on information from a 5-year trial, or extrapolating from a drug's effect on a surrogate marker such as tumor shrinkage on computed tomographic scan to a change in mortality). Because complex models require large numbers of assumptions (including generalizing trial results to other patient groups), they are difficult to understand and highly susceptible to both error and introduction of bias. In many cases, the incremental cost-effectiveness ratio is made up of a tiny difference in incremental cost divided by a tiny difference in life expectancy (eg, the immunization program in the Table). As such, small changes in assumptions can lead to wide variations in these ratios. Also, the vast majority of cost-effectiveness estimates for pharmaceutical products are produced or funded by the companies who have a vested interest in showing that their products are economically attractive.^{11 - 12}

QALYs and complex models in cost-effectiveness analyses can be useful, especially if the results of sensitivity analyses all point in the same direction (that the drug is either economically attractive or economically unattractive). However, the information used to generate these sophisticated but difficult to understand ratios can be made more useful to most clinicians and decision makers through an analytic approach known as a cost-consequence analysis.^{13 - 14} This approach involves displaying the differences between the new technology and the existing one in a disaggregrated fashion. For example, a cost-consequence analysis for an influenza vaccine would produce a vector of outcomes and costs including the difference in the incidence of influenza in immunized and nonimmunized groups, difference in the incidence of adverse effects such as muscle aches or Guillain-Barre syndrome between groups, the costs of the immunization and costs for treating its adverse effects (treatment of muscle aches and Guillan-Barre syndrome), and the cost savings from preventing cases of influenza. Because the units of measurement are concrete and familiar to clinicians, this approach may be easier to interpret than cost per QALY ratios. A downside of the cost-consequence approach is that it can be difficult for the decision maker to integrate the information if the therapy is associated with multiple benefits and harms, and if the benefits and harms are so disparate that they cannot be directly compared. Many numbers need to be considered in a cost-consequence approach rather than the single number provided by a cost per QALY ratio. Cost per QALY ratios are more easily comparable across programs. Thus, the 2 approaches are complementary. In some ways, the cost-consequence approach takes the decision maker “behind the scenes” of the cost per QALY ratio, showing which variables drive both the numerator and denominator.

Anyone who has been involved in drug reimbursement decision making knows that many factors in addition to the cost-effectiveness ratio affect the ultimate decision. For instance, available resources constrain funding options, although in reality the budgets of publicly funded drug plans in the developed world are not fixed but have increased markedly during the last 2 decades.¹⁵ The drugs for some extremely rare diseases such as Gaucher disease are very expensive per treatment, but some jurisdictions have decided to pay for them even though those drugs may not be economically attractive investments, because the number of patients with that disease is very small and therefore the overall budgetary impact will be small. On the other hand, when a disease is common, cost-effectiveness information may be considered carefully even if the cost per patient is small because the total budget impact is very large.

Some decision makers value a drug that decreases mortality more than one that does not, even if the cost-effectiveness ratios are identical. Decision makers may decide not to fund a drug that they think is economically attractive in a small subgroup of patients because they are convinced that it will be used in a much wider population in which it is not economically attractive, and the budget implications would be huge. Moreover, some investments in health care are undertaken for distributional reasons (to benefit a particular group in society), even if these interventions are not an efficient use of resources.

A full cost-effectiveness analysis requires allocating a fixed budget across all competing programs and is specifically aimed at maximizing an outcome variable subject to budgetary constraints. Although theoretically the correct approach, conducting a full cost-effectiveness analysis on every new health care technology is clearly impossible. Examination of a single ratio is a cost-effectiveness measurement (not analysis) that provides insight into the relative economic attractiveness of the technology compared with an arbitrary threshold, but does not maximize the benefits that can be gained from a limited budget. It also does not incorporate some societal values or budget impact. However, this information is still a useful part of the overall decision making process for policy makers and is made more understandable by using the more disaggregated cost-consequence approach.