Using Effectiveness and Cost-effectiveness to Make Drug Coverage Decisions:  A Comparison of Britain, Australia, and Canada FREE

Expenditures on pharmaceuticals are the fastest growing sector within health care in developed countries, including Canada,^1- 2 the United Kingdom,³ Australia,⁴ and the United States,^5- 6 where federal expenditures for Part D of Medicare and Medicaid are projected to reach $4299 billion cumulatively from 2010 to 2014.⁷ In an attempt to control expenditures and to assess the value of new drugs, many countries, including Britain (National Institute for Health and Clinical Excellence [NICE]),^8- 11 Australia (Pharmaceutical Benefits Advisory Committee [PBAC]),^12- 14 and, most recently, Canada (Common Drug Review [CDR]^15- 16), have established agencies to determine whether new pharmaceutical treatments should be listed in public formularies. The US Department of Veterans Affairs has also established a national drug review process and formulary to reduce geographic variability of access to pharmaceuticals and reduce drug acquisition costs.^6,17

Concerns over rising health care costs have sparked debate within the United States about different ways health care costs might be controlled while maintaining high-quality health care.^18- 21 In 2008, US legislation was introduced (Senate bill S 3408) to create a public-private comparative effectiveness institute (the Health Care Comparative Effectiveness Research Institute).^22- 23 Creation of this institute, as well as whether such an agency would include cost-effectiveness as an additional criterion^20- 21 to comparative effectiveness when considering reimbursement of pharmaceuticals, remains a subject of debate in the United States. NICE, PBAC, and CDR have included cost-effectiveness as part of drug coverage decisions,^12,16,24 whereas drug reimbursement decisions within publicly funded health care (Medicare and Medicaid) in the United States largely exclude consideration of cost and cost-effectiveness at present.

We describe the key issues in evidence-based coverage facing 3 jurisdictions that use effectiveness and cost-effectiveness data in evidence-based coverage of pharmaceuticals. Using a retrospective analysis of past decisions, we describe how these committees use evidence on effectiveness and cost-effectiveness (including any barriers to such use), and what additional factors have influenced decisions, and explore how these issues may be associated with listing decisions. We also consider 3 drugs that highlight these key issues across agencies and report in detail these deliberations. To frame discussion within the US context, we also compare qualitatively these 3 agencies with the US Veterans Affairs and Medicare drug insurance plans.

Drug funding decisions were reviewed for the CDR, NICE, and PBAC. These agencies were selected because they consider both effectiveness and cost-effectiveness, publish information in English on reimbursement decisions, and are similar in their underlying populations and public pharmaceutical insurance coverage. There are some important differences in process and remit between the agencies, which are described here and in eTable 1). To frame discussion within the US context, we also describe the US Veterans Affairs and Medicare drug insurance plans, 2 national programs that provide insurance coverage for drugs (eText).

Canada has universal, publicly funded health care,^15,25- 26 although the availability of publicly funded drug insurance varies across provinces and territories (eTable 1). In 2002, federal, provincial, and territorial ministers of health, concerned with differences in coverage of prescription medications across public formularies and duplication of effort in reviewing new medicines, established the CDR.¹⁵ The CDR provides recommendations for listing of new drugs to the 18 participating federal, provincial, and territorial publicly funded drug plans.¹⁵ Manufacturers submit their medication to the CDR, with submissions being considered by the Canadian Expert Drug Advisory Committee, an independent committee composed of 11 professional members (physicians, pharmacologists, and members with expertise in health services research and health economics) and, since 2006, 2 members of the public. The committee considers the safety and clinical effectiveness of the drug in appropriate populations as well as cost-effectiveness, both in comparison with current accepted therapy.² Participating drug plans are not required to follow the committee's recommendations because they also must consider their own health care priorities, available resources, and the precedence of previous formulary decisions. Nonetheless, drug plan decisions are in agreement with the committee's recommendations about 90% of the time.²

Britain has publicly funded health care (the National Health Service [NHS]) and a small parallel private system. Drugs are provided to all British citizens through the NHS, although for drugs that have not been reviewed by NICE, coverage decisions are made at local levels and may vary across regions. NICE issues guidance on selected clinical topics as chosen by the British government¹⁰ and often a class of drugs is reviewed in 1 review (multiple technology assessment). In addition to an independently conducted systematic review and economic evaluation, the manufacturer submits clinical and economic evidence, and other interested parties (patients, carers, and advocacy groups) may submit evidence and experience profiles. In 2006 a process that critiques manufacturer-submitted evidence was introduced as a more streamlined alternative for single drugs or technologies (single technology assessment). The evidence is then discussed by a committee of 33 members selected from the NHS, patients, academia, and industry. Recommendations are legally binding in England and Wales and local resources must be reorganized to implement NICE guidance.

Australia also has a national publicly funded health care system and a parallel private system. Drug insurance is provided to all Australians through the Pharmaceutical Benefits Scheme. Drug companies submit their drug for funding consideration by the PBAC, an independent statutory body established in 1953 to make recommendations and give advice to the minister about which drugs should be made available as pharmaceutical benefits. The PBAC considers effectiveness and cost in comparison with alternative therapies; formal consideration of cost-effectiveness began in January 1993.^10,27 The PBAC consists of 18 members appointed by the government, including physicians, health professionals, 1 health economist, and 1 consumer representative. Recommendations are acted on by the minister for health, who cannot list a drug on the Pharmaceutical Benefits Scheme unless the PBAC gives a positive recommendation. In the case of a negative recommendation, resubmissions with new evidence or lower prices are permitted, as are submissions to broaden the specified indication.

All publicly available documents as of December 31, 2008, were included. Given that each committee adopted transparency measures at different time points, the available data sets for the PBAC, NICE, and CDR begin in July 2005, February 2001, and January 2004, respectively. During this time frame, the publicly available documents reviewed for NICE and PBAC consistently provided an overview of the clinical and economic reports considered by the committee, as well as a summary of the committee's perspective on the drug. Since 2007, this level of detail has also been publicly available for the CDR. However, prior to 2007, the publicly available information for the CDR was often limited to the funding recommendation and a review of the major clinical and economic reasons for this recommendation. To ensure that a similar level of detail was available for drugs reviewed by each agency, confidential summaries of committee discussions were also reviewed for CDR submissions, along with the clinical and economic reports. For NICE, all technology appraisals including drugs were reviewed. Each drug and disease indication combination was considered a unique observation.

The primary outcome considered was the final decisions or recommendations of the committees available within the study time frame. The 3 categories were list, list with criteria, and do not list. The list rate was calculated as the number of list or list-with-criteria recommendations divided by the number of submissions. Basic drug information was collected: generic drug name, indication sought, date of submission, and whether the drug had been considered for the same indication previously. To define the clinical context of the decision, and to be consistent with previous research,¹² variables were developed to indicate if the indication for the drug was life threatening (less than 50% 5-year mean survival rate), if the goal of treatment was life extension or quality-of-life improvement, and if other treatment options were available for the condition.

We collected information on the primary end point used in the supportive clinical studies and categorized end points as clinical end points (eg, mortality or occurrence of a myocardial infarction), clinical scales (eg, American College of Rheumatology 20% improvement criteria for rheumatoid arthritis²⁸), or surrogate end points (eg, changes in blood pressure or parathyroid hormone level²⁹).^30- 31 For surrogate end points, we also determined whether the committee felt the surrogate was a valid predictor of changes in the relevant clinical end point.²⁹ The advantages and disadvantages of accepting clinical efficacy on the basis of surrogate end points have been reviewed in detail,^29- 33 but surrogate end points, even ones that have not been formally validated, continue to be accepted as proof of efficacy by regulatory agencies.^31- 32

Based on prior work, we expected that certain issues would create problems for review committees and focused our data collection on these variables. We defined these key issues as clinical and economic uncertainty (both categorized as none, some, and considerable uncertainty). Considerable clinical uncertainty was present if efficacy data were based on nonrandomized clinical trials or if the supportive randomized trials used what the committees felt to be an inappropriate comparator or an unvalidated surrogate end point. Considerable economic uncertainty was present if there were major flaws in structure of the economic model, if assumptions used with respect to clinical efficacy or effectiveness were substantively different from the committee's view, or if there was inappropriate mapping of the clinical evidence to a final end-point quality of life. In both cases, the definition of uncertainty was also judged on whether the summary documents (or the summary of committee discussions in the case of the CDR) noted considerable clinical or economic certainty as an issue during committee deliberation.

With respect to the supportive clinical trials, we also considered the relevance of the clinical trial evidence, including the magnitude of the clinical effect (ie, effect size). We also considered socially relevant characteristics of the patient group (ie, unmet need in disadvantaged population, severity of condition [eg, life threatening]).

We also considered when estimating the cost per quality-adjusted life-year (QALY) was seen as necessary by the decision maker. For example, when the committee judged that clinical efficacy was “equivalent” based on head-to-head studies, only a comparison of cost was required and the calculation of an incremental cost per QALY was not required for the listing decision made. We recorded the base-case cost per QALY submitted by the manufacturer for the indication in which coverage was being requested as well as committee's estimate of the cost per QALY (ie, the cost-per-QALY estimate that was felt to be more accurate by the expert committees based on review of sensitivity analyses or reanalysis of the model conducted by the agencies). All subjective variables were extracted in duplicate and disagreements resolved by consensus.

To identify similarities and differences in the characteristics of submissions considered by each country, submission characteristics were compared using χ² tests (2-sided significance of .05). Based on previous work and publications,¹² we identified key issues that might impact the likelihood of funding a treatment and described whether these key issues were associated with the likelihood of listing. Acknowledging that there were differences in the drugs assessed within the 3 countries, we also determined the listing rates for the subgroup of drugs that were considered in common across agencies. For those drugs for which there was no major uncertainty on cost-effectiveness, we examined whether there was a threshold range of cost-effectiveness (cost per QALY) above which the committee was unlikely to fund a drug. The existence of a threshold value for drugs where there was a clinical benefit was estimated graphically by ranking the committee's estimate of the cost per QALY from low to high. SAS version 9.0 (SAS Institute Inc, Cary, North Carolina) was used for all analyses.

To illustrate the similarities and differences between the different agencies, we selected 3 drugs that had been reviewed by all of the agencies. Each serves to highlight different key issues that are commonly encountered in the evidence review process, as well as how review processes may be associated with drug reimbursement decisions. We selected insulin glargine for diabetes mellitus, ranibizumab for age-related macular degeneration, and teriparatide for osteoporosis.

During the study period, the CDR considered 121 submissions (114 submissions, 7 resubmissions) while PBAC considered 282 submissions (207 submissions, 75 resubmissions). NICE completed 144 technology appraisals: of those, 47 were excluded because they did not consider drugs, leaving 97 submissions, which resulted in 199 drug appraisals (184 submissions, 15 resubmissions) (eFigure 1). The Table presents the baseline characteristics of all submissions to each agency. Of note, resubmissions were common for the PBAC, with 75 of 282 submissions (26.6%) being resubmissions of previously rejected drugs, often resubmitted for a narrower clinical indication and sometimes at a lower price.

More than 40% of all submissions reviewed by the CDR and PBAC were associated with considerable clinical uncertainty (Table), which was more common than for submissions to NICE (54/199; 27.3%; P = .009), perhaps reflecting the fact that NICE typically evaluates classes of drugs that have had regulatory approval for longer. For the CDR and PBAC, 26 of 121 submissions (21.7%) and 81 of 282 submissions (28.8%) reported use of a nonrandomized study design or an inappropriate comparator in a randomized controlled trial. Surrogate end points were often the main end points in the clinical studies that formed the basis of the submissions to each of the agencies (Table).

A cost-per-QALY estimate was more commonly required in the evaluation of drugs considered by NICE (Table). When a cost-per-QALY estimate was required for the decision, considerable economic uncertainty existed for 46.1% (90/192), 58.2% (118/203), and 55.7% (41/73) of submissions considered by NICE, PBAC, and CDR, respectively. Of note, considerable economic uncertainty was based exclusively on the presence of considerable clinical uncertainty in 57 of 245 cases (23.4%) where economic uncertainty existed, underscoring the central role of good-quality clinical evidence in decision making.

NICE recommended 87.4% (174/199) of submissions for listing compared with 49.6% (60/121) for the CDR and 54.3% (153/282) for the PBAC. The list rates for CDR and PBAC were lower when there was considerable clinical or economic uncertainty (eTable 2). In addition, the use of a relevant clinical end point was associated with a higher probability of recommending coverage for the CDR and PBAC. Alternatively, listing decisions by NICE did not appear to be associated with the existence of significant clinical or economic uncertainty, possibly reflecting its approach of identifying subgroups for which the uncertainty might be lower and the cost per QALY more acceptable. There did not appear to be an association between the listing decision and whether the underlying condition was life threatening.

Where the calculation of a cost per QALY was necessary for the decision made and the committee was confident in the evidence supporting the incremental cost per QALY, the committees' decisions appeared consistent with a cost-effectiveness framework (eFigure 2). There was some evidence of a threshold range for each agency. However, it is clear that even when there was confidence in the cost per QALY, other factors appeared to result in decisions to not recommend coverage for a drug with a cost per QALY below these ranges and, conversely, to recommend funding for some drugs above these ranges.

For 13 submissions (4, 8, and 1 for the CDR, NICE, and PBAC, respectively), the drug was rejected for the patient population submitted within the economic evaluation but was recommended for listing in a more restricted patient subgroup in whom a cost per QALY was not available—presumably in an attempt to improve the efficiency of drug use and reduce expenditures (eFigure 2). For the 66 submissions in which committees noted significant economic uncertainty (7, 6, and 53 at the CDR, NICE, and PBAC, respectively), the listing rates were 28.6% (2/7), 66.6% (4/6), and 3.8% (2/53), respectively (eFigure 2).

In 91 submissions, the same drug was reviewed for the same indication by more than 1 of the agencies (eFigure 3). There was poor agreement between funding recommendations made by the CDR and PBAC (κ = 0.27) and NICE and PBAC (κ = 0.13) and moderate agreement between the CDR and NICE (κ = 0.55). Consistent with the trend observed overall, for this subset of common drugs NICE was more likely to recommend funding (eFigure 3). For the drugs considered by all 3 agencies (n = 19), the list rates were 52.6% (10/19), 84.2% (16/19), and 73.6% (14/19) for the CDR, NICE, and PBAC, respectively.

We qualitatively analyzed the sets of common drugs to determine the most common reasons for the discrepant listing recommendations. The most common reasons included an apparent intention by NICE to find limited niches for drugs rather than recommending not to list (n = 7), the use of price negotiation by the PBAC to ensure cost-effectiveness (n = 3), and a different attitude to the proliferation of drugs within an established therapeutic class (n = 7). The CDR appeared reluctant to list subsequent similar drugs while the PBAC had a “cost-minimization policy,” which appeared to encourage proliferation by listing therapeutically similar drugs at the same (or lower) price as the originator, thus using competition to ultimately lower prices. In 4 cases, the committees appeared to base their decisions at least in part on a different estimate or valuation of cost-effectiveness data across the agencies. Not unexpectedly, in the absence of good-quality clinical trial data with respect to the best approach to drug sequencing in conditions where several different agents are available, the 3 agencies made discrepant recommendations regarding whether to provide an open listing, to list as a second-line agent after failure of less expensive therapies, or not to list (n = 4).

Insulin glargine (eTable 3) was recommended for listing in October 2002 by NICE in patients with type 1 diabetes and in a subset of type 2 patients on the basis of 13 randomized trials. In 2006, the CDR reviewed the results of 20 unblinded randomized trials (many of which were unpublished), noting variable results for overall and nocturnal hypoglycemia. Although the CDR felt that the use of insulin glargine may reduce the frequency of nocturnal hypoglycemia, it did not feel that these benefits justified the 3-fold cost and did not recommend listing. The PBAC rejected listing for insulin glargine on 5 separate occasions on the basis of clinical uncertainty resulting from reporting bias in the presented meta-analysis as well as unacceptable cost-effectiveness. On the fifth resubmission, and after extraordinary discussions with the manufacturer, the PBAC agreed to list insulin glargine as an unrestricted benefit based on acceptable cost-effectiveness at a new proposed confidential price. Thus, although each of the committees agreed that insulin glargine offered small incremental benefits over insulin NPH, all felt that unrestricted use at the price submitted was not cost-effective. In response to this, NICE listed insulin glargine for patients with type I diabetes and identified a small niche of type II patients who might be more likely to benefit, while the PBAC was able to negotiate a price that offered reasonable cost-effectiveness, an approach outside of the scope of the CDR's mandate.

Each of the agencies recommended listing for ranibizumab for age-related macular degeneration (eTable 4). The clinical evidence was from well-performed randomized trials using the appropriate comparator and demonstrated that ranibizumab reduced the incidence of blindness in patients with wet macular degeneration. Despite the high cost, given the significant negative implications of blindness, the use of this agent was associated with a cost per QALY gained in the range of other funded interventions. Given the very high cost of this agent, each agency recommended product-listing agreements to ensure cost-effective use of this agent and to ensure a maximal expenditure per patient, thus shifting some of the financial risk associated with prolonged use of this medication to the manufacturer.

Each of the committees agreed that teriparatide (eTable 5) had been shown to reduce the incidence of vertebral and nonvertebral fractures in comparison with placebo but felt that bisphosphonates would have been a more appropriate comparator within randomized trials. The CDR and PBAC were also concerned that there were no clinical trials in patients who did not tolerate or who continued to experience fractures despite bisphosphonates, a subgroup of patients who might benefit from an additional treatment option. Given significant clinical uncertainty, the high cost, and resultant unacceptable cost-effectiveness, the CDR and PBAC did not recommend listing. NICE felt that the use of this agent might be cost-effective in a small subgroup of patients with severe osteoporosis for whom bisphosphonates had failed and listed it for this small subset of patients.

Given Medicare's regulations, it is expected that these 3 medications would be covered for Medicare beneficiaries. Of note, the Medicare Evidence Development and Coverage Advisory Committee is planning to review the data in support of ranibizumab to ensure optimal use.³⁴ Veterans Affairs reimburses the cost of insulin glargine, ranibizumab, and teriparatide with specific restrictions for each medication by clinical indication or specialty. Detailed reasons for recommendations were not available publicly.

NICE, CDR, and PBAC all have expert committees that consider comparative effectiveness and cost-effectiveness, among other factors, in their listing decisions. We noted differences in listing decisions by these committees, even for the small subgroup of drugs that were assessed by all agencies. This is not surprising given that the mandates and processes of each of the committees differed. Moreover, the differences in listing decisions often appeared less about the interpretation of the clinical or economic evidence and more about differences in agency processes that may reflect differences in risk attitudes, including the willingness to fund drugs with a given quality of evidence on effectiveness and cost-effectiveness and the role of competition.

The Australian system allows sponsors to resubmit an application an unlimited number of times with variation in requested price, indication, and associated evidence. If we consider only the last submission for drugs that were resubmitted after rejection for a particular indication, the listing rate for the PBAC increases to 62%, suggesting that resubmissions may impact listing decisions. As occurred for insulin glargine and teriparatide, however, the final acceptance often involved tighter restrictions in indication and a lower price. Revised submissions are unusual in Canada, given more stringent criteria around resubmissions and possibly the lack of a process to enable price negotiation. There is an increasing use of risk-sharing arrangements, particularly in Australia, to reduce uncertainty around both financial costs and cost-effectiveness.

Consistent with previous studies reporting problems with the quality of the evidence presented to reimbursement agencies,^14,35- 36 we noted ongoing issues with the quality and strength of the experimental evidence in support of the claim of a clinically meaningful effect. While all of the agencies noted problems with the quality and validity of economic evidence, each has adopted different approaches to handling this. NICE has sought independent economic analysis while the CDR has conducted its own sensitivity analysis of manufacturer models. The PBAC has adopted a structured approach to the presentation of clinical and economic evidence, emphasizing rigor in the steps needed to translate clinical trial data into evidence of cost-effectiveness.³⁷

Our study has limitations, including that the data set is based on publicly available data for NICE and PBAC. Although the public summary documents are thorough, there may be subtle issues that were not captured, particularly in the deliberation process. Another limitation is that there are surprisingly few common drugs across the 3 systems, making comparisons across committees less conclusive. In part, this reflects that NICE, unlike the other agencies, selects which drugs or drug classes to consider and when and that the CDR has not reviewed chemotherapy drugs for cancer since 2007. Finally, given the heterogeneity in both the drugs considered and the drug funding systems, a formal statistical analysis of the reasons for decisions was not possible. The results suggest that there are some differences in the way these jurisdictions use effectiveness and cost-effectiveness information in coverage decisions, but further research is needed to establish the cause of these differences.

Although our study does not provide direct evidence to inform the question of whether these agencies improve efficiency of care for populations, early work suggests that the system in Australia has reduced prices below those in comparable countries³⁸ without compromising health outcomes. Moreover, research suggests that establishment of the Veterans Affairs national formulary has achieved significant cost savings while ensuring access to a wide range of prescription drugs.^6,17

What can be learned from this study by the United States or other health care systems regarding pharmaceutical reimbursement? First, the existence of these 3 agencies confirms that it is feasible to establish an agency that considers comparative effectiveness in pharmaceutical reimbursement decisions. The successful establishment of the CDR, which operates in an environment of multiple payers, all with varying budgets, is particularly relevant to the United States. While cost-effectiveness is not required for all drugs, economic evidence is critical in some cases to provide information on comparative value for money.

Second, the differences that exist in the processes of these agencies confirm that they can be adapted to local health care circumstances. In fact, a key component of sustainability of these agencies appears to have been the ability of each committee to adapt to national decision-making needs.^39- 40

Third, a primary concern in the United States appears to be that the use of comparative effectiveness and cost-effectiveness would reduce choice in therapeutic options.^20- 21,23 As illustrated by ranibizumab, the use of cost-effectiveness in coverage decisions need not be an undue barrier to drug funding,^12,40 even for expensive medications, when there is robust evidence of effectiveness, at least in some patient subgroup, or where there are factors that appeal to the values of decision makers beyond the simple metric of cost and health gain.¹¹ Moreover, a system need not necessarily consider a simple dichotomous listing decision. Medications can be reimbursed in specific subgroups where they are felt to be cost-effective or can be listed with a higher co-payment if choice and access to therapy are valued highly. Information on cost-effectiveness could be used to inform the payment level (or co-payment) for a particular drug. In the end, however, in any health care system, coverage and pricing choices need to be made.²³ As Wilensky puts it:

Cost-effectiveness information should be an important consideration in setting reimbursement rates by public and private payers. If an intervention doesn't do more, why should a payer pay more for it? If it does do more, asking how much more and for what additional price becomes relevant. Payers will have to make difficult decisions, and different payers may make different decisions.²¹

In summary, our study demonstrates that comparative effectiveness and cost-effectiveness can be used by national reimbursement agencies, although several key issues exist, the most important of which is clinical uncertainty. Perhaps the main lesson from the experience of the 3 countries is that systematic, durable, and widely accepted decisions can be made using comparative effectiveness and cost-effectiveness, although it is evident that other information beyond these 2 criteria can be incorporated into decision making. Given that the number of expensive, targeted pharmaceuticals for cancer and other chronic conditions is increasing, pharmaceutical reimbursement will continue to be a key challenge to formularies in all countries.