Essential Elements of a Technology and Outcomes Assessment Initiative

The mismatch between US health expenditures and the resources devoted to learning which health interventions are most effective is both striking and unwise. Each year US individuals spend more than $2 trillion on health care.¹ More than $100 billion is spent for research and development and for regulatory approval of new technologies. Yet total spending on technology assessment almost certainly falls short of $1 billion per year—0.05% of all US health care spending.

Some of the $2 trillion in health care expenditures buys services of little or no value. This waste has been attributed to misleading advertisements, media hype, misguided state and federal mandates, fear of malpractice litigation, misaligned reimbursement incentives, and generous insurance that encourages patients to ignore the cost of services.^{2 - 4} Efforts to curb the inappropriate use of medical technologies, however, can have only limited success unless they address the paucity of reliable information about their benefits, cost, and value.

For decades, calls for more systematic assessment of medical technologies and outcomes have gone unheeded.^{5 - 7} Recently, however, federal legislators and officials have recognized that better information is imperative. The Medicare Prescription Drug, Improvement, and Modernization Act of 2003 mandated research on “outcomes, comparative clinical effectiveness, and appropriateness of health care.”⁸ The former Medicare administrator, Gail Wilensky, has described alternative structures for a technology assessment organization.⁹ Representatives, senators, and presidential candidates have supported legislation to fund comparative effectiveness initiatives.¹⁰

Renewed interest in technology and outcomes assessment efforts can be traced to several factors: disillusionment with traditional cost-containment approaches, deepening anxiety about the safety and effectiveness of drugs and medical care, recognition that little is known about the optimal use of existing diagnostic procedures and treatments, and the explosion in health care expenditures anticipated as baby boomers age. By 2015, the number of US individuals in their 60s—a decade of heavy use of medical care—will increase by nearly 50%. Simultaneously, because of scientific advances, many new technologies will enter clinical practice. The combination of new technologies and greater use of older medical interventions are the fundamental drivers of increasing health care costs.

Increasing health care costs have induced employers and insurance companies to shift more financial responsibility onto individuals through “consumer-directed” health plans and health savings accounts.¹¹ In addition, private health plans, Medicare, and Medicaid are likely to urge hospitals and clinicians to become agents of cost control. Essential to these efforts to enhance quality and lower costs is comprehensive, objective information about the absolute and relative costs and benefits of medical interventions.

Technology assessment in the United States has been hampered by pressure and limited resources. In the early 1990s, key federal agencies dedicated to technology assessment, such as the Congressional Office of Technology Assessment, were eliminated. Efforts by other federal agencies are fragmented and underfunded. The Department of Veterans Affairs, the National Institutes of Health, and the Centers for Medicare & Medicaid Services have little money for technology assessment.¹²

The mission of the Agency for Healthcare Policy and Research (AHCPR) was technology and outcomes evaluation. But, in 1994, when the AHCPR sponsored research showing that there was inadequate evidence to support commonly performed back operations, its funding was almost eliminated at the behest of disgruntled orthopedic and neurosurgeons and congressional critics of the Clinton health plan.¹³ While AHCPR survived, it was chastened. Its name was changed to the Agency for Healthcare Research and Quality and it has generally avoided controversial issues. Most importantly, little of its small budget—$320 million—is dedicated to evaluative research.¹²

State and private technology evaluation activities supplement federal efforts. In 1985, the Blue Cross and Blue Shield Association established the Technology Evaluation Center “for assessing medical technologies through comprehensive reviews of clinical evidence.”¹⁴ The Drug Effectiveness Review Project “is a collaboration of organizations [including 13 states] that have joined together to obtain the best available evidence on effectiveness and safety comparisons between drugs in the same class, and to apply the information to public policy and decision making” especially for Medicaid coverage.¹⁵ Private corporations provide similar information for purchasers. Physician specialty societies undertake increasingly sophisticated medical technology assessments and issue rigorous guidelines.

While commendable, these efforts are not equal to the problem. Their sponsors understandably focus on their own needs and priorities, which are largely uncoordinated and far from comprehensive. That is to be expected. Technology evaluations are a public good—they can benefit everyone, not only the organizations that bear the costs—creating disincentives for groups to invest in them.

Technology evaluations in health care can provoke controversy, anger, and hostility. A suggestion that a popular or expensive treatment is minimally effective or lacks data on long-term risks could be inimical to the interests of manufacturers, advocacy organizations, physician groups, or other groups, and will be received accordingly. To avoid political opposition, any agency concerned about its future might eschew analysis of topics that affect powerful companies or a large number of patients or clinicians and about which there is considerable uncertainty. In other words, it might avoid the very questions that most need answering.

To mitigate such concerns and facilitate the creation of objective information, any new technology assessment initiative must include 6 features: administrative independence; dedicated funding; production of objective and timely research; use of reliable methods; widespread dissemination; and a governance and organizational structure that lend it legitimacy.^{16 - 18}

Administrative Independence. Any technology and outcomes assessment initiative must balance accountability with the ability to pursue the long-term good of the public without inappropriate interference. The Federal Reserve Board is the preeminent model for such administrative independence.¹⁹ It conducts monetary operations and is often considered to be the federal agency with the most significant influence on the economy. Because it creates winners and losers, the Fed's decisions are inevitably controversial. Yet the Fed generally avoids the perception of favoritism.

What generates the Fed's independence? It is a semi-autonomous agency whose leaders are appointed for multi-year terms and cannot be removed at will; its staff are highly trained professionals who conduct independent, objective research to inform decisions; and its leaders regularly brief Congress.¹⁹ These characteristics are essential for a technology assessment initiative.

Dedicated Funding. Annual congressional appropriations, which determine the budgets for most federal agencies, are discretionary. Such funding makes agencies vulnerable to political retaliation whenever they issue controversial decisions.¹³ Conversely, the Fed does not depend on annual congressional appropriations.¹⁹ Similar dedicated funding is necessary to ensure that a program on technology and outcomes assessment could pursue research without fear of intimidation by powerful interest groups.

Funding obtained by imposing a fee on all health expenditures would offer not only stability but fairness, placing the cost for such an initiative on the beneficiaries of its work. Such a fee could be imposed only on health expenditures that are not subject to other taxes—employer-based insurance, Medicare and Medicaid benefits.

Moreover, a substantial funding commitment is needed to conduct a comprehensive set of rigorous assessments rapidly, and to be able to undertake original research and clinical trials. Britain's National Institute of Health and Clinical Effectiveness (NICE) is often lauded as a model of rigorous evaluation of technologies but is also criticized for its slow pace.¹⁸ In part, this is a consequence of NICE's limited size and budget—little more than 200 employees with a budget of just over $50 million. High-quality work can be done quickly only if the resources equal the task.

High-Impact Research. A credible technology and outcomes assessment initiative must have a well-defined mission: to assess the effectiveness, comparative effectiveness, cost, and cost-effectiveness of drugs, devices, diagnostic tests, medical practices, and procedures as actually implemented in the real world. The technologies being evaluated should be commonly used, of high individual or aggregate cost, subject to rapid change, or for which there are many alternatives and substantial uncertainty about which intervention should be used for which patient population. Topics that might be pursued include the best treatments for metastatic colorectal cancer and multiple sclerosis.

Any initiative should systematically and comprehensively assemble and analyze published and unpublished data, including population and clinical databases. Assessing the overall effect of different care processes as actually practiced also will be important. However, it will often be necessary to sponsor clinical trials and other types of research to generate new data for evaluations.

Trustworthy Methods. A permanent advisory board of distinguished methodologists is necessary to ensure the adherence to validated research methods and dissemination of objective results. A methodology advisory board would be able to resolve methodological controversies and oversee the refinement and development of new methods when appropriate.

Dissemination. Effective communication—of both cost and effectiveness information—is necessary to ensure the widespread and appropriate implementation of the results of technology and outcomes evaluations.^{15 ,17} The initiative must integrate diverse evaluations and communicate well with professional stakeholders, industry, physicians, and the general public. This requires the development of a standard reporting format for effectiveness evaluations, and the implementation of a formal review process before the final release of official reports. The review should include both internal evaluations and external commentaries.

In Britain, the results of NICE evaluations are binding on the National Health Service.¹⁸ In the current US health care system, binding coverage or medical necessity determinations from a new assessment initiative are neither feasible nor desirable. However, technology and outcomes assessments must directly address the key questions faced by government payers, such as the Centers for Medicare & Medicaid Services, health plans, and professional societies. The evaluations will be particularly important because they are objective and authoritative, and are not produced by a body with direct financial interest in the findings. A critical test is whether practices consistent with the evaluations are sustained as standard of care in litigation.

Legitimacy. Critical to ensuring independence, objectivity, relevance, wide dissemination, and especially legitimacy of the process is a permanent stakeholder advisory board that includes representatives of patients, insurers, employers, physicians, other clinicians, and federal agencies, as well as drug and device manufacturers. Important stakeholders must be engaged in selecting technologies for evaluation, designing studies, and interpreting and disseminating results. Having key stakeholders involved in a transparent process, even one that may generate research results contrary to their interests, will foster greater support for the process, methods, and results.

Manufacturers of medical technologies, along with many physicians, frequently criticize systematic technology assessment initiatives as a barrier to medical innovation. Their concerns often find expression in rhetoric that conflates new with innovative and latest with best.²⁰ However, novelty cannot be equated with benefit. An intervention's value resides in its ability to reduce mortality, morbidity, or save money, not in its unique mechanism of action. What is needed is better information on whether new tests and treatments really do improve health, how the improvement compares with the effects of currently available tests and treatments, and at what incremental cost.

Better information about effectiveness and costs will almost certainly redirect manufacturers' research and development activities. But redirection is not restriction. New interventions that offer substantial value will be rewarded with high demand and prices commensurate with their benefits—providing strong incentives for research and development. Conversely, new products that offer no or only incremental benefits will not command high prices. In medical care, as in other industries, new products that cannot prove their worth should not be assured of market success. Those that can should be rewarded generously.

A new technology assessment initiative built on administrative independence, dedicated funding, reliable research, trustworthy methods, wide dissemination, and legitimacy will offer a solid foundation for efforts to balance the benefits of medical technologies and the costs that result from their adoption. But information alone will not be sufficient. Information must be tied to appropriate infrastructure and financial incentives to affect medical practice. Health plans need appropriate incentives to use the information in their coverage decisions. Hospitals and physicians will need incentives to use the information in their treatment decisions. Simultaneously, evaluative research can guide incentives, insurance benefits, and the organization of care, ensuring that efforts to control costs and improve care are firmly grounded in the best evidence. In an era of increasing costs and growing complexity of care, few health initiatives are as important as a substantial program in the evaluation of medical technology and outcomes.