Toward Evidence-Based Policy Making and Standardized Assessment of Health Policy Reform

In the United States, deficiencies in health care quality, value, and access are well documented.^{1 - 5} Recent trends such as pay-for-performance, increased patient cost-sharing, and state health insurance expansion programs may represent important reforms and even a “tipping point” for the US health care system.^{6 - 7} Nevertheless, experts have cautioned that not only could unintended consequences occur but that no systems are in place to ensure accountability among policy makers.^{6 ,8 - 9}

For example, a company switches its workers to high-deductible health plans. Because of increased out-of-pocket costs, a low-income employee with diabetes begins deferring care, resulting in worsened blood glucose control. Meanwhile, the employee's physician is paid more if a certain proportion of his or her patients with diabetes achieve glycated hemoglobin levels below 7%. Two outcomes are possible: the physician may continue to care for such patients and accept lower compensation or may choose to terminate the clinical relationship to avoid the financial penalty.¹⁰

In the first outcome, a quality-improvement system promoted by health plans and employers (pay-for-performance) penalizes the physician for a cost-control measure (high-deductible insurance) also created by health plans. In the second, the policies create a conflict of interest for the physician that induces gaming behavior. In both cases the patient's health worsens and the physician is held accountable for a situation beyond his or her control.

Are such scenarios playing out today? Despite high-deductible health plans and pay-for-performance having moved beyond their infancy, conclusive answers may not be available for years. Similarly, inadequate and untimely evaluation of state or national health insurance reform could occur. These patterns point to a larger problem in the US health care system: although structures exist to systematically evaluate epidemiologic trends, new drugs and medical procedures, and even physician performance, standardized assessment of major health policy change does not occur.⁸

Some have suggested that this policy-making approach is akin to “social experimentation.”^{11 - 12} Epstein equated pay-for-performance to an unproven and potentially risky medical intervention.⁶ Grudzen and Brook⁸ and Fisher⁹ called for more accountability in health policy implementation.

In this Commentary we propose a framework and practical steps for moving toward that goal. As policy makers debate strategies for improving quality and expanding health insurance coverage, the need for such a system has become urgent.

Assessment of new health policies is rarely systematic and typically is undertaken by a haphazard collection of the curious, concerned, or adequately funded. The objectivity of these investigators may be difficult to assess. Studies are usually retrospective and often include populations convenient from a sampling perspective rather than relevant to broader policy making. Determining if intervention and comparison groups have representative health care access or outcomes may be impossible. In addition, results often are not published for years after policy implementation and tend to describe use of health services rather than clinical outcomes. Thus, even if high-quality studies are performed, they may be difficult to interpret and apply.

Even though the concept of evidence-based decision making is widely accepted in the clinical world, the approach has not fully permeated health policy.¹² Legislation creating policies does not necessarily include “impact assessments,” commissioned studies do not use standardized measures, and outcomes examined may be of questionable relevance.¹³ Policy change therefore may be based on expert opinion, funding circumstances, or political sentiment rather than evidence of benefits or harm.

The lack of systematic and ongoing evaluations of new health policies has led to the discovery of unintended consequences years later. Capitation created widespread discontent in the 1990s and was advanced despite suggestions that the approach was inherently unethical.¹⁴

Adverse health outcomes also have been documented. A New Hampshire policy limiting the number of Medicaid-reimbursable drugs was found to be associated with increased nursing home admissions 10 years after implementation.¹⁵ A study of Medicare recipients revealed that women without supplemental insurance had substantially lower breast cancer screening rates relative to women with coverage.¹⁶ Ten years after Maine introduced a performance-based contracting system for high-priority substance abusers, an investigation demonstrated reduced treatment of the highest-severity patients.¹⁷ A recent study found that Medicare beneficiaries whose annual drug benefits were capped had higher mortality compared with those without capped benefits.¹⁸

It is unlikely that adverse consequences of policy innovations are a thing of the past. Performance-based physician compensation could compromise patient trust, physician professionalism, and access for vulnerable populations.^{9 - 10} Concerns exist that increased cost-sharing will adversely affect the poor and chronically ill by reducing appropriate health care utilization.^{8 ,19 - 20} State health insurance expansion programs likely will benefit many, but some have suggested that premiums and high deductibles will adversely affect persons with moderate incomes.^{21 - 22}

A system that maximizes the effectiveness and ethical characteristics of health policy reform would include 4 essential elements: (1) Review to ensure that the policy's fundamental precepts are ethical. While political and economic forces will undoubtedly continue to drive change, policy formulation could be accompanied by structured “ethical review” involving public debate, qualitative research, and expert opinion.^{11 ,21} (2) Targeted pilot projects or timely retrospective assessments to address benefits and harms for stakeholders. Key realms to be assessed would include health care quality, value, and equity. (3) Studies to determine if unintended consequences can be satisfactorily minimized. (4) Feedback systems to maintain acceptable outcomes after policy implementation.

To define benefits, harms, and “acceptable outcomes,” the development of measures and standards would be necessary. Because the primary purpose of reform should be improvement in health outcomes, this represents the most important measurement domain, particularly among the poor and chronically ill.

Measures currently being developed to gauge physician performance also may be appropriate for health policy assessment. However, such measures depend on patients' access to care, a realm substantially affected by cost-sharing and other health plan structures beyond a physician's control. Therefore, access-to-care standards also are needed to promote the provision of health insurance with structures that encourage appropriate utilization. While this may imply distributing larger shares of public funding to insuring the chronically ill or offering plans with income-based benefits, such outcomes may be considered just if achieved through a fair and transparent process requiring accountability.¹¹

The need to promote equity implies that standards for percentage of income spent on health care should be created. These would minimize unjust levels of cost-sharing and promote adequate access to care.

Structured health policy assessment raises complex issues regarding development of standards, the methodology of measurement, and mechanisms of implementation.

Developing standards requires key stakeholder involvement in selecting measures and assigning priority to policy trends, populations, and realms of care. An experienced entity such as the National Quality Forum could oversee the process. Good measures have been described as specific, measurable, achievable, realistic, and time-bound.²³ The Table lists examples of Health Employer Data and Information Set (HEDIS) measures that could be broadly used to monitor health policy trends and provides an example of a cost-sharing measure. The actual values of standards could be based on a nationally representative reference population or other approaches. For instance, systematic reviews of studies examining access, income spent on health care, and health outcomes in various populations could delineate a range of acceptable values. Expert opinion and public input also could be used to set benchmarks.

This approach would require determining the locus of performance oversight. Demonstration projects created by congressional legislation could continue to be studied by relevant government entities. For decentralized health insurance trends, the institutional review board system for clinical trials could serve as a prototype, with responsibility for reporting outcomes (and funding the effort) resting on health plans or states implementing the policy. Alternatively, monitoring could occur through an external body, modeling the Centers for Medicare & Medicaid Services' review of Medicare managed-care plans.²⁴

Creating adequate incentives for implementing measurement also would be necessary. States and legislators would find standards useful for gauging policy effects and for comparisons to other systems. Eventually, formal integration of standards into legislation could lead to structured monitoring.

Private insurers may be less willing to undergo assessment, though they would be hard-pressed to argue against its necessity. A stepwise approach initially could involve permitting voluntary self-assessment and public reporting. In a competitive marketplace, insurers would benefit from demonstrating success in promoting outcomes and access. For example, state health insurance expansion programs may preferentially include insurers achieving excellent outcomes among vulnerable populations. However, stronger incentives eventually may be needed.²⁵

Standardized evaluation of any system is associated with its own limitations, and health policy assessment would be no exception. For instance, there is risk of including easily measurable domains rather than those of greatest importance. Measurement also oversimplifies the complex factors associated with adequate health.³ Setting standards too high or low could make them ineffective. Even if set appropriately, measurement inherently prioritizes specific realms, potentially diverting resources from other important areas. In addition, systematic assessment is expensive, though likely cost-effective if health care value is improved. Measurement could induce health plans to forgo insuring the poor or chronically ill to increase performance. This could be avoided, at least in part, by using case-mix adjustable standards based on a reference population (Table).

These drawbacks are real but could be minimized by careful development. In addition, measurement would have multiple distinct benefits. Public and private insurers could integrate standards into existing quality improvement systems so that outcomes feed back to improve health plan structures and processes. Insurers and employers would retain freedom to implement creative, localized solutions to improving quality and equity. Key benefits promoted by modern health policies such as efficiency, competition, health plan flexibility, and member autonomy could persist.

The broad uptake of properly constructed standards would promote a safety net for vulnerable populations. Trivedi et al found that quality improved and disparities decreased in Medicare managed-care plans assessed with HEDIS measures.²⁴ Standards could focus attention on vulnerable patients and encourage policy strategies such as income-based deductibles or fully funded health savings accounts for the poor (Table). Insurance providers could tailor plans to maximize outcomes among specific populations or among members not meeting standards.

Case-mix adjustable standards could establish an objective basis for comparing policy innovations across states or health plans. Supporters and opponents of a policy would benefit. Supporters would use unmet standards to improve their favored approach, and opponents would use them to argue against the policy. Either way, improvement of the health care system could be hastened, and the United States might move toward evidence-based policy making.

A framework for the structured assessment of health policy reform is needed and should include development of standards in the realms of health outcomes, access, and income spent on health care as an initial step. Such an “accountability for policy making” approach could help enhance evidence-based policy comparisons, promote a safety net for vulnerable populations, and hasten quality improvement. Without it, assessment will continue to be arbitrary, and unintended consequences will go undetected.