Comparative Effectiveness—of What?: Title and subTitle BreakEvaluating Strategies to Improve Population Health

How much, then, should go for medical care and how much for other programs affecting health, such as pollution control, fluoridation of water, accident prevention and the like?There is no simple answer, partly because the question has rarely been explicitly asked.—Victor Fuchs¹

Significant comparative effectiveness research (CER) efforts should be dedicated to understanding the effectiveness of investments across broad determinants of health instead of focusing primarily within the health care domain alone. For instance, the obesity epidemic poses a threat to health outcomes, and costs related to obesity unquestionably arise from a combination of determinants, including medical care; health behaviors; social, economic, and cultural factors; characteristics of the physical and built environment; and genetics. However, limited evidence exists to guide public and private policy makers regarding investments across these determinants. Can CER be used to address such important questions?

Calls from policy experts over the past decade have now resulted in substantial resources for CER, including $1.1 billion in the 2009 American Recovery and Reinvestment Act as well as subsequent codification in Title VI, Subtitle D, of the 2010 Patient Protection and Affordable Care Act (PPACA), in which, in addition to medical care, “any other strategies or items being used in the treatment, management, and diagnosis of, or prevention of illness or injury in, individuals”² are under the purview of the CER agenda.

Even accepting that the language in the PPACA opens the door to studying a broader array of interventions, there will likely be a strong tendency for federal funding agencies to focus primarily in the medical care areas emphasized by the Federal Coordinating Council definition of CER as “ the conduct and synthesis of systematic research comparing different interventions and strategies to prevent, diagnose, treat and monitor health conditions. . . . [D]efined interventions compared may include medications, procedures, medical and assistive devices and technologies, behavioral change strategies, and delivery system interventions.”³

A call to extend the domain of CER⁴ did so by advocating compellingly that CER not be limited only to drug-drug comparisons but also should encompass nondrug clinical interventions. While the Department of Health and Human Services has not yet made public the entire portfolio of American Recovery and Reinvestment Act–supported projects, the following major categories were identified for Office of the Secretary research-related investments: behavioral economics and change (National Institutes of Health/Agency for Healthcare Research and Quality); delivery system (Agency for Healthcare Research and Quality); regionalized emergency care delivery (Assistant Secretary for Preparedness and Response); comparative effectiveness of chronic disease prevention (Centers for Disease Control and Prevention); centers of excellence for racial and ethnic minority–focused CER (Office of Minority Health/National Institutes of Health); and centers of excellence for persons with disabilities (Office on Disability).⁵

There is a critical need to examine clinical effectiveness and, ideally, cost-effectiveness. However, the need is even greater to examine the clinical and cost-effectiveness across the broader spectrum of population health domains, including the social and physical environments. Estimates of the importance of nonclinical determinants of health vary, and meaningful effects on significant segments of the population are likely. Significant methodological and empirical work is needed to generate the evidence base. For example, in the 1970s in Gary, Indiana, one of the few social determinant randomized trials found that a negative income tax was associated with increased birth weight, likely because of better maternal nutrition.⁶ A more recent example of the kind of research that merits support is that of Stringhini et al,⁷ who used British Whitehall II longitudinal data to shed more light on the relationship between individual behaviors and their social context in relation to health outcomes.

The need is great for more robust and relevant empirical work to make these estimates more precise so that policy makers can depend on them for investment choices. Limitations of data sets and methods make such policy-oriented population health analysis difficult across broad determinant categories, much less between and across specific programs and policies.⁸ Yet the “evidence base” offered by much clinical CER is often limited because of narrowly defined outcomes, lack of sample generalizability, and other shortcomings of clinical trials.

The complexity of establishing these relationships in population health is why Stoddart⁹ previously referred to such work as a “fantasy equation,” commenting that “ . . . at present we but vaguely understand the relative magnitude of the coefficients on the independent variables that would inform specific policies rather than broad directions, even if we are beginning to see the variables themselves more clearly.” But such cause-effect relationships do exist—large or small, positive or negative—waiting to be discovered. Policy makers seldom require firm causal relationships for policy decisions made in the public or private sector, so the job of population health research is to get as close to causal understanding as is possible to guide political or managerial efforts.

Current decision makers may be more open to a population health perspective. A prominent example is the recent Obama Administration initiative on childhood obesity. The presidential task force charged with devising strategies for “solving the problem of childhood obesity within a generation,” will consist of high-level representatives including the secretaries of Agriculture, Interior, Education, and Health and Human Services and will produce “a comprehensive interagency plan that . . . includes comprehensive, multisectoral strategies from each member executive department, agency, or office.”¹⁰

Ultimately, without an adequate evidence base on which to judge the effectiveness of any particular strategy or intervention launched across such multiple sectors, this childhood obesity initiative—as well as any other broad, multisectoral initiative on important population health problems—will succeed only by chance. The narrowness of the current CER research agenda and funding is not the most effective way to achieve better health outcomes. Might there be an alternative or companion CER research agenda that is more comparatively effective than those that have been advanced? Although what kind of CER is most effective in improving health could be a CER question, the answer is yes. Moreover, if a more balanced portfolio included a specifically earmarked share to understand better the relationships across multiple determinants of health, the potential payoff in improved population heath outcomes, including disparity reduction, could be substantial.