Health Affairs

The U.S. government has jump-started an unprecedented effort to better understand what works and what doesn't in health care. The effort, called comparative effectiveness research, is designed to determine which treatments, diagnostic tests, public health strategies (such as broad-based cancer screening), and other health care services accomplish the most good for people in general or for different groups within the population.

Recent legislation, including the Affordable Care Act, has directed new funding toward this research. But this expansion of effort raises a number of questions: What methods should be used to conduct the research? Will physicians and other health care providers change what they do for patients based on comparative effectiveness research findings? How will patients and providers learn about the results? Will the research be conducted openly and soundly enough that patients and providers will trust the outcomes? Will private insurers and other payers use the research findings to make decisions on whether to cover treatments, and how much to pay for them?

This brief examines some of these key issues and areas of controversy.

With 2010 health spending estimated at $2.6 trillion, the United States spends more than any country--and more per person--on health care. Yet it's widely agreed that much of the health care provided in the United States is of little value, and in some instances may even harm patients.

There is also little or no scientific evidence to support much of U.S. health care. In fact, more than half the treatments provided to patients lack clear evidence that they are effective at all, according to the Institute of Medicine, part of the National Academies. And in cases where there are two different treatments for the same condition--for example, surgery versus medication--there is only rarely adequate evidence about which one is more effective.

Life-Or-Death Concerns: For many patients and their health providers, this lack of understanding what works best in health care can be a life-or-death issue. For insurers and government officials seeking to spend health care dollars as wisely as possible, knowing which approach works best could enable them to guide patients to optimal treatments, and help them make decisions about which treatments to cover and what to pay for them.

The Institute of Medicine has defined comparative effectiveness research as "the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or improve the delivery of care." The purpose of the research is to help consumers, clinicians, purchasers, and policy makers make informed decisions about health care for individual patients and the population as a whole.

These studies, however, are not designed to look for the most cost-effective alternatives. Cost- effectiveness research, which has been debated in Congress for many years, is largely excluded from the comparative effectiveness research effort created through national health reform legislation enacted in March 2010. Exhibit 1 defines some of the basic terms used in health care research in general, and comparative effectiveness research in particular.

Comparative effectiveness research has been carried out in the United States for years, mainly under the aegis of the National Institutes of Health (NIH), the Agency for Healthcare Research and Quality (AHRQ), and the Department of Veterans Affairs. But this type of research received a big boost in 2009 under the American Recovery and Reinvestment Act, or stimulus legislation.

Boost In Funding: That law authorized $1.1 billion more to be spent on the research and designated the money to three agencies: NIH, AHRQ, and the Office of the Secretary of Health and Human Services. To oversee how that money was spent, the law also created a Federal Coordinating Council for Comparative Effectiveness Research, an advisory board mostly made up of clinicians. Exhibit 2 outlines how the three agencies have directed their comparative effectiveness research funds.

The stimulus law also directed the Institute of Medicine to recommend national priorities for comparative effectiveness research. After soliciting nominations through a Web-based questionnaire and receiving testimony from a wide range of interested parties, the IOM recommended 100 research priorities. These included determining the best test strategies for coronary heart disease, the best strategies to prevent older adults from falling, and the best ways to treat lower back pain.

Ongoing Support: The provisions incorporated into the stimulus law were just the start of a much larger comparative effectiveness research effort. The national health reform legislation, known as the Affordable Care Act, established a new, nongovernmental entity called the Patient-Centered Outcomes Research Institute to oversee and set guidelines for the research (Exhibit 3). The law also created a steady stream of research funding. Starting in 2013, Medicare and all private health insurance companies will pay a tax into a trust fund that will support the activities of the new institute. This funding is estimated to reach $500 million annually by 2015.

Under the law, a 21-member board of governors for the institute was picked by the acting comptroller general, head of the Government Accountability Office (the arm of Congress charged with evaluating and investigating the federal government). Although the institute's mandate has not been clearly defined, its main function appears to be formulating a portfolio of research projects, with a methodology committee involved in setting research standards.

Under the legislation, the institute will contract with NIH, AHRQ, and private sector organizations to oversee funding and research, suggesting that it will outsource everything from soliciting proposals to evaluating outcomes.

Side-Stepping Controversy: In the health reform legislation, Congress rejected using cost-effectiveness analyses to aid Medicare coverage and reimbursement decisions. In particular, it wanted to stay away from the metric called quality-adjusted life-years (QALYs)--which is used by England's and Wales' National Institute for Health and Clinical Quality (NICE)--to define health outcomes as part of cost- effectiveness determinations.

Cost-effectiveness analysis has aided coverage and reimbursement decisions elsewhere in the world. NICE has adopted a cost-effectiveness threshold range of £20,000-£30,000 per QALY, or about US$33,000-$50,000. The agency doesn't accept or reject technologies on cost-effectiveness grounds only, although the calculus does play a key role in NICE's decisions.

During the debate over national health reform in the United States, the notion of weighing QALYs and costs as part of the calculus for deciding whether to cover treatments became a political minefield. Critics associated the metric with "rationing"--that is, with explicit decisions to withhold certain types of care from patients because they were too costly. As a result, the final language of the Affordable Care Act forbids the government from using QALYs and other cost-effectiveness estimates "as a threshold to determine coverage, reimbursement, or incentive programs" under Medicare (Exhibit 4).

The government is also forbidden from making decisions on "coverage, reimbursement, or incentive programs" under Medicare "in a manner that treats extending the life of an elderly, disabled, or terminally ill individual as of lower value than extending the life of an individual who is younger, nondisabled, or not terminally ill."

There continues to be confusion over the topic of comparative effectiveness research as well as the creation of the Patient-Centered Outcomes Research Institute and its role in devising the future comparative effectiveness research agenda. Supporters of the research stress the positive outcomes that it could produce. Among them:

More Clarity On Appropriate Treatments: Advances in science often lead to new health care treatments, but don't necessarily provide information about which ones work best, and for which patients. Comparative effectiveness research aims to develop a better understanding of treatment outcomes that best fit an individual's needs and preferences. Clinicians and patients need to know not only how treatments work for the general population, but also which ones work best for specific types of patients, such as the elderly, racial and ethnic minorities, and those with more than one disease or condition. With this greater clarity, the odds are better for accelerating the use of beneficial innovations and delivering the right treatment to the right patient at the right time.

More Information On Neglected Diseases Or Populations: Among the priorities the IOM proposed for comparative effectiveness research was a focus on historically neglected issues, such as minority health care and mental illness. Supporters of comparative effectiveness research say it could help reduce disparities in health and health care and make the system fairer for those people and conditions that have often been left out.

Broad Input On The Research Focus: The coming comparative effectiveness research agenda will be set by the Patient-Centered Outcomes Research Institute, whose board includes a diverse range of private stakeholders in addition to government policy makers. The 21 board members include representatives of consumers and patients, hospitals, industry, nurses, payers, physicians, researchers, surgeons, and the leaders of NIH and AHRQ. The multi-stakeholder orientation of the board has attracted broad support.

More Value For The Money: Despite the language in the Affordable Care Act that restricts the use of cost-effectiveness analysis in Medicare's coverage decisions, backers of comparative effectiveness research say it could lead to making better use of the nation's health care dollars. If there's more clarity about which treatments work best--and for which types of patients--there's potential for shifting money to those interventions and away from less effective treatments.

Critics of comparative effectiveness research still have concerns about how research results may be used, including the following:

Mandating Treatment Decisions: As noted above, Congress has barred the federal government from using simple rules based on measurements such as quality-adjusted life-years to determine coverage, reimbursement, or incentive programs under Medicare. Some Republican lawmakers and other conservatives fear that the law could be changed or ignored--and that there might still come a time when such metrics will be used to tie the cost of care to the value of supporting a person's life.

Undermining Access To Care: There is evidence that the general public is concerned about how comparative effectiveness studies could be used to limit their health care choices. Two recent national public opinion surveys found broad support for using comparative effectiveness research results to provide additional information to doctors and patients, but less support for using the results to allocate government resources or mandate treatment decisions. Only about half the respondents to a poll conducted in May supported using comparative effectiveness research to determine whether Medicare and private insurance companies will cover new and existing medical treatments.

The board of governors for the Patient-Centered Outcomes Research Institute now has important decisions to make as it helps to shape the comparative effectiveness research agenda. The board is likely to have substantial input into developing priorities for research, and will also be responsible for overseeing the distribution of information to patients and providers.

The board and the institute must also contemplate a number of technical questions, such as how to make sure that the research fairly compares different interventions. They will also have to insure that long-neglected diseases or conditions, as well as health care issues affecting minorities and ethnic groups, are properly addressed through the research.

Thorny Issues Ahead: One of the thorniest questions remaining is the future use of cost-effectiveness research. Nothing in the law prohibits this type of research from being carried out; rather, the institute and the government cannot develop a dollars-per-quality-adjusted life-year or some other cost-effectiveness metric as a "threshold" to recommend for or against coverage of specific health care interventions.

But researchers carrying out federally funded comparative effectiveness studies can include a cost- effectiveness analysis, or information enabling others to perform those analyses. As yet, we don't know what patients, providers, and payers will do with that information--ignore it, or use it in some fashion to achieve better value for the health care dollars the nation spends.

Association of American Medical Colleges, "Summary of Patient- Centered Outcomes Research Provisions," March 2010.

Avorn, Jerry and Michael Fischer, "'Bench to Behavior': Translating Comparative Effectiveness Research into Improved Clinical Practice," Health Affairs, 29, no. 10 (2010): 1891-1900.

Benner, Joshua S., Marisa R. Morrison, Erin K. Karnes, S. Lawrence Kocot, and Mark B. McClellan, "An Evaluation of Recent Federal Spending on Comparative Effectiveness Research: Priorities, Gaps, and Next Steps," Health Affairs, 29, no. 10 (2010): 1768-76.

Garber, Alan M. and Harold C. Sox, "The Role of Costs in Comparative Effectiveness Research," Health Affairs, 29, no. 10 (2010): 1805- 11.

Gerber, Alan S., Eric M. Patashnik, David Doherty, and Conor Dowling, "The Public Wants Information, Not Board Mandates, from Comparative Effectiveness Research," Health Affairs, 29, no. 10 (2010): 1872-81.

Institute of Medicine, Board on Health Care Services, "Initial National Priorities for Comparative Effectiveness Research," June 30, 2009.

Neumann, Peter J. and Dan Greenberg, "Is the United States Ready for QALYs?" Health Affairs, 28, no. 5 (2009): 1366-71.

Patel, Kavita, "Health Reform's Tortuous Route to the Patient-Centered Outcomes Research Institute," Health Affairs, 29, no. 10 (2010): 1777-82.

Robinson, James C., "Comparative Effectiveness Research: From Clinical Information to Economic Incentives," Health Affairs, 29, no. 10 (2010): 1788-95.

Wilensky, Gail R., "The Policies and Politics of Creating a Comparative Clinical Effectiveness Research Center," Health Affairs, 28, no. 4 (2009): w719-29 (published online).

Written by
John Donnelly
Vice President and Senior Editor
Burness Communications
(Donnelly is a veteran journalist who often writes on health policy issues.)

Editorial review by
Alan M. Garber
Staff Physician
VA Palo Alto Health Care System
Professor of Medicine, Economics, and Health Research & Policy
Center for Health Policy/Center for Primary Care and Outcomes Research
Stanford University

Gail R. Wilensky
Senior Fellow
Project HOPE
(Wilensky is an economist and former administrator of HCFA, the predecessor agency to the Centers for Medicare and Medicaid Services.)

Susan Dentzer
Editor-in-Chief
Health Affairs

Ted Agres
Senior Editor
Health Affairs

Health Policy Briefs are produced under a partnership of Health Affairs and the Robert Wood Johnson Foundation.

Cite as:
"Comparative Effectiveness Research," Health Affairs, October 5, 2010.

Sign up for free policy briefs at:
www.healthaffairs.org/healthpolicybriefs