Briefs

Search Briefs


Alerts

Sign up for Health Policy Brief Alerts




Article

MARCH 8, 2012

Public Reporting on Quality and Costs



Do report cards and other measures of providers' performance lead to improved care and better choices by consumers?

What's the issue?

Public reporting is a strategy to address quality and cost in the health care system by providing consumers, payers, and health care providers, such as doctors and hospitals, with information about the performance of these providers and insurance plans. It can include such tools as "report cards" on hospital performance, including the information found on Medicare's Hospital Compare website about how well hospitals deliver heart attack care. Public reports can allow for the comparison of costs, quality (such as rates of hospital-acquired infections), and how satisfied patients are with service.

Advocates of public reporting believe that it helps consumers make informed decisions when choosing among physicians, hospitals, and health plans; guides employers and other purchasers when selecting insurance plans; and aids providers when making referrals to specialists. Providers and health plans, in turn, are motivated to improve their performance to protect their reputations and the demand for their services. Publicly reported information may also be useful to policy makers when assessing system performance and value.

This brief describes the theory behind public reporting, its evolution over time, and the evidence as to whether public reporting is making a difference in improving the quality of health care and lowering costs.

What's the background?

Modern efforts to promote public reporting date back to the 1980s, when the Health Care Financing Administration (the predecessor to today's Centers for Medicare and Medicaid Services) began publishing death rates at the nation's hospitals. The effort was highly controversial among hospitals; many complained that the data did not take into account that their particular population of patients may have been older or sicker and at greater risk of mortality than patients at other institutions. Although the effort was ultimately abandoned, it paved the way for growing use of public reporting in the decades that followed.

ADVANCEMENTS IN REPORTING: Since that time, public reporting has advanced considerably in depth and scope. As explained further below, various measures have been developed to capture information about the quality of health care. Some of these measure providers' performance--for example, whether a person who presents at a hospital with a heart attack receives an aspirin within an hour of arrival. Other measures focus on health care outcomes--for example, how likely are patients to die after receiving emergency cardiac procedures at one hospital versus another.

Today public reports increasingly are being developed and used by a range of actors, including federal, state, and local governments; hospitals and other health care institutions; professional associations; health insurance plans; employers; and consumers. The enactment of the Affordable Care Act of 2010 created a new context for these initiatives by framing a national strategy for quality improvement, including through public reporting.

FEDERAL ACTIVITIES: Two federal agencies within the Department of Health and Human Services (HHS) share primary responsibility for these activities: the Agency for Healthcare Research and Quality (AHRQ) and the Centers for Medicare and Medicaid Services (CMS). AHRQ supports research and works with public and private stakeholders to develop quality measures, report aggregate national- and state-level data, and conduct research on the science of public reporting. It does not, however, report measures at the provider level.

CMS collects data on performance measures from providers participating in the Medicare, Medicaid, and Children's Health Insurance Program. CMS posts comparative provider-specific information about hospitals, doctors, nursing homes, home health agencies, and kidney dialysis facilities at www.healthcare.gov. The amount of information CMS posts varies by type of provider.

The most information reported at the national level is for general hospitals and, as noted, is available at www.hospitalcompare.hhs.gov. Since 2005 this site has reported on quality measures focusing on heart attack, heart failure, pneumonia, and surgical care for all US acute care hospitals. The Hospital Compare website also includes measures developed from patient surveys in such areas as communication with doctors and nurses, responsiveness of hospital staff, pain management, cleanliness and quietness, and instructions about medications and discharge.

CMS also maintains www.medicare.gov, which provides information allowing consumers to compare the Medicare Advantage and Part D drug plans available in their area. (See the Health Policy Brief published June 15, 2011, for more information on Medicare Advantage plans.)

NEW REQUIREMENTS UNDER HEALTH REFORM: The Affordable Care Act directed the secretary of HHS to establish a national strategy for quality improvement that includes public reporting of performance information through health care quality websites. CMS and AHRQ were required under the law to convene multistakeholder groups and develop performance measures tailored to the needs of "hospitals and other institutional health care providers, physicians and other clinicians, patients, consumers, researchers, policy makers, states, and other stakeholders." The resulting performance measures were to include clinical conditions, to be provider specific, and to be detailed enough to meet the needs of patients with different clinical conditions.

The Affordable Care Act also called for public reporting of performance measures on quality, cost, and other metrics. Public reporting will be used for insurance plans that will be offered through new state-level health insurance exchanges starting in 2014. Reports will also be prepared on hospitals, physicians, and other health care providers who participate in Medicare's new "value-based purchasing" program, which will base hospital payment in part on whether providers achieve targets for delivering higher-quality care. These performance data are also to be posted at www.healthcare.gov. For Medicaid, the law required HHS to adopt an initial core set of quality measures; develop a standardized format for reporting by states; and make the information publicly available annually, beginning in 2014.

OTHER INITIATIVES: About half of the states currently have public reporting programs in place. The information collected varies considerably from state to state, by health condition, and by process and outcome measures reported. Some of these activities are publicly sponsored and funded; others are carried out in conjunction with nonprofit organizations or regional or community collaboratives in which provider systems, large purchasers of health care, and other stakeholders work together to push for quality improvement.

A variety of organizations use public reporting in their activities. Their efforts are supported by a variety of sources, including grants and contracts from federal and state governments, private foundations, large purchasers of health care, and membership fees.

Some of the more prominent of these organizations include the National Quality Forum (NQF), a nonprofit organization that works with providers, consumer groups, and governments to establish and build consensus for specific health care quality and efficiency measures, and the National Committee for Quality Assurance, which evaluates and accredits health insurance plans based on quality and value. The committee's Healthcare Effectiveness Data and Information Set measures are used by 90 percent of the nation's health plans.

Other groups involved in public reporting include the Leapfrog Group for Patient Safety (created by employers); the Informed Patient Institute, a group that rates the usefulness of existing online doctor, hospital, and nursing home report cards; and the Commonwealth Fund's www.whynotthebest.org, which allows users to compare hospitals based on specific performance measures using data from a variety of sources, including the Leapfrog Group, the Hospital Quality Alliance, and selected state reporting systems.

What are the issues?

Although the concept of public reporting has broad support, its implementation has not always been met with approval--and there is at best mixed evidence about the degree to which it has sparked changes within health care or been used widely by consumers.

CAUSES FOR CONCERN: Skeptics of public reporting have a number of concerns, including about the accuracy and reliability of the information contained in the reports. For example, only 30 percent of physicians surveyed in 2005 believed that quality measures used in public reports were generally accurate. The costs associated with collection, analysis, and dissemination of data can be high, especially for physicians and providers who have not fully implemented electronic health record-keeping systems.

There is also potential for unintended consequences that might result from providers gaming their report card scores, for example, by declining to treat patients with serious conditions that might negatively impact their ratings. Similarly, there is a risk of misinterpretation by consumers if they do not understand the terms used, what an indicator is supposed to reveal about the quality of care, or whether high or low rates reflect good performance.

Despite the degree of investment in developing and implementing public reporting systems, the jury is also still out as to the extent, or even if, public reporting will improve quality and reduce health care costs. Studies have reached different conclusions concerning the degree to which consumers and physicians use public reports, whether providers respond to public reporting of performance measures by changing their behavior, and even whether public reporting improves outcomes.

For example, Andrew Ryan of Weill Cornell Medical College in New York and colleagues found that public reporting of outcomes data on the Hospital Compare website led to no reduction in mortality beyond existing trends for heart attack and pneumonia and only a modest reduction in mortality for heart failure at virtually all US acute care hospitals. On the other hand, Maureen Smith of the University of Wisconsin--Madison's School of Medicine and Public Health and colleagues found that physician groups and clinics that publicly reported their performance on diabetes care were more likely to adopt diabetes improvement interventions than those that didn't report their performance.

Both supporters and critics recognize the need to address a number of issues in public reporting. These include choice of performance measures, data collection and system capabilities, formatting and content of reports, education and outreach to promote use of the information, and evaluation and continuous refinement to make sure that public reporting achieves its objectives rather than becoming an end in itself.

In March 2011 AHRQ convened a National Summit on Public Reporting for Consumers. The summit produced a set of recommendations that address how to expand the use of public reports, how to address issues related to content and format of reports, and the methods and data sources needed to implement the system.

MEASURE SELECTION: As noted, hundreds of measures are available to those developing public reports. Many of these have been vetted by the NQF, which has worked since the 1990s to create national consensus standards for measuring and reporting on health system performance. The federal government as well as other major health care purchasers use NQF-endorsed measures to ensure that measures are scientifically sound and to help standardize performance measures across the industry. To date, NQF has endorsed more than 750 measures.

The choice of measures to be used in any given instance requires consideration of their relevance to the target population, the availability of data (including whether the number of events is sufficient to reliably measure performance), and how the measures will be presented, for example, as a series of separate measures or those that pull together disparate information into aggregate scores. Although the science of measure development and use has evolved from an exclusive reliance on performance measures to those focused on outcomes, more work is needed to develop measures that will be the most meaningful to consumers at various points in their decision-making process.

Providers are particularly concerned that public reports fairly and accurately reflect their performance, and not things that are beyond their control, such as the risk profile of the population they treat. For example, a hospital located in a poor urban area may be more likely to treat higher-risk patients with more complex medical problems than its suburban counterpart.

This problem can be addressed through risk adjustment--statistical methods that adjust scores or values of reported data to account for these factors before results are made public. But since no risk-adjustment technique is perfect, there are concerns that patients may avoid providers whose lower scores may not accurately represent the quality of care they provide, and providers may avoid seeing patients with complex health issues whom they fear might depress their performance scores.

DATA COLLECTION: There is a general consensus that data for performance reports should be obtained automatically as part of ongoing care processes. For example, information can be collected through electronic health records while a patient is visiting the doctor's office; it can also be taken from claims that providers send to insurers or other payers requesting payment for care provided.

Although use of electronic health records is spreading, thanks in part to federal incentives, it still varies greatly across providers and geographic areas. Many consumers also remain somewhat leery--primarily for privacy reasons--of having their data collected and shared with others. Meanwhile, claims processing systems do not always capture critical elements that are needed for performance measurement. A claim won't always state whether a patient died after undergoing a particular procedure, for example.

To advance data collection and assembly of well-validated performance measures, some new models have been proposed. For example, one idea, advanced by Hal Luft, director of the Palo Alto Medical Foundation Research Institute in California, is to create a public-private data aggregator that would receive patient and provider data from payers that are deidentified in such a way as to remain useful for reporting and research. The result would be a large volume of highly standardized, granular data from multiple payers that would allow creation of provider-specific assessments.

INFORMATION USE BY CONSUMERS: One of the main concerns associated with public reporting is increasing its use by consumers and improving reports' usefulness in helping them make informed decisions among health care providers. In general, consumers' use of public reporting is low. A 2011 study of 16 community collaboratives in the AHRQ Chartered Value Exchange Program found that some websites comparing hospital performance were used primarily by consumers who were white, college educated, and over age 45. There was little use by vulnerable populations, and only about half of those visiting the sites indicated they were likely to use the data to choose a hospital. A 2008 poll from the Henry J. Kaiser Family Foundation found that 30 percent of Americans said they saw information comparing the quality of different insurance plans, hospitals, or doctors, but only 14 percent reported having used such information.

Moreover, the information contained in public reports is often not presented in such a way that is understandable and relevant to consumers. A 2007 study of five hospital-reporting services found that they used disparate measures and lacked standard definitions for reporting. The unsurprising conclusion was that consumers were more likely to be confused than informed.

In addition, consumers have preexisting ideas about health care that can be difficult to influence through public reporting. An experiment performed by Judith Hibbard of the University of Oregon and colleagues put information about providers' costs before roughly 1,400 consumers, and found that most assumed that these providers' costs were low because they provided low quality.

The bottom line is that many consumers are at risk of information overload, or otherwise disinclined to make good use of publicly reported information. Much remains to be done to make public reports accessible, understandable, and relevant. Academics and others who study public reporting say that "best practices" should focus on generating scores that show differences in performance among those being arrayed, describe measures in laymen's language, organize measures to reduce the number of factors a person must consider before making a choice, and select points of comparison for high and low performance, for example, explaining that high rates of mammography screening are indications of high performance while high rates of hospital-acquired infections indicate poor perfor-mance.

KEYS TO SUCCESS: Consumer outreach and education are also keys to public reporting success. Consumers not only need to be aware of sources of information and how to access them, but they also need to understand how the information is meaningful.

Participants at AHRQ's March 2011 National Summit on Public Reporting for Consumers discussed ways to make consumers more aware of--and inclined to use--public reports. Among strategies they suggested were ongoing campaigns to raise awareness of and demand for quality information among consumers. They also recommended engaging consumers through their providers, for example, by involving consumers and their families in practice redesign and improvement, and basing measure development on consumer needs, for example, by understanding their priorities and the information they value at specific decision points, rather than just relying on the data that are available and routinely collected. Finally, they recommended doing research on the best means to present results clearly to consumers and exploring alternative ways to deliver reports, such as through mobile technologies and automated telephone systems.

What's next?

More work lies ahead if public reporting is to meet expectations and fulfill hopes that it can spur broad change in health care delivery. A survey of 29 experts and participants at AHRQ's 2011 national summit revealed that, while none doubted the value of public and private investments made in public reporting, most agreed that the information provided so far has had little positive impact on consumers' choices of health care providers.

The challenge, the experts said, is to make further advances in measurement, data collection, and use of information technology; deliver more consumer-oriented report cards; and accomplish these objectives within the constraints of limited public funding and providers' willingness to be subjected to such scrutiny.

Resources

Bardach, Naomi S., Judith H. Hibbard, and R. Adams Dudley, "Users of Public Reports of Hospital Quality: Who, What, Why, and How?" Agency for Healthcare Research and Quality, December 2011.

Department of Health and Human Services, "Report to Congress: National Strategy for Quality Improvement in Health Care," March 2011.

Hibbard, Judith H., Jessica Greene, Shoshanna Sofaer, Kirsten Firminger, and Judith Hirsh, "An Experiment Shows That a Well-Designed Report on Costs and Quality Can Help Consumers Choose High-Value Health Care," Health Affairs 31, no. 3 (2012): 560--8.

Kaiser Family Foundation, "2008 Update on Consumers' Views of Patient Safety and Quality Information," October 15, 2008.

Luft, Harold S., "Advancing Public Reporting through a New 'Aggregator' to Standardize Data Collection on Providers' Cost and Quality," Health Affairs 31, no. 3 (2012): 619--26.

Ryan, Andrew M., Brahmajee K. Nallamothu, and Justin B. Dimick, "Medicare's Public Reporting Initiative on Hospital Quality Had Modest or No Impact on Mortality from Three Key Conditions," Health Affairs 31, no. 3 (2012): 585--92.

Sinaiko, Anna D., Diana Eastman, and Meredith B. Rosenthal, "How Report Cards on Physicians, Physician Groups, and Hospitals Can Have Greater Impact on Consumer Choices," Health Affairs 31, no. 3 (2012): 602--11.

Smith, Maureen A., Alexandra Wright, Christopher Queram, and Geoffrey C. Lamb, "Public Reporting Helped Drive Quality Improvement in Outpatient Diabetes Care among Wisconsin Physician Groups," Health Affairs 31, no. 3 (2012): 570--7.

Werner, Rachel, Elizabeth Stuart, and Daniel Polsky, "Public Reporting Drove Quality Gains at Nursing Homes," Health Affairs 29, no. 9 (2010): 1706--13.

About Health Policy Briefs

Written by
Julia James
(James previously worked on Capitol Hill and as a consultant in Washington, DC, and Oregon.

Editorial review by
Sue Felt-Lisk
Senior Health Researcher
Mathematica Policy Research

Rachel Werner
Core Investigator
Center for Health Equity Research and Promotion
Veterans Affairs Medical Center
Philadelphia

Ted Agres
Senior Editor for Special Content
Health Affairs

Anne Schwartz
Deputy Editor
Health Affairs

Susan Dentzer
Editor-in-Chief
Health Affairs

Health Policy Briefs are produced under a partnership of Health Affairs and the Robert Wood Johnson Foundation.

Cite as:
"Health Policy Brief: Public Reporting on Quality and Costs," Health Affairs, March 08, 2012.

Sign up for free policy briefs at:
www.healthaffairs.org/healthpolicybriefs