PDF File: 

HCT1_wp_barnhisel.pdf

Capgemini

Measuring is a way to assess and improve performance. It is important to measure the things that matter to stakeholders: the quantitative and qualitative aspects of service, quality, and financial performance.

Why Measuring Matters

When a health care organization undertakes a major initiative there is an important need to measure whether the initiative is making a difference. Opinion and anecdotal evidence don't count for much when the stakes are so high. But the question is, make a difference in what? And to whom?

All too often, organizations fall into the trap of measuring only what they can easily quantify, rather than measuring the things that matter. Most often, this trap turns into a narrow focus on measuring financial performance. However, clinical transformation can also make a dramatic difference in a hospital's service and quality of care — measures that matter a great deal to patients and clinicians.

In this article, we argue that service, quality, and financial performance are all important to a health care organization's health and must all be measured. We provide guidelines and tools for measuring these aspects of performance before and after a clinical transformation implementation. We show how clinical transformation not only improves performance, but also provides a better way of managing performance throughout the organization. Our discussion is guided by three related principles:

1. Don't just measure what you "can." Instead, measure what matters. Otherwise, in your quest for performance improvement, you will end up like the man who kept looking, unsuccessfully, for his lost glasses under the streetlight because "that was where the light was."

2. We measure what we value, and we value what we measure. What an organization chooses to measure communicates what it values and what management believes is worth working for. That is, measuring not only enables management to assess performance, it enhances performance.

3. "Soft" benefits may be harder to measure, but they matter a great deal to an organization's major stakeholders. Moreover, soft measures often have a direct and indirect impact on hard measures, such as financial performance.

What Is Important to Measure?

When an organization invests millions of dollars in clinical transformation, it is only logical to want to measure its effectiveness in financial terms. How has the investment improved the organization's financial health? Has it cut costs? Increased revenues? Improved productivity? Reduced overhead? Financial information is numerical, and numbers seem to promise a clear-cut way of discovering whether an initiative has worked.

Unfortunately, all too often, organizations think of measurement only in financial terms, even though clinical transformation promises many other equally, if not more, important benefits. These include:

• Dramatic reductions in medical errors from the administration of medications
• Many fewer omissions in care that can lead to serious complications, even death
• More effective, evidence-based treatment provided more consistently from clinician to clinician
• Clinicians who understand their practice competencies more clearly and provide care as members of interdisciplinary clinical teams
• Patient-clinician relationships that lead to greater patient satisfaction
• Healthy work environments that lead to satisfied clinicians providing the level of care of which they are truly capable

To evaluate the full impact of clinical transformation on an organization, all the service, quality, and financial benefits relevant to the organization's goals must be measured. Some of these benefits are less easily quantified than financial performance. Some might even be called "unmeasurable." Nevertheless, all of these so-called "soft" benefits make important contributions to an organization's overall performance and must be factored into any critical assessment of a clinical transformation's performance.

Different Kinds of Measures: Service, Quality, and Financial

The benefits of clinical transformation are generally grouped into three major categories: service, quality, and financial. Some of these benefits are relatively easily quantified, such as cost reductions, while others are qualitative and can only be quantified indirectly, by tabulating survey results, for example. Nevertheless, all of these benefits matter to a clinical transformation's success and must be measured and factored into an evaluation of its performance.

• Service measures mark any change in the way patients are treated by clinical and non-clinical staff. If a patient is registered or scheduled more quickly (shorter wait times); needs to give blood only once instead of twice; or is treated by a team of caregivers who know and respect his or her values, then service has been improved by "some" amount. This amount must be measured.

• Quality measures mark any change in patient outcomes. If the number of adverse drug events drops; if the incidence of omission in care decreases; and if the number of codes outside the OR declines, then quality has improved and is an important measure of a hospital's performance.

• Financial measures mark any change in a hospital's financial health. Increased revenues and productivity and reductions in costs and waste all signal a measurable improvement in the financial strength of an institution. Since an institution's financial health is most frequently expressed in numbers, it is often the most straightforward measure.
  

It should be clear that many of the measures within these broad categories are hybrids and synergistic. That is to say, an improvement in any one measure may involve, lead to, or cause improvements in one or both of the others. Similarly, an improvement that is objective and quantitative may have a corresponding (and equally important) subjective and qualitative improvement.

For example, if a new system enables a hospital to register patients more quickly and accurately, service is improved (the patient didn't have to wait as long), but so did quality (patient information flowed accurately to clinicians) and financial measures (the patient didn't give up and leave, and the registration department was able to register more patients in less time). The full impact of this one improvement could only be accurately evaluated if the service, quality, and financial benefits were measured.

This example shows how improvements can have equally important quantitative and qualitative aspects. More efficient registration can lead to fewer elopements (quantitative), but may also increase patient satisfaction (qualitative) with the hospital's service. Their perceptions of the hospital's service may change. Similarly, it isn't hard to imagine how improved patient outcomes (quantitative) would tend to give patients a favorable view (subjective) of a hospital's quality of care. Again, it is important to understand and measure both quantitative and qualitative changes to appreciate the full performance impact of a clinical transformation initiative.

Tools for Measuring Performance

The tools for measuring the service, quality, and financial performance of clinical transformation — both quantitatively and qualitatively — follow a basic structure.

• Gather baseline performance information
  
• Implement the clinical transformation system(s)
  
• Compare the before and after to determine any changes in performance
  

Gathering baseline performance information involves deciding which indicators will be used to measure service, quality, and financial performance. For example, a hospital might decide to track the number of adverse drug events, average length of stay, or efficacy of new protocols, before and after implementation, in order to measure quality. A broad range of relevant indicators should be used to create an accurate and information-rich picture of performance — one that can inform intelligent strategic and tactical decision-making.

Since health care organizations operate (and compete) within a market environment, a hospital's baseline information must also include leading industry practices and benchmarks. For example, it is not enough for a hospital's quality to improve in relation to its own prior performance; it must also improve relative to its competitors and the top performers in the industry. Clinical transformation promises to help make health care organizations more competitive. Effective measurement not only helps them achieve their personal best, it can help them win the race.

Additionally, a hospital's baseline information needs to include benchmarks required to demonstrate compliance with accreditation and regulatory requirements. Specifically, the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) now requires that hospitals track and meet certain clinical outcomes — or ORYX® — standards.

The tools for gathering and comparing this baseline information are the database and the survey. The database is used to gather and analyze quantitative data, such as the number of adverse drug events, while the survey is used to gather qualitative information, such as a patient's perceptions of service or quality. Once gathered, survey information is then entered into a database for tabulation and analysis. Nevertheless, qualitative information can never be entirely reduced to numbers, nor should it be. Surveys should leave room for respondents' comments, and a representative sampling of these comments should accompany tabular results to help executives better understand them. To know that patients were "very unsatisfied" with service is one thing, to read that they "waited for four hours" and "the registration desk was rude" is another.

In many cases, hospitals are already gathering information about service, quality, and financial performance, but have never captured it in useable form. That is, in a form that can inform decision making. In other cases, baseline data, including industry benchmarks, will initially have to be gathered manually by consulting disparate and largely paper-based sources. In either case, the new clinical transformation systems should be designed to gather data on key performance indicators in an analyzable form automatically.

Using Measures to Manage Performance

Hospitals measure performance to help them improve performance. To do this, performance must be measured regularly so that it can be used to track progress and guide corrective decision making when goals aren't met. Since clinical transformation systems automate data gathering (in some cases, automatically generating data based on a series of triggering events), they can dramatically enhance performance management. In a manual world, it is close to impossible to devote sufficient resources to gathering data every month or quarter; often it's simply not done.

Thus, in addition to improving service, quality, and financial performance, clinical transformation is a powerful tool for improving the management of performance. Here, the principal tool is the executive dashboard, which provides executives with a simplified, but accurate and actionable view of the organization's service, quality, and financial performance. Much like a car's dashboard that tells the driver how fast the car is going, how far it has traveled, and the status of key systems (gas, oil, and temperature), the executive dashboard provides a snapshot of the organization's performance at that moment relative to the goals it has set.

For example, if management has set a goal of reducing average length of stay by one day, the executive dashboard aggregates many bits of data to provide a quick status report on progress toward that goal. If the dashboard shows "green," performance is on target; if "yellow," progress may have slowed or stalled; and if "red," average length of stay may be increasing. Since the dashboard is fed by databases and survey results, executives can always view the situation in greater detail by requesting reports containing the raw data.

Measuring performance to improve performance requires that management set performance objectives and incentives for staff and communicate them clearly. For example, if management wants to reduce wait times in the ER, this goal must be built into the performance reviews and incentives for ER staff and clinicians (and for whomever else has a direct or indirect impact on ER wait times). Staff must understand their objectives and know that their performance will be evaluated and rewarded, in part, on how well they meet these objectives. Since individual staff objectives contribute to meeting the organization's strategic goals, staff should understand how doing their part fits into the larger picture of helping the hospital become a more successful organization.

Management must also stipulate desired performance measures in designing the clinical information system. Although the JCAHO's ORYX measures are required for all accredited hospitals, not all CIS packages currently available from vendors include these measures as a standard feature. Management must ensure that clinical staff build these measures into the system's design and implementation process.

Summary

Measuring matters, because it is a way to assess and improve performance. What an organization chooses to measure communicates to stakeholders the kinds of performance it values. Thus, it is critically important to measure the things that matter to stakeholders: the quantitative and qualitative aspects of service, quality, and financial performance. Measuring is a three-step process:

1. Establish internal and industry baselines and benchmarks.
2. Implement a clinical transformation designed to improve performance in those areas.
3. Measure any performance improvements.

Databases and surveys are the principal tools used to measure performance. Clinical transformation itself increases an organization's ability to measure performance because it automates data capture and generates reports useful for managing staff performance relative to the organization’s performance and strategic goals.and generates reports useful for managing staff performance relative to the organization's performance and strategic goals.