Successful measurement is a cornerstone of successful improvement. How do you know if the changes you are making are leading to improvement? Simple: you measure. Measurement doesn’t have to be difficult or time-consuming. The key is to pick the right measurements, so that you can see results quickly and adapt your interventions accordingly, putting less strain on resources and more focus on outcomes.
Want Different Results? Create a New System
If you own a Honda Civic and want to be a race car driver, you probably won’t spend your money on a souped-up engine and extra turbo for your Civic. In the same way, simply adding more people or otherwise "tweaking" the same system won’t achieve breakthrough results. You must create a different system in order to get different results.
Make Measurement Rewarding
In health care, measurements are often used for reporting aggregate results to regulators, legislators, and other parties that "judge" the data against specific standards or rules. This "measurement for judgment" can be understandably daunting to staff.
How can we make measuring an exciting learning experience in our organizations? Here are some signs of a culture in which measurement is an exciting, productive process:
- Senior management communicates improvement as an exciting challenge for the organization.
- Measurement initiatives for improvement are clearly separated from measurements needed for accreditation or other external purposes.
- Improvement teams set attainable goals and get constant visual feedback.
When people see that they can effect change, results will become more of a personal goal.
Some Measurement Guidelines
The following guidelines are important to have in mind when beginning to measure and can help lead to a successful improvement project:
- Be sure that the improvement goal, or aim, is:
- Strategic
- Relevant
- Compelling
- Important
- A stretch (i.e., challenging but not unattainable)
- Achievable
- Unambiguous
- The key measure should clarify the aim and make it tangible.
- Don’t track too many process measures (vs. outcome measures).
- Use sampling to make measurement efficient and representative.
- Integrate measurement into people’s daily routine.
- Plot data on the measures over time.
The strategies outlined below expound on these guidelines.
Five Strategies for Successful Measurement
Strategy 1: Use multiple measures.
Choose measures from the following three categories to ensure that you have an accurate picture of the effects on the system of changes you are making:
- Outcome Measures (voice of the customer or patient):
How is the system performing? What is the result?
- Process Measures (voice of the workings of the system):
Are the parts/steps in the system performing as planned?
- Balancing Measures (looking at a system from different directions/dimensions):
Are changes designed to improve one part of the system causing new problems in other parts of the system?
For more detailed information, see:
Strategy 2: Choose appropriate statistics to plot.
Annotate graphs so that the reader can see the effect of changes you are testing. When measuring incidents that are extreme but episodic (for example, the outbreak of an organism), track the time between episodes, as it will give you more useful information.
Use ratios (percents) to adjust for the impact of natural changes to the systems, such as volume. The numerator is the key measure (costs, patients waiting, etc.) and the denominator is the unit of production or volume (total costs, total patients waiting). Ratios give a more realistic picture than simply counting numbers of incidences. For example, if patients waiting for more than an hour increased dramatically, you might draw one conclusion. If you knew that overall volume had also increased (which would show in the ratio), you’d mostly likely draw another, more realistic conclusion.
On the other hand, sometimes the denominator is so large that the change looks imperceptible. Zero percent or 100 percent may be fine for regulators, but it won’t help you find the problems.
Strategy 3: Conserve resources through sampling and integration into daily work.
The pure scientific approach to sampling implies that a bigger sample creates more stable and reliable results. However, it may not show the effect of interventions or changes over time, which is most important when pursuing improvement. Large sampling efforts command a large investment in resources. Yet often a year’s efforts have the same results as that of three months, but at four times the cost. In addition, the feedback cycle is longer, making change a much slower process. Use sampling to get "just enough data" to see if changes are leading to improvement.
Graph and display your measures often enough to give your team feedback in a timely manner, both to keep momentum and to stop changes that are having adverse effects. Monthly graphs are recommended, as weekly graphs may be too variable to suggest trends. Supply time for staff to review the results and strategize.
Also, build the data collection into the daily work of staff, instead of making it a separate project. This not only aids timely, relevant collection of data, but also reduces stress by making measurement something that’s "easy" to do. Create data collection forms that include only the information you need and are easy to fill out. When integrating measurement into a staff member’s role, be sure to build in a contingency plan for ongoing collection should that person become unavailable.
Strategy 4: Plot data over time.
Although one of the more common ways to collect and display data is the pre/post method (i.e., collect data before and after a change to the system or process), this does not help improvement efforts because it doesn’t answer the question, "What were the effects of making this change?" Summary statistics hide information such as outliers and patterns. In improvement efforts, changes are not fixed but continuously adapted over time.
The best way to collect and display data is to use a run chart — a graphical record of a measure plotted over time (usually months), sometimes called a time series graph. Charts annotated with changes and events provide even more information and can help you to make connections between interventions/events and outcomes more accurately.
Strategy 5: Develop excellent visual displays of measures.
Visual displays are motivators, reality checks, and validators of work already done. They don’t need to be perfect, just useful. And don’t wait until the glitzy information system is ready; start with simple data collection such as paper and pencil.
