Model for Improvement: How will we know that a change is an improvement?
Measurement is a critical part of testing and implementing changes; measures tell a team whether the changes they are making actually lead to improvement. Measurement for improvement should not be confused with measurement for research. This difference is outlined in the table below.
|
Measurement for Research |
Measurement for Learning and Process Improvement |
|
Purpose | To discover new knowledge | To bring new knowledge into daily practice |
|
Tests | One large "blind" test | Many sequential, observable tests |
|
Biases | Control for as many biases as possible | Stabilize the biases from test to test |
|
Data | Gather as much data as possible, "just in case" | Gather "just enough" data to learn and complete another cycle |
|
Duration | Can take long periods of time to obtain results | "Small tests of significant changes" accelerates the rate of improvement |
See also:
Tips for Effective Measures.
The
Whole System Measures, a set of health system performance measures, keyed to the six dimensions of quality outlined by the Institute of Medicine in the
Crossing the Quality Chasm report — safe, effective, patient-centered, timely, efficient, and equitable — that can be used to evaluate the overall performance of a health system.
Three Types of Measures
Use a balanced set of measures for all improvement efforts: outcomes measures, process measures, and balancing measures.
Outcome Measures
How does the system impact the values of patients, their health and wellbeing? What are impacts on other stakeholders such as payers, employees, or the community?
For diabetes: Average hemoglobin A1c level for population of patients with diabetes
For access: Number of days to 3rd next available appointment
For critical care: Intensive Care Unit (ICU) percent unadjusted mortality
For medication systems: Adverse drug events per 1,000 doses
Process Measures
Are the parts/steps in the system performing as planned? Are we on track in our efforts to improve the system?
For diabetes: Percentage of patients whose hemoglobin A1c level was measured twice in the past year
For access: Average daily clinician hours available for appointments
For critical care: Percent of patients with intentional rounding completed on schedule.
Balancing Measures (looking at a system from different directions/dimensions)
Are changes designed to improve one part of the system causing new problems in other parts of the system?
- For reducing time patients spend on a ventilator after surgery: Make sure reintubation rates are not increasing
- For reducing patients' length of stay in the hospital: Make sure readmission rates are not increasing
Sample Measures
See the
Measures section for sample measures.
Using Sampling: An Example
Here is how one team used sampling in measuring the time for transfer from Emergency Department (ED) to inpatient bed.
Rapid movement from the Emergency Department (ED) after a decision to admit the patient is critical flow for entry to the entire system for emergent patient care. It represents the ability of patients with various illnesses to get into the system through the most common admission route.
Sampling approach: The measurement will consist of 6 weekly data collections of 25 patients each. The patients can be sampled in several ways:
- 5 patients per day for 5 days of the week. The patients must be consecutive and at least one day must be a weekend day.
or
- 25 consecutive patients regardless of any specific day, except that it must include some weekend admissions.
or
- If there are fewer than 25 admissions for a week, the total admissions for the week should be included in the sample.
The time is measured from the decision to admit to the physical appearance of the patient into the inpatient room. The destination cannot be a "holding area" but must be a "real inpatient bed." The sample collection should be done in real time, so a data collection process needs to be worked out by members of the team to achieve this goal. The collections must be done weekly and summarized as the percentage of patients in the sample that achieved the goal for that week. Six weeks of data needs to be collected and six data points placed on a run chart.
Plotting Data Over Time
Plotting data over time using a run chart is a simple and effective way to determine whether the changes you are making are leading to improvement. Annotate the run chart to show the changes you made.
Example 1:
Example 2:
