Science of Improvement: Establishing Measures

​Model for Improvement: How will we know that a change is an improvement?

Measurement is a critical part of testing and implementing changes; measures tell a team whether the changes they are making actually lead to improvement. Measurement for improvement should not be confused with measurement for research. This difference is outlined in the table below.

Measurement for Research Measurement for Learning and Process Improvement
PurposeTo discover new knowledgeTo bring new knowledge into daily practice
TestsOne large "blind" testMany sequential, observable tests
BiasesControl for as many biases as possibleStabilize the biases from test to test
DataGather as much data as possible, "just in case"Gather "just enough" data to learn and complete another cycle
DurationCan take long periods of time to obtain results"Small tests of significant changes" accelerates the rate of improvement

 

See also: Tips for Effective Measures.


The Whole System Measures, a set of health system performance measures, keyed to the six dimensions of quality outlined by the Institute of Medicine in the Crossing the Quality Chasm report — safe, effective, patient-centered, timely, efficient, and equitable — that can be used to evaluate the overall performance of a health system. 


Three Types of Measures 

Use a balanced set of measures for all improvement efforts: outcomes measures, process measures, and balancing measures.

Outcome Measures

How does the system impact the values of patients, their health and wellbeing? What are impacts on other stakeholders such as payers, employees, or the community?

  • For diabetes: Average hemoglobin A1c level for population of patients with diabetes
  • For access: Number of days to 3rd next available appointment
  • For critical care: Intensive Care Unit (ICU) percent unadjusted mortality
  • For medication systems: Adverse drug events per 1,000 doses

Process Measures
Are the parts/steps in the system performing as planned? Are we on track in our efforts to improve the system?

  • For diabetes: Percentage of patients whose hemoglobin A1c level was measured twice in the past year
  • For access: Average daily clinician hours available for appointments
  • For critical care: Percent of patients with intentional rounding completed on schedule.

Balancing Measures (looking at a system from different directions/dimensions)
Are changes designed to improve one part of the system causing new problems in other parts of the system?

  • For reducing time patients spend on a ventilator after surgery: Make sure reintubation rates are not increasing
  • For reducing patients' length of stay in the hospital: Make sure readmission rates are not increasing


Sample Measures

See the Measures section for sample measures.

 

Using Sampling: An Example

Here is how one team used sampling in measuring the time for transfer from Emergency Department (ED) to inpatient bed.

Rapid movement from the Emergency Department (ED) after a decision to admit the patient is critical flow for entry to the entire system for emergent patient care. It represents the ability of patients with various illnesses to get into the system through the most common admission route. 

Sampling approach: The measurement will consist of 6 weekly data collections of 25 patients each. The patients can be sampled in several ways:
  • 5 patients per day for 5 days of the week. The patients must be consecutive and at least one day must be a weekend day.
    or
  • 25 consecutive patients regardless of any specific day, except that it must include some weekend admissions.
    or
  • If there are fewer than 25 admissions for a week, the total admissions for the week should be included in the sample.
The time is measured from the decision to admit to the physical appearance of the patient into the inpatient room. The destination cannot be a "holding area" but must be a "real inpatient bed." The sample collection should be done in real time, so a data collection process needs to be worked out by members of the team to achieve this goal. The collections must be done weekly and summarized as the percentage of patients in the sample that achieved the goal for that week. Six weeks of data needs to be collected and six data points placed on a run chart.
 

Plotting Data Over Time

Plotting data over time using a run chart is a simple and effective way to determine whether the changes you are making are leading to improvement. Annotate the run chart to show the changes you made.

Example 1:
Graph_ReducingDelaysForERPatients.gif 

Example 2:
Graph_MedicationErrorsperDay.bmp 

Average Content Rating
(8 users)
Please login to rate or comment on this content.
User Comments