Video Transcript: Run Charts (Part 1)

Bob Lloyd, PhD, Executive Director Performance Improvement, Institute for Healthcare Improvement

One of the fundamental tools of quality improvement is being able to make a run chart to understand the variation in your process over time. A run chart is very simple. On the x axis, we have data in some sort of chronological order. For example, we have Monday, Tuesday, Wednesday; it could be January, February, March. And on the y axis, we put our measure ― I’m just going to call it ‘m,’ measure of interest ― and that could be percent, it could be a count, it could be money. We get the data in chronological order, and we plot them. We connect the dots with a line, and then we need to start figuring out how to interpret the chart. Well, we do that with several simple steps.

The first step is to put a center line through the middle of the plotted dots. This center line, sometimes called CL, for a run chart is what’s called the “median.” Now the median is also known as the 50th percentile. It is shown mathematically as an “x” with a little tilde above it.

We have the data plotted in time sequence, we have our center line, a form of the average, if you will, but it’s the median for the run chart. Now what we’re going to do is define a “run.”

What is a run? Now a run is one or more data points on the same side of the median. So here we have one data point, here we have one. One, one, one; but here we have one, two, three, four data points. A run can be one or more data points on the same side of the center line. That’s the key. As the data flip and flop back-and-forth across the center line, we count how many dots end up in a little cluster. Here we have one, one, one; here we have three, maybe two here, one, one.

So we get the number of runs, and then we are going to be able to interpret the chart, and we do that by using a series of simple run chart rules. Now there are many run chart rules that people have used over time, and you will see different people using different rules. Here at the IHI, we have what we call the four simple run chart rules. They are basically: a shift in the data, a trend in the data, whether you have too many or too few runs, and, finally, an astronomical data point. And let me explain each of these quickly.

A shift in the data is when you have too much of the data hanging above or below this median center line. And the way we make this determination is if you have six or more data points hanging in a group above or below the center line, that’s an indication of a shift, that the data have moved to a level and stayed there too long. So you have random, and then all of the sudden you have one, two, three, four, five, six, and then it goes random again. This run of six data points in a row above the center line signals a shift, and the data have hung there for too long when they should have just randomly been flipping and flopping. And you can see there could be a shift downwards as well.

The second one is a trend. And while some people think this is a downward trend or this is an upward trend, two data points does not make a trend. What we are looking for to get a statistical trend in the data is to have five data points constantly going up or constantly going down. Now if you had data points that went up, up, up, repeat, repeat, repeat but kept going up, you don’t count the repeats, but as long as it continued its upward journey or downward journey, it’s still a trend. If it went up, up, equal, equal then dropped, then the trend would be cancelled. But a trend ― and this is one that a lot of people struggle with ― is five or more data points constantly going up or constantly going down.

Third one, [too many or too few runs] , requires a table. What you do is find out how many runs you have on your chart, and then you look up on this table, for the total number of data points (‘n’), what was the low number of runs and the high number of runs? And for a given number of data points, say 20 data points, it’ll tell you that you should have no fewer than x number of runs and no more than y. And the idea here is that if data are randomly arrayed, you should see some sort of random flipping and flopping back-and-forth. If you get data, again, that are hanging on one side or the other, and only two runs in your data, you’re going to have not enough data that forms, essentially, a normal distribution. So this table, which has been figured out mathematically for years, is designed to tell you how much variation there should be in a given set of data. So if you had 15 data points, 20, 30, it will tell you the lower and upper boundaries of the number of runs.

The final test ― or rule, if you will ― is whether or not we have an astronomical data point. Now this is a judgment call, something that I’ve referred to as the inter-ocular test of significance. We have data that are going along, and then, all of a sudden ― “wonk!” ― we’ve got this huge spike, and we wonder why. Well, often times two things. One, we could have collected the wrong data, and for some reason data got into our data set that shouldn’t be in there, cause here’s where the bulk of the data typically fall. Or, in fact, something special was going on on that day: This is food trays being delivered to the medical units, and this is the day that the elevator people came and shut down three banks of elevators and all the food trays backed up. If you’re looking at percent of food trays delivered on time, you’d see this big spike. Well, an astronomical data point is not a statistical determination on a run chart, it’s an eyeball test, and its guidance that you should either look at your data or put your data on a control chart ― which will be in a subsequent session ― to find out if, in fact, that is truly different than the rest of the data.

So there you have it, the run chart in a nutshell. You have the elements, x and y axis, the median, plot the data over time, figure the number of runs, and then apply the four simple run chart rules.