Video Transcript: Family of Measures

Bob Lloyd, PhD, Executive Director Performance Improvement, Institute for Healthcare Improvement

Central to any quality improvement project is measurement, and often times people struggle with not knowing, what are the different types of measures? So, let’s start with an aim – and that aim for today’s argument is to reduce the number of falls, inpatient falls, by twenty-percent within twelve months. But, now it raises the question: if falls is the outcome, the operative word that we’re going to focus our work on, what do we mean by falls? This is where we start to think about the measures of an outcome and then the related process and balancing measures. So we have three groups we want to think about. We have outcome measures, we then have process measures that we believe drive the outcome, and finally there’s a category of measures that aren’t always essential, but are useful, and I’ll talk more about those in a moment. Those are called balancing measures.

Now let’s go back to our outcome – if we were looking at falls, and the outcome is the reduction of falls, we have choices. On the one hand, you could count the number of falls each day, each week. You could then also look at the percent of patients who fell once or more. Because a percentage is a binomial, the patient either fell or didn’t fall, and you don’t count duplicates, that’s one of the problems with doing a percentage. The third way, is to do a falls rate – this is where we take the total number of falls, including duplicates because some patients can fall more than once, the number of falls over inpatient days, and we typically normalize this to one-thousand. So, in this case we would have 3.2 falls per one-thousand patient days, alright? So, now we have a simple count – number of falls, perfect of falls, and a falls rate. Typically, you have one major outcome measure, or you have a couple. In this case, falls, it’s all the same category, so we should pick one or other of these. As I mentioned, percent of falls is often times not a good measure because it only looks at whether the patient fell once or more. Typically, people will look at the count of the number of falls or compute a falls rate. So, for today, let’s just stick with the number of falls as our outcome measure and save those for another day.

Now, we have the process measures. There could be many process measures – one might be the percent of patients properly assessed, at time of arrival and admission, for falls. We have numerous falls assessment tools. Secondly, it could be the percent of patients reassessed within twenty-four hours, because patients should be reassessed regularly to find out if the falls potential changed. Thirdly, we could look at the number of times a patient is monitored and checked for getting out of the bed or having a problem that they needed to move when in fact they shouldn’t, and finally, you could look at the number of times that you do certain procedures and programs. There’s a program called Intentional Rounding, IR, where nurses or nurse assistants make rounds on patients regularly. So, now what we’ve done is we’ve identified three process measures that we believe have an impact on changing the falls that we’re observing.

Now, let’s move to the balancing measures – as I mentioned, balancing measures are not always necessary, but if you think about balancing measures it’s almost like a teeter-totter, and you want to find out if in fact we are keeping the system in balance or if it’s teetering one way or the other. Balancing measures are things that you want to make sure you understand that reflect whether or not the system is being disturbed. For example, let’s imagine that volume of admissions went down – what effect would that have on the number of falls? Well, most likely, if the volume of patients coming into the system went down, the likelihood that you might see a drop in falls as well, just because there’s fewer people in the hospital, people can attend to those that are admitted, and watch them more vigilantly. So, balancing measures are there to help you make sure that you’re not disturbing another part of the system, or making an assumption about a change in the system, when in fact it wasn’t due to anything you did, but because of something you hadn’t been observing. Balancing measures are typically dealing with volume, sometimes they deal with financial, they can deal with flow, throughput, turnaround time, a variety of things we look at as balancing measures, but again they aren’t as essential as having your outcome and process measures. By in large, what you’re going to have, are a few of these outcome measures. You’re going to have more of these process measures, there’s going to be more of these. Why? Because there’s more processes involved effecting a single outcome. And then again, you’re going to have a smaller number of balancing measures if you decide they’re relevant.

Now, the final thing that’s important to realize is that each of these measures should then be tracked on its own run chart, or control chart. So, if we looked at the number of falls, and we did it by week - week one, week two, week three, and week one, let’s say zero to twenty falls, week one we saw eight falls, week two we say twelve falls, week three we saw five falls. We would want to make sure that we’re linking someway to analyze our data through a run or control chart so that we can track each of our measures that are processes and each that are outcomes. Final thing is that you’ve got to be aware of the fact that the process measures are going to change much more quickly, and in a shorter period of time then your outcome. So you may be working on falls assessments, and get one-hundred percent of all patients reviewed and assessed, but falls aren’t declining because assessment alone is not enough to drive the outcome measure. So, what you need to do is be able to track all of your process measures, realizing full well that they’re going to change faster than your outcome. It’s that time ordering that often times people don’t realize – outcomes change more slowly than process measures.

Hopefully this will help to get you started on thinking about outcome process and balancing measures, a family of measures.