Why It Matters
Quality improvers can apply all five domains of the IHI Psychology of Change Framework to determine measures and collect and analyze data.
Processing ...

How to Measure the Impact of the Psychology of Change

By IHI Multimedia Team | Thursday, January 3, 2019

How to Measure the Impact of the Psychology of Change

Photo by Patricia Serna |Unsplash

Health care professionals may possess improvement expertise, but many find it difficult to sustain improvement in their organizations. To address this challenge, IHI’s innovation team developed the IHI Psychology of Change Framework to Advance and Sustain Improvement. Almost 4,000 people have so far downloaded the white paper on the framework since its release this past November. The following excerpt of the white paper shares strategies for measuring the impact of the Psychology of Change Framework on improvement projects.

How can teams and organizations know that the IHI Psychology of Change Framework is having the desired impact on improvement efforts? A measurement system for psychology of change should focus on the effect of the framework on achieving the goals of the specific improvement activity. Given the variety of improvement initiatives, there is no single measure of effect of the IHI Psychology of Change Framework. Instead, each improvement initiative or project will have its own process and outcome measures. Compared to sites that have no experience with the framework, sites that do have this experience should achieve greater, faster, or more sustained improvements in the initiatives’ desired outcomes and processes.

For example, an improvement initiative on adverse drug events may include the main measures of outcome (e.g., adverse drug events) and the rate of spread of the specific processes (e.g., the number of care units using unit briefings to identify medication safety issues). In addition, quality improvers may assess the effectiveness of the IHI Psychology of Change Framework through a series of potential activity measures (e.g., the number of trainings held, number of individuals who participate in improvement work, and number of core leaders who are engaged). An increase in agency across an improvement team could also be measured, for instance, as team members report daily or weekly the percentage of team members that respond “agree” or “strongly disagree” to the statement, “I feel able to contribute my knowledge and skills to advance our team’s shared purpose.” By tracking this percentage over time, leaders can gain a sense of whether or not team members’ agency has improved.

Strive for Improvement At All Levels

Ultimately, beyond improvement-related aims and measures, improvement leaders should see system-level outcomes consistent with the behaviors that people are embracing: higher adoption of improvement methodologies across the system, improved psychological safety, and lower staff burnout and turnover.

In addition to these system-level measures of the effectiveness of the IHI Psychology of Change Framework, organizations can use individual-level measures of the framework’s effectiveness. Process and balancing measures, as applicable, may indicate such things as whether those involved in improvement work experience increased wellbeing and satisfaction; higher discretionary effort and voluntary participation (or lower attrition over time); and increased retention and willingness to participate in future improvement work. Likert-scale-based assessments of those involved in improvement work may allow self-reported assessments of people’s intrinsic motivation to participate (e.g., How motivated are you to participate in this project?); feelings of involvement in design and implementation (e.g., How involved were you in the design of this initiative?); sense of belonging and improved quality of relationships (e.g., How much do you feel that you are a part of something?); perceptions of how valued people on the team feel (e.g., How valued do you feel?); and resiliency to adapt and willingness to learn (e.g., How ready are you to adapt if needed?).

Quality improvement teams should determine operational definitions for these concepts and associated measures based on the practices employed to advance the psychology of change, as well as other contextual factors. Quantitative assessments may use a Likert-type scale of measurement, and qualitative methods may include story harvesting, observational data, and feedback. Specific improvement logic models may further subdivide these measures by short-, medium-, and long-term outcomes.

Stratify Your Data for Greater Accuracy

In addition, stratification and rational subgrouping (by unit, department, discipline, and other demographic factors such as race and ethnicity) are helpful. By drilling down into different units or groups of staff or patients, organizations can identify the people who can activate others, as well as those to target first for engagement in improvement efforts. More frequent and tailored data collection, and transparent sharing of results with each work unit and its leaders, allows for more real-time improvement and a better way to track the impact of changes over time. Data stratification and rational subgrouping should be determined by the improvement team and subject-matter experts, and teams should collect only data they will use.

Quality improvers can apply the practices of all five domains of the IHI Psychology of Change Framework in the process of determining measures and collecting and analyzing data. For example, make measurement people-driven by co-designing measures with the people affected by the changes and co-produce metrics that matter to them. Identify and share biases among team members before establishing measures and analyzing data. To distribute power, make data transparent. Only collect and track data that will be used for learning. Encourage people to measure by connecting measurement to their intrinsic motivations to learn and improve individually and together. Make measurement fun by incorporating play and celebration, while making failing forward transparent as adaptations of the measurement strategy occur. Use data over time to tell the story of improvement by annotating key events, changes tested, and other relevant information that adds context to the data. Share qualitative stories to connect data to people’s lived experience.

Last, although the quality improvement efforts described in [the Psychology of Change] white paper are successful examples of the applied tools and methods of the IHI Psychology of Change Framework, no single effort to date has evaluated the entire framework as presented. Intentional testing is necessary to understand the framework’s effectiveness, and to provide an evidence base for increased adoption and sustainability. IHI invites feedback on organizations’ experiences with the framework, tools, and measurement in order to identify bright spots, fail forward, and improve together.

To learn more about the psychology of change methodology, read the IHI Psychology of Change Framework to Advance and Sustain Improvement white paper.


You may also be interested in:

How to Activate the Power and Courage to Change

WIHI — How to Make Change Happen: An Introduction to IHI's Psychology of Change Framework

first last

Average Content Rating
(0 user)
Please login to rate or comment on this content.
User Comments