When an Important Problem Is Hard to Measure

For patients and families, they can mean delayed care, uncertainty, and frustration. For clinicians and hospitals, they often reflect larger challenges across the healthcare system.

The more difficult question is: How do we measure that problem in a way that leads to improvement?

For Julie Marcinek, DO — a family physician, primary care medical director, and member of the Partnership for Quality Measurement® (PQM) Pre-Rulemaking Measure Review (PRMR) committee — that question gets to the heart of quality measurement. A measure must address an important issue, but it must also be meaningful, feasible, and fair in the real-world settings where it will be used.

“These conversations are about risks, benefits, and tradeoffs. We are looking at the whole system,” Marcinek explains.

An Urgent Problem, a Complex Measure

During a recent review cycle, committee members discussed the Emergency Care Access & Timeliness (ECAT) measure, which addresses one of healthcare’s most visible challenges: emergency department overcrowding and long waits. The measure includes indicators such as wait times after triage, patients leaving without being seen, prolonged boarding after an admission decision, and extended emergency department stays.

The committee agreed that emergency department access and timeliness of care matter. Patient advocates emphasized that long waits are among the top concerns they hear from patients and families. 

But the discussion also raised a fundamental question: Should hospitals and emergency departments be held accountable for delays driven in part by factors outside their direct control?

Emergency department wait times can reflect problems throughout the care continuum. A patient may be waiting not because emergency clinicians are unresponsive, but because inpatient beds, staffing, post-acute services, or other resources are unavailable. A measure that captures a real and urgent patient concern must also account for where responsibility — and the ability to make improvements — actually lies.

The committee did not reach consensus on recommending the ECAT measure. Rather than signaling disagreement about the measure’s importance, the discussion highlighted critical questions about accountability, feasibility, and whether the measure could drive meaningful improvement without unfairly assigning responsibility.

For Dr. Marcinek, the ECAT discussion showed why quality measurement needs input from people with different experiences and perspectives. Patients and families can speak directly to the burden of long waits. Clinicians and health system representatives can help explain how a measure may function across settings, including those with different resources and operational challenges.

Looking at the Whole Measurement Landscape

Committee members do not review measures in isolation. They consider whether a proposed measure addresses an important gap, how it fits with existing measures and programs, and how it may interact with other incentives or create unintended consequences. 

The Excess Antibiotic Duration for Adult Hospitalized Patients with Uncomplicated Community-Acquired Pneumonia measure provided another example. Committee members supported its underlying goal to reduce unnecessary antibiotic use and prolonged treatment. But they also considered how the measure might interact with existing sepsis care requirements that encourage prompt antibiotic treatment.

The concern was not the measure's intent. It was that incentives intended to improve one aspect of care could create risks elsewhere in the care process. The committee ultimately did not recommend the measure.

That is a critical distinction in measure review. The goal is not simply to approve or reject measures. It is to understand how they may affect patients, clinicians, and organizations once they are implemented — and whether the likely benefits outweigh potential burdens or harms.

Why Practicing Clinicians Belong in the Conversation

As a practicing physician, Dr. Marcinek brings a perspective grounded in day-to-day patient care and the realities of implementing quality initiatives. 

“If we aren’t at the table expressing what these things look like for real patients, real offices, and real hospitals, then other well-meaning people with different perspectives will be making those decisions,” she explains. 

Clinicians are one essential voice, but they are not the only one. Patients, caregivers, researchers, health system leaders, payers, and others bring insights that can identify gaps, test assumptions, and help a measure evolve before it affects care more broadly. Their experience can reveal whether a measure will work across care settings, create an unreasonable reporting burden, or produce effects that are not apparent from technical specifications alone.

Making Better Measures Through Deliberation

For Dr. Marcinek, participating in PQM has been an opportunity to work with people who may approach a measure differently but share the same commitment to improving care.

“Participating in PQM gives me the opportunity to improve health outcomes for patients, contribute to infrastructure, such as payment reform, and make a positive difference for my patients on a national level.”

The discussions surrounding ECAT and antibiotic duration demonstrate that meaningful quality measurement is not a simple technical exercise. It requires careful examination of evidence, feasibility, patient experience, accountability, and the relationships among measures across the healthcare system. 

A measure is most valuable when it does more than identify a problem — it offers a credible, practical way to improve care. Getting there requires thoughtful deliberation, diverse perspectives, and a willingness to ask difficult questions before a measure ever reaches implementation.

Julie Marcinek, DO, is a family physician and primary care medical director for Vandalia Health in southern West Virginia. She has served on the American Academy of Family Physicians Commission on Quality and Practice and is a member of the Partnership for Quality Measurement’s Pre-Rulemaking Measure Review committee. Her experience includes population health, transitional care management, value-based care, and quality improvement.

Kaely Burgess is a Project Manager, Amy Minichino is a Project Manager, and Tara Bristol Rouse is a Senior Project Director at IHI.

Learn More

Battelle is a CMS-certified consensus-based entity and administers the Partnership for Quality Measurement® (PQM). To get involved:

• Become a PQM member to receive updates and engagement opportunities throughout the year.
• Submit public comments on proposed measures, process enhancements, committee rosters, and recommendations.

All committee and membership opportunities are free of charge. Questions? Contact PQM at PQMSupport@battelle.org.

Photo by Fleur on Unsplash

You may also be interested in:

• Quality Measurement Is the Starting Point, Not the Destination
• Getting Involved with the Partnership for Quality Measurement: A Beginner’s Guide