From 3M Health Information Systems
Podcast Episode Transcript: Measuring what matters in health care
Dr. Gordon Moore: Welcome to 3M’s Inside Angle Podcast. Today, I’m speaking with Dr. John Wasson, a physician colleague, with whom I’ve worked in the past and have had interesting conversations with over the years. Welcome to the podcast, John.
Dr. John Wasson: Thanks, Gordon.
Gordon: I would like to talk to you today about some work that you’ve titled Kiss quantophrenia goodbye. I want you to explain what do you mean by that title?
Dr. Wasson: First of all, let me apologize to any listener who might have heard me talking about organized crime and disorganized healthcare and then say, “This guy is a real curmudgeon.” I hope this conversation will not sound curmudgeon-like. Instead, KISS—if you remember back in the day we used to do that to say, “Keep it simple stupid” and this happened a lot when we were working in our Institute of Healthcare Improvement projects to make healthcare meet patients’ needs and we constantly say, “Keep it simple. Keep it simple. Keep it simple”.
Unfortunately, healthcare in the last 30 years has gotten a bad, bad, bad case of quantophrenia and that’s what we need to talk about today.
Gordon: Tell me what quantophrenia refers to.
Dr. Wasson: Well, first of all, it could be your bumper sticker that people would chase you down the street and say, “What does that mean?” It’s a term that was first popularized—in a narrow sense—in the field of sociology, I’d say, 50, 60 years ago. It referred to measurement quantification for its own sake, making quantification the value rather than, what are we really trying to do? If you look at the amount of measurement now in healthcare, both measurement for so-called valued-based purchasing, or quality of care, or regulation—or measurement for reasons that are totally unclear—it’s just out of hand.
Gordon: Yeah, I’ve heard a lot—I remember in my own practice that the measurement around quality was fascinating and fun, but after I covered a couple of different chronic conditions, it became exhausting. I’ve read a lot of articles around physician burnout, dissatisfaction, with a lot of fingers pointing at how hard it is to document everything that’s required for measurement and quality. That sounds like where you’re going with this and if that’s true, first verify and then tell me, what do we do about it?
Dr. Wasson: Well, yes, that’s true. In fact, the estimate is that if you look at five highly-trained physicians who’ve spent 12 years being trained, one out of those five you might as well take off the shelf and put him or her in a backroom and hand them paper, because that’s how many are being withdrawn. Out of every five, one doc is being withdrawn nowadays to do paperwork. It is a huge waste, not only of doctor’s training, but a huge waste for society. So that’s what we need to be focusing on. I’m not alone, obviously, in pointing this out. The Institute of Medicine and many of other prestigious organizations have done the same.
The problem, I see, is built in to the fact that they want to do the changes from the measurers down. In other words, “Oh golly, why don’t you whittle down some of the measures?” The point is, it’s such a mess, such a huge collection of quantophrenia that whittling down won’t work; it just won’t work. Instead, you’ve got to start asking, “What can we do from the bottom up to make it simple?” That’s the gist of this talk now, is to share some of the work we’ve been able to do and show its validity.
Gordon: You’re saying that the current approach to measurement is so vast and there’s so many matrix in there that we can’t start the conversation with saying, “Let’s knock some of these off the list; it’s too big.” We have to basically wipe the slate clean and start again?
Dr. Wasson: Yeah, yeah, it’s unfortunately almost impossible. If you think about the politics behind whittling down from thousands and thousands of measures that are out there now, whittling down is a political process. You’re going to have certain organizations say, “Oh, we’ve always liked that” and you’re going to have certain researchers say “but that’s my instrument.” You get the idea. It’s just not going anywhere, and there are groups doing that right now.
We started, a number of years ago, building from the bottom up and I think we have a pretty exciting and useful approach now which, I might add, could easily be paid for by Medicare, CMS, using current payment mechanisms if they adapted to what we’re promoting.
Gordon: Why don’t you give me some insight into where your research has taken you for measuring important things?
Dr. Wasson: Well, where the research is taking us is to simple, simple measures. I’m going to talk about six of the seven right now. One of the seven I won’t talk about is the single measure of poverty, but we know that everything we do in health care and in society in general, is influenced to a great degree by your financial status, so you’ve always got to have some measure of that type of determinate. Some people call a whole bunch of those social determinance of health. It’s simplest just to say, “Nay, just keep it simple.” It’s all about poverty and everything else falls under that. I’m going to leave that one off the table and go with six others.
Let me start with the first one that was researched by a number of your colleagues in primary care practices. A number of your colleagues, years ago, said, “Why are we doing the CAHPS survey?” They’re multiple items. Vendors come in sometimes they nag the patients, but the response rates are miserable. Typically now, about one of every five patients will respond. When we finally get this report, months later, we don’t know what to make of it. It says, “Yeah, your quality is good or your quality is not good.” Then sometimes somebody will even adjust our payment on it, but we don’t know and we don’t trust it.
What the group did is said, “Why don’t we just see if it makes any difference having one measure versus all this CAHPS stuff?” The study was done and at the end of the study it showed that ranking was virtually identical whether you used multiple item CAHPS or a single item asking patients, “When you think of your healthcare, are you receiving exactly the care you want and need exactly when and how you want and need it?” That single item, although it sounds like a mouthful, was actually put forward by a former acting head of CMS or Medicare, Don Berwick. That single item, does just as well as multiple item CAHPS. All those multiple item CAHPS, by the time you go through the costing, it’s about $40.00 per completed survey; whereas, asking patients a single item is no cost at all.
Gordon: It’s interesting. I can just sense the degree of challenge from colleagues of ours as they hear that boiling the complicated CAHPS survey down—CAHPS is Consumer Assessment and Health Plans and this is one of the surveys that has to do with, “What’s the experience of your care as a patient in whatever setting.” If we’re talking about the outpatient setting, you’ve studied the relationship of all the different matrix of how hard is it to get an appointment for an urgent visit, how hard is it to get an appointment for a routine visit, waits and delays in the office; all sorts of questions that get into pretty granular detail.
You would think with that detail I, as a practice, could see how I perform, relative to others, and use that information to improve things. So it’s valuable and useful information and yet you’re saying that when you studied this all those questions had such a high relationship to this one that they’re really not discerning. Tell me more about that.
Dr. Wasson: Well, you summarized it very well. The bottom line is if you look at it from a patient’s point of view. If you’re happy with the practice, you’re generally going to be happy with all the things you just described: access, efficiency, communication, etc. If you’re not happy, you’re going to say all of these things were bad. Now, yeah, you might in theory be able to discern if someone spent hours with you, “Well, yeah, come to think about it, maybe on balance they were a little more efficient than I gave them credit to.”
But when you look at the actual data that comes in from patients and you think about us going through life at this point, we generalize; and in fact corporations everywhere now are putting out kiosks that ask a single question, “How are the bathrooms in the airport?” They don’t say, “Was the toilet a problem? Was the paper out?” Just, “How was it?” Because they know if you didn’t like it, they got to look at the bathroom no matter what.
Gordon: I have a sense now, okay, I can ask a person—I understand what you’re saying around the global attribution of: I like the way they treated me. I didn’t like the way they treated me. As a person, an average consumer of healthcare is going to have that broad sense and apply it to a whole set of questions, glossing over the discernment that we’re trying to get at.
What do I do if I see that I take a one-question approach and it’s less than I’d hope as a practice now? How do I then get into the level of detail? Why would I want to lose that information? I guess the answer is, because it’s not discerning at its start; therefore, it’s not real information. So I’m still going to have to do work to see if I’m getting a bad score.
Dr. Wasson: Yeah.
Gordon: Why and then get into it?
Dr. Wasson: And more importantly let’s back up. That particular measure, just like all of the measures that are sucked out of your medical record about clinical benchmarks, those type of measures: experience of care and clinical benchmarks, are when you graft them related, but not very strongly related. Benchmarks are related to how patients rate their quality of care and vice versa.
Yeah, and organizations who are paying for care can draw quadrants on those data and say, “Ah, you’re good on those and you’re not good on those, therefore, we’ll pay higher and those of you who are lower paid, you ought to get your act together.” Well, that’s all well and good but that’s really not what we are about in healthcare. That’s where all our money is going. That’s the quantophrenia. We’re doing all of this process measurement and missing the key point.
When you ask patients about what matters to them—and that’s who we should be serving: patients—you see across 100 patients a variation going from 20 to 90 percent of the patients saying, “The doctor or nurse is aware of what matters to me.” A huge amount of variation and that’s where the real waste is occurring, because by not knowing what matters to my patients, I’m going to be applying resources they may be very efficiency applied, the processes might be perfect, but they’re going to have no relationship to the patient’s quality of life. So that’s where we decided to focus.
We said, “Yeah, we’re going to go with one measure for quality, one measure for social determinance, and that’s to keep everybody out there who’s still doing money measurement happy.” If they want to suck benchmark data out of our records that’s fine they can do that, but that’s not where the action is. The action has to be: let’s simplify and standardize the interaction with our patients so that they and we are on the same page and we’re avoiding unnecessary hospitalizations, etcetera, etcetera. That’s where the real money is.
Gordon: Let’s go to the next layer around measuring what matters. You started with seven questions and we’ve already talked about the money and we’ve talked about the global experience of care matrix. Are we down to four or five questions?
Dr. Wasson: Yeah, the “What Matters Index” is five questions. What we did over the last decade with hundreds of thousands of patients responses, and lots of literature reviews, etc—and control trials—we found that five items really work exceedingly well at number one, being very good proxies for patients quality of life. That’s what we should be about when we’re serving patients: “What can we do to improve their quality of life?”
In addition to that, those five questions are very efficient, easy to act on without having to go through any fancy scoring or using computers, etc. The five questions are simply: How confident are you that you can manage and control most of your health problems and concerns? Clearly 100 diabetics who say they’re not very confident are going to have much lower benchmarks for diabetic control than patients who say they are confident.
The next question is: How bothersome is the pain? Obviously, there’s more wording than I’m using right here on the radio, but it’s a single item about pain, a single item about emotion: Are you on multiple, multiple medicines? because we know those can be problematic with interactions. Then finally: Do you think your medications are making you sick? Those are the five questions: confidence, pain, emotion, polypharmacy, and are your medicines making you sick?
Gordon: I want to know about the validity of a person telling me that they have confidence. This is a challenge that comes up a lot when I talk to colleagues about this kind of thing where they say, “People don’t really know, or they’re going to blow smoke and they’re going to express confidence when they don’t actually have competence to be effective at taking care of their conditions.”
Dr. Wasson: Yeah, this comes up, that type of statement comes up again and again in almost every arena in which patient voice is listened to. The skeptics will say, “Well, I had a patient who didn’t know their blood sugar. Why should we ask patients how well they’re controlling their blood sugar?” The answer is, “Because for 80 percent of the time, it works and it’s so much easier to do that than to try and hook into medical records that have blood sugar data from four months ago and use that for today’s care.” The same with health confidence.
Gordon: Actually, before you go to health confidence, I just want to go back to the blood sugar measurement. There’s a big burden right now in practices when they have to find the hemoglobin A1c value, aggregate that, make sure it meets the inclusion/exclusion criteria and submit that to national groups that measure quality, maybe in the quantophrenia mode.
You’ve studied in the past the relationship between those aggregate hemoglobin A1c values and people’s response to: How’s your blood sugar been? when you’ve given them a range of responses. How strong is the relationship between the aggregate person reported response versus actual lab value of A1c?
Dr. Wasson: Exceedingly strong in almost everything that we, and others, have looked at. Obviously, recall for events that have happened a long time ago drops off depending on the nature of the event, but if you’re asking about blood sugar, it’s a whole lot more helpful, right? When you’re in clinic today with a patient and you say, “How’s your blood sugar been in the last week or so?” That’s going to help you decide what to do today—and we do that all the time—instead of pulling out with months and months of lag, some sort of number from the past, and using that as though it were the better reality.
It’s like forecasting tomorrow’s weather based on a random day weather experience from three months ago. It just has very little face validity. So, yes, there will be some patients who respond, depending on how you ask them, in dishonest or not-remembering ways, but on balance it works very well.
Gordon: I think about measurements having different sources of data and import, depending on what I’m trying to do. So if I’m thinking about what’s the aggregate burden of the A1c, or how well am I managing glucose, I have now two pathways. I can get the data out of the EMR, and that could take some time and effort, or I could ask people with diabetes in the practice, and those are equally efficacious ways of getting to the answer.
If I’m going to ask that question as part of a health risk assessment, for instance, it may obviate the need of the technical lab interface response and therefore reduce the burden on practice, which is part of reducing the quantophrenia approach. I’m attracted to that, but let’s get back now to the other aspects of the questions that matter for people.
You mentioned what’s your capacity to be effective at managing—the self-confidence question. You talked about pain, emotions, how many meds, and are your meds making you sick? What’s interesting to me is that it sounds like each and any one of those questions is kind of like a quick assessment which then tells me: do I need to go further? Then I can take action which, to me, is the level of work that is important at the human interface.
When I’m working with a patient, this is where now I have work to do; I need to get going. Those questions are the start of the work. I have a binary at that top level to say, “I have work to do, or I don’t have work to do.” Is that how you think of using this?
Dr. Wasson: Yes. So the What Matters Index in practice can be used and should be used in several ways. First, it’s totally geared to getting clinicians and patients on the same page. In other words, Gordon and John are both going to get the same standard questions answered by their patients. So the unwanted variation between John and Gordon, in terms of actions taken, is going to be minimized—the variation is going to be minimized, because we’re all using the same standard questions.
Secondly, as you implied, those questions are so clear in intent for action that you can begin to standardize the actions as well. In fact, using HowsYourHealth.org as an example of information and communication technology, for a patient who has no What Matters Index problems you can pat them on the head and say, “You’re doing great.” It turns out that if you look at diabetics, for example, who have a WMI of zero, only four percent will have a blood sugar out of control—four percent. So on balance, don’t lose sleep over it and certainly don’t go pulling medical records trying to pull up blood sugars on patients with low WMIs. They’re all going to be fine—almost all—just leave it alone.
Now, if they have one WMI question, most often it’ll be the confidence question, but it could be any of the others. There about eight percent of the patients have an abnormal or out of control glucose levels, blood sugar levels, in the recent past. There you’d say, “Well, I want to pay a little attention to that one item that the patient answered” and hopefully by knowing that item I’ll indirectly get the sugar, but most importantly I’ll be dealing with what matters to the patient.
The patient may be lacking confidence and I’d be asking, “What would it take to make you more confident? Well, I need to do X, Y, or Z.” You get the idea. Once you get up to two or more on the WMI, or What Matters Index, then the percent of patients with abnormal blood sugars is about 20 percent. So one out of every five is going to be out of control with their diabetes.
In addition to that, their quality of life, as you’d expect, is going to be a lot less and they’re going to need a lot more time from you; and most importantly, we documented it in both Medicaid and private practice patients that those were the WMI, greater than or equal to, have at least a two-fold increase odds of being hospitalized or placed in the emergency room in the next year. These folks are going to need much more attention. That’s a very rational way to stratify care, know exactly what the issues are, and begin to address them and, hopefully, increase quality of life.
Gordon: That’s very interesting to me in the sense of I have two things that I want to think about as a clinician in practice or as a medical director over a group of practices. One is: I want to work with individuals and help them work on what’s important to them in terms of quality of life and healthcare and the stuff that I’m doing. So I’m going to use this screening tool to find out what’s important to them and where things are going wrong. I’m also going to go beyond that and say, “Here’s preventive stuff you ought o be doing.” I’ll probably still measure their blood sugar if they have diabetes and measure their blood pressure if they have hypertension.
The second piece of work is thinking: who’s out there in my practice group who maybe lost a follow up not coming in or I’ve touched on them, but not in the last six months or so. This is a way to say these individuals are at greater risks. So if I have limited resource and time, maybe I should focus on those who are at greatest risk of things going bad for them and the index can help me with that. I may start just with the most extreme people with the greatest number of questions going in the wrong direction. Have you seen that work in practices? Have they used that kind of strategy?
Dr. Wasson: Well, what we have designed and, again the practices often do help do the design and you’re articulating exactly the type of thing they want, which is: I don’t care about the patients who are coming in the office as being difficult to reach. I can ask them to complete the WMI before they come to the office. But what about those others out there I haven’t seen? Again, the WMI is so short it is easily used on the smartphone or on paper, etcetera. It’s very easy to reach out. As a matter of fact, you could put it in Ladies Home Journal and get it across everyone who reads that magazine, or the Wall Street Journal. The WMI offers a lot of potential in terms of its ease of administration and since it doesn’t cost anything, that even better.
One last point is that HowsYourHealth.org, we have now the WMI available so you can immediately use it as your screener for patients who want to use the smartphone or before the office visit, complete it on the computer, and you can customize it. You can get it mailed to your practice so that you can add additional questions to the WMI as well. In other words, it has all the functionality of the full How’s Your Health Assessment that you would routinely want to give to patients who have a WMI two or higher. In fact, the automated WMI does a lot of that work for you.
Gordon: We’ve got an approach to a person coming into a practice or to whom I want to reach out using the tools that assess at the high-level five to seven questions and then in the more detailed level with HowsYourHealth.org. Now I want to loop it back into the quantophrenia discussion. What we’ve been talking about it how to keep it simple, measuring what matters.
If I’m standing back now as a policymaker or I’m working on a health plan or I’m an employer telling the employees to do this or that or choosing a different plan o work with, how can I have confidence that the What Matters Index, or something like that, is telling me that the quality of care delivered by those practices is adequate? Would you go there? Is it possible to use the tool for that and if so, why is it valid compared to some of the other things that are out there?
Dr. Wasson: Well, I think that is where we are right now today. We have on the one hand incredibly expensive wasteful approaches. I alluded to or mentioned the CAHPS survey—one in five response rates. It’s just silly and it’s very costly, $40.00 a completion—somebody is paying for that and it’s driving clinicians out of practice.
We are sucking ancient benchmark data out of medical records and using that to pay clinicians along with the CAHPS survey and assuming somehow that Gordon Moore, in his wisdom, is going to be able to take some sort of scores from that and figure out how to make health care better when he’s not on the same page with his patients, in terms of what matters to them, and he’s getting no feedback on that. The current system is a mess. It’s expensive and it doesn’t serve what matters to patients and, therefore, it’s not quality of life enhancing for patients.
The status quo and, in particular the latest stratification using so-called predictive analytics, is very, very expensive. So back to your head of an ACO or an insurance company or a group practice, they need to just do a head-to-head comparison on: no, I don’t care, 100 patients, six practices, etc, and compare something that has no cost—the WMI is free online or they can print it out themselves or put it in their own EMR template. It’s free. It serves quality of life—and compare all the costs of using a WMI-based approach versus the status quo. Count the dollars, count the hospitalizations, count the waste of time, and see which wins. It’s unlikely, given the zero cost of what I’ve just described, that it’s going to lose.
Gordon: Well, John, that is a challenge that I am hoping that our listeners are up to testing. They can follow the threads that we’ll link at the bottom of the podcast so that they know where to pick that challenge up and actually give it a try and give it a head-to-head comparison.
We didn’t get into details on the difference between measuring primary care, on the basis of a handful of conditions specific metrics, versus global. That’s been sort of implicit and there are other podcasts where I’ve had the opportunity to speak with folks like Bob Berenson and Harold Miller, who addressed that kind of thing. We’ll provide links to those as well. I want to thank you for your time today.
Dr. Wasson: Thanks, Gordon.