S4E2: Wellness ROI with Novel Data Metrics (ft. Gaurava Agarwal, Northwestern Medicine)

Jordan Cooper   2:01
Mhm.Mhm.

Agarwal, Gaurava   2:18
It wasn't. I didn't think it was going to be right to just take our physician playbook and apply it to now nurses or our environmental service workers. I didn't feel like the the interventions would land correctly. And so for us we had to figure out how what was our going to be our mechanism to scale.Interventions that mattered to individual work families. And so our flagship program that you mentioned is called the Scholars of Wellness. The idea behind this is that the folks that do the work, the frontline folks know how to do their work best.And know what the pain points are, what know what those work determinants of well-being are and how you might improve them. What they don't know is sort of how organizational change happens. They don't know the well-being science very well when they go into the program and they don't know how project management occurs at a local.

Jordan Cooper   3:06
2.

Agarwal, Gaurava   3:12
Institution. And so a lot of times you'll hear clinicians say, oh, I don't know why they just don't fix this one thing. I would fix this like that. And the truth is that's not how organizations get fixed. And so for us, what we had to do was make sure that we train these individuals in the Wellness science.

Jordan Cooper   3:18
Mm-hmm.Mm-hmm.

Agarwal, Gaurava   3:29
The process improvement, project management science and the change management science so that they can then take ideas that they have that are pain points for folks in their local environments and start to improve them and so.

Jordan Cooper   3:33
Mhm.Mhm.I think so. I think Gee, that what's most innovative and perhaps most relevant to healthy data podcasts is the way you kind of abstract custom metrics per specialty area. It seems like you have a federated method of solving issues locally within the organization and you actually.

Agarwal, Gaurava   3:44
Yeah, good.

Jordan Cooper   4:02
Force local individuals in radiology and pathology and ulmonology and the hospital is for and say what are the metrics that most matter to your particular secialty and that's you're generating your own data sources, is that right?

Agarwal, Gaurava   4:15
That's that's right. And the other thing I would add is we're we're generating complete data sources. So for instance, if I send someone an e-mail or the pathology department an e-mail, I at best I'm going to get a 30% response rate at best, but when one of their fellow colleagues sends out a survey or a a focus group request.

Jordan Cooper   4:21
Mhm.Mhm.

Agarwal, Gaurava   4:36
Or any other sort of data request. Some of the response rates we get are 708090% response rates and so we can trust the data far more because it's complete data. Otherwise there's a real risk that a lot of the data, the well-being folks around the country are using.

Jordan Cooper   4:42
Mhm.

Agarwal, Gaurava   4:53
Are incomplete 4050% response rates. Well, what are the other 50% feeling and thinking? Are we sure we know and can extrapolate based on the the 50% that responded to our data sources?

Jordan Cooper   5:06
So Gee, I think what I hear you saying is that it's not necessarily, at least in this example, a tech problem. It's a social engineering problem. And the idea is you need in order to get valid data that people trust, you need to get clinical buy in. In order to get clinical buy in, it's all about leveraging the right kinds of organization, the people within the organization.In order to get that response rate so that you have valid data that clinical leaders trust from their colleagues. Is that right?

Agarwal, Gaurava   5:33
That's exactly right. That's exactly right.

Jordan Cooper   5:35
Excellent. And so you know you spoke about one getting the actually, you know what, let's take a concrete example. Can you speak about one particular area of the organization could be radiology or another specialty and just give us a three-minute anecdote about particular metrics that you received that was that our value.To that to that area.

Agarwal, Gaurava   5:58
Sure. I'll give you an anesthesiology project, a project that we just worked on. You know, anesthesiology is a very, very tough field right now from a recruitment, retention, satisfaction standpoint. And our scholar highlighted that, you know, they signed up for anesthesiology knowing that the hours.

Jordan Cooper   6:01
Mhm.

Agarwal, Gaurava   6:18
Hours are tough. It's not like they they expected a nine to five Monday through Friday job. But what's also true is that those hours have started to become far less predictable. And so she said you can't just look at our shifts, you have to look at when the shift was supposed to start and stop.

Jordan Cooper   6:18
Mm-hmm.Mhm.

Agarwal, Gaurava   6:36
And then when, when do we actually leave the hospital? And the reason that's important is because the the the time card isn't getting it right. We're not able to leave when we when we thought we were going to leave, which has huge downstream implications in terms of the conflict that's being created at home. If I told my kid I was going to come home at six.

Jordan Cooper   6:47
Mhm.OK.

Agarwal, Gaurava   6:55
To go to their game and now I'm not there. You know, that's a whole lot of a lot of conflict. And what we found was that we were able to see who was actually leaving on time. And when they didn't leave on time, it was causing the exact conflicts this scholar had hypothesized would occur.

Jordan Cooper   7:02
Mhm.Mhm.

Agarwal, Gaurava   7:13
And then when they have those conflicts, those were the exact people that were experiencing the most burnout, were telling us that they were going to reduce their hours or telling us that they were frankly going to leave. And so it was this very elegant, beautiful data point that someone like me, sitting from where I do, would have no idea I should be where I should be looking for that data.

Jordan Cooper   7:13
Mhm.

Agarwal, Gaurava   7:33
Where she was able to pinpoint that this is actually the data point that matters the most.

Jordan Cooper   7:38
I really appreciate you delving deep into that anesthesiology project. I think that may resonate with many listeners across the country. Just real quick, someone listening to this show right now say that's amazing. You know, you validated that hypothesis that resonates. I've heard that maybe a CMIO was listening and saying, yes, my clinicians have also said something like this.

Agarwal, Gaurava   7:39
Yeah.

Jordan Cooper   7:57
Have you been able to present the intervention and how do you, now that you know this is a problem, how do you reduce burnout and help those anesthesiologists leave on time?

Agarwal, Gaurava   8:07
The power of the program is is in most organizations, pilots are doable. When you just say, hey, I'm going to change something and you don't really know how it's going to work, it's tough. And so the power of this was once we presented the data very clearly.

Jordan Cooper   8:16
Mhm.Mhm.

Agarwal, Gaurava   8:25
That this is a very causal link you're seeing. Then people were open to saying, well, let's do a two-month pilot. Let's do a two-month pilot where we have what's called a relief attending. A relief attending comes in at a set time when you were supposed to get off shift and relieves you of your duties. So you know that when.

Jordan Cooper   8:36
Mhm.

Agarwal, Gaurava   8:44
If this shift was supposed to end, you are going to leave because someone can take over your case. And so they implemented this relief attending. And what was interesting is that when you asked the anesthesiologist how many days a week would you like a relief attending?

Jordan Cooper   9:00
Mhm.

Agarwal, Gaurava   9:00
Probably to no surprise, everyone said every day. But when we looked at the data and we said how many days a week does it take to, at least on the well-being front, make the difference? It was four days a week. The folks that had had were a relief attending for four days a week.

Jordan Cooper   9:04
Yeah.

Agarwal, Gaurava   9:18
Or more were not having those work home conflicts at a rate that was sufficient enough to cause enough conflict, enough burnout. So if we had to choose, and sometimes you have to make these choices in life, can't four days was going to give you enough, even though obviously everybody wanted somebody to relieve them every single day and so.

Jordan Cooper   9:19
Mhm.And again.

Agarwal, Gaurava   9:38
That's the kind of thing that really resonates with organizations, that we're being efficient and thoughtful with our resources, not just more is better.

Jordan Cooper   9:45
Gee, I appreciate your answer to my question about that intervention with the pilot and the relief attending. I do want to get to the data analytics platform portion of the conversation, but one more question before we do. Some listeners may who are from the clinical space may say you know that sounds fantastic, you know my anesthesia.Aesthesiologist is supposed to get off at 7:00 AM and should be home to get the kids ready for school. And that's great. But what if they're in the middle of surgery or they're in the OR, you know? Or what if there's a handoff? Like maybe the relief attending starts at 7:00, but it takes 2540 minutes for me to do a handoff because there's some complex cases coming up.You know, I still can't get out on time. So do you have the relief attending come at like 6:00 AM? How do you handle the handoff and also like I'm in the OR up to my and I can't do anything?

Agarwal, Gaurava   10:37
I would say that I'm probably not the expert on on the the ways the anesthesiologists have operationalized the relief attending, but I would say that we all know what a handoff looks like and you sort of build that in to what it's going to take to have a clean handoff and or you build that in.

Jordan Cooper   10:41
Yeah, yeah. Mm-hmm.Hmm.Got it.

Agarwal, Gaurava   10:56
Into the predictability of when I'm going to get home. If I, you know, you sort of know that the handoff might take 30 minutes and so now I'm comfortable telling my family I'm going to add that 30 minutes in and and I'll know what time I'll get home. That is far more predictable than I have no idea when the surgery is going to end and I obviously can't leave until the surgery ends.

Jordan Cooper   11:05
Mhm.Yeah.Uh huh.Reliability. That's a real big take away for our listeners. Now let's transition to the data analytics platform. First, I'd like to allow you, can you mention the coal mine metaphor, please? I think that would resonate.

Agarwal, Gaurava   11:25
So my dad was a coal miner, and so the idea was that you hear a lot of us are familiar with the Canary in the coal mine. They took the Canaries down there. If the air was toxic, the Canary would die first. That would be the warning sign for the coal miners to get out of the coal mine.

Jordan Cooper   11:37
Mhm.Mhm.

Agarwal, Gaurava   11:45
And the the story I had heard was actually that the Canaries were viewed as pets by the coal miners. So they created this thing called a Canary resuscitator, where if the the Canary passed out, they would put it in the Canary resuscitator, pump it full of oxygen, the Canary would live and everyone would get to leave the coal mine.

Jordan Cooper   11:52
Mhm.

Agarwal, Gaurava   12:03
And that resonated with me a lot later because I'm a psychiatrist and I take care of of of impaired physicians especially and other healthcare workers. And one of the doctors that I was taking care of said, you know this, well, this stuff isn't gonna work unless someone decides.

Jordan Cooper   12:03
Huh.

Agarwal, Gaurava   12:19
That they don't want to just make stronger Canaries. Unless someone decides they want to have a healthier coal mine, it's not going to be enough for people to just try to make stronger and stronger canaries. And so that's been our guiding light or our North Star, whatever you want to call it, as to what it is that we do in well-being as we create healthier coal mines.

Jordan Cooper   12:24
Mhm.Mhm.Now, Gee, you said the data analytics platform has enabled you to see the entire coal mine across silos. Many listeners will know it's no secret that healthcare across the country at almost every organization, it has many different silos by specialty area, by vertical finance, HR, clinical.Areas and whereas we just spoke about the scholars of Wellness who in each specific specialty area have their own metrics, you're also interested in leveraging AI to look across the different silos and see if there are metrics that nobody's even really thinking.Might be worth working on to improve Wellness. Can you speak about the data analytics platform, how it works, what it does, and some of its data sources, please?

Agarwal, Gaurava   13:25
Yep. So the idea here is that Wellness for us in in the in a future state has to be an operations exercise. It can't be an initiative that sits in the side. It has to be embedded in the operations because if you're fixing coal mines to fix a coal mine that the operation of the coal mine is what you're.

Jordan Cooper   13:31
Mhm.Mm-hmm.Mm-hmm.

Agarwal, Gaurava   13:45
Dressing and what we found is that if if that's true, then we Wellness must be at Wellness data has to be the same kind of data that our operators are used to seeing. And so if our operators wanted to fix turnaround times in the operating rooms.

Jordan Cooper   13:54
Mhm.Mhm.Mhm.

Agarwal, Gaurava   14:03
If our operators want to improve access to care, for instance, they can look up at the their data that day, that moment, and know exactly how many ORS are empty. They can know exactly how many people discharged by noon.

Jordan Cooper   14:08
Mm-hmm.And.Mhm.

Agarwal, Gaurava   14:21
To to be able to to have all of this operations data. The problem is Wellness data has historically been a survey done maybe once a year that is responded to by maybe 4050% of the workforce if you're lucky.

Jordan Cooper   14:26
Mhm.Mm-hmm.

Agarwal, Gaurava   14:39
And so now you have this stale data that's incomplete. And when I gave the operators that kind of data and say, hey, your burnout rate in March was X, how are things going? They don't know what to do with that. Or hey, I, you know, you're right, it was that in March. I've been doing a lot of things.

Jordan Cooper   14:49
The.Yeah.

Agarwal, Gaurava   14:57
I don't know if it's making a difference. Maybe I'll wait 17 months until the next survey for you to tell me if if the changes we made worked. And so that wasn't gonna work to me. We couldn't. We could keep talking about Wellness as an operations thing, but until we gave them data in the same form that the operations folks use, I didn't think we were going to be able to do it.

Jordan Cooper   15:11
Mhm.M.

Agarwal, Gaurava   15:17
So we have been collating data sources including the EMR. So looking at a lot of signal data, looking at people's burnout scores, engagement scores, other well-being metrics, survey scores, having a bunch of HR demographic data, looking at their workload data as measured by.

Jordan Cooper   15:18
Right.

Agarwal, Gaurava   15:37
RV use looking at their paid time off data, what's the rest and recovery look like and putting that all together in a cloud to say how are these various things related? So for instance, everyone sort of talks about the EMR, electronic medical record being the source of all evil and the source of all burnout.

Jordan Cooper   15:48
Mhm.

Agarwal, Gaurava   15:56
And healthcare providers, it's actually been pretty difficult to find any signal data that that you can say this one signal data predicts the burnout is what's causing all the burnout. People have these sort of theories about it that has been difficult, but if you add signal data with paid time off data.

Jordan Cooper   15:57
Mm-hmm.Mhm.

Agarwal, Gaurava   16:16
Data with other data, now you're able to see things emerge, and particularly for us, what we see emerge are the work determinants of well-being. So for instance, when we did this with our nurses and this is all preliminary data.

Jordan Cooper   16:26
Mm-hmm.

Agarwal, Gaurava   16:32
To be to be sure, we found that nurses that had a certain level of PTO accrued, so meaning that they have all this PTO stored, they haven't been taking their PTO for various reasons. But if they have too much PTO hours stored, a certain number, their risk of being burnt out was 2X.

Jordan Cooper   16:40
Mhm.Mhm.

Agarwal, Gaurava   16:52
The nurses who had less stored, meaning they had been taking PTO more regularly. So now you can see my ask of an operating team. The managers on the unit is to say, hey, you know what?40% of your nurses have X number of PTO hours accrued, which puts them at double the risk of burnout. What I'd like from you is to not figure out or have to deal with what's burnout. How do I deal with burnout? All I'm asking from you.

Jordan Cooper   17:15
Mhm.Mm-hmm.

Agarwal, Gaurava   17:22
Is to take that 40% number and reduce it to 20%. Let's figure out some interventions that you know best that can get these nurses to take their PTO as they understand how big of an impact it's having on their health. That is what our operators are world-class at.

Jordan Cooper   17:29
Hmm.

Agarwal, Gaurava   17:39
That is how they know how to to to use data and effectively move the needle.

Jordan Cooper   17:39
I think dance.You know, I want to mention a an analog analogy that that may be a little bit outside the box for some of our listeners. I come from a political world and some people may remember that the Speaker of the House of Representatives in the 1980s is a man named Tip O'Neill, I think. And he had a famous expression that said all politics is local.

Agarwal, Gaurava   17:59
Mhm.

Jordan Cooper   18:04
And I feel like that may ring true at Northwestern Medicine. You're relying on your metrics locally on the people on the floor. What do you think needs to happen to improve retention and reduce burnout? And you're also relying locally with your data analytics platform. You're putting that data in the cloud and you're trying to allow them to.To implement. So you come up with the insight and then you go back to the people on the floor, to the local people in your healthcare organization and you say this is the insight, this is the recommended action, but I'll leave it up to you to figure out how your team needs to best get there. So instead of a top down directive.You give local people on the ground agency in the solution and not only did that agency allow you to get better completion rates when you had a local colleague asking them for responses, but now you have more agency and more buy in when you have local people figuring out how they're going to get from A to B.You define the outcome that you want to see.

Agarwal, Gaurava   19:06
That's exactly right. And you know what we say is we centralized the approach to decentralize the action, right? Because I do think the top down heavy-handed approach won't work. We'll just get it wrong to be honest with you. And so this really allows.

Jordan Cooper   19:11
Mhm.Mhm.Mm-hmm.

Agarwal, Gaurava   19:22
People to say, you know, we have to solve these problems differently at a small Community Hospital compared to our large academic Medical Center. The solution isn't going to be the same. Now having said all of that, if the problem, once they try to solve it, they can't solve it because there's a, you know, larger systems issue, then that's where someone like Mai's role.Can be important. We're having a seat at the table at the executive level can say, OK, we've tried all of these things to move the needle on this PTO problem. But the real reason we can't take PTO is when our staff asks for PTO and we're not staffed appropriately, then I can't give them the time off because someone's got to take care of our patients. Now it's a central problem.Now it's a central problem to say, hey, what are we doing around our staffing models that isn't allowing people to take the PTO that is their rightful benefit?

Jordan Cooper   20:02
Right. So Gee, we are somewhat approaching the end of this podcast episode. Now again, it is Healthy Data podcast, so I would be remiss if I didn't raise the the question of how do you aggregate, normalize and deduplicate these various data sources. You're identifying new data metrics, you're bringing in data from HR systems.Putting it in Databricks and Azure. You're getting data on PTO. You're getting data on attrition. You're getting all kinds of data from across the organization. How are you reconciling and integrating all of that data?

Agarwal, Gaurava   20:44
So and again, I want to be clear, this is a very preliminary work and what we're trying to do is work with our medical social science teams to help validate that the data isn't junk, right. So our data says, hey, this is what we think we're seeing. Now we're beginning to go into local areas doing qualitative.

Jordan Cooper   20:53
Mhm.Mhm.

Agarwal, Gaurava   21:04
Of interviews to say this is what our data shows. Is this the real lived experience for folks? Is this data have face validity? And and you know, because I think your point is well taken and what we have found frankly over the last two years is this data isn't as easy as just putting it together. It's dirty, it's messy.

Jordan Cooper   21:08
Mm-hmm, mm-hmm.Mhm.

Agarwal, Gaurava   21:24
And if you oversimplify the chances that you, you know garbage in, garbage out problem is real. And so we are beginning to see if it if it is valid, valid data and if it's actionable data and and hopefully so far things have been looking pretty good on that front.

Jordan Cooper   21:29
Mhm.And you mentioned the phrase, and this is very popular among health systems across the country. I want real-time operational data. I don't want an 18 month lag, a 12 month lag, whatever. So a lot of people are thinking about FHIR repositories in the cloud in order to get that data in real time. Are you leveraging any of those sorts of technologies in order to get real time?Data. How are you facilitating that real time data access?

Agarwal, Gaurava   22:05
I've got really good data analysts that help me make sure that you know for the things that are real time. So certain things you know as I said we have the survey data in this model that that is only asked once every whatever year it is. There's other data that search such as PTO accrual that the way they've created this database is live.

Jordan Cooper   22:10
Yeah.Mhm.Mhm, mhm.Mhm, mhm.

Agarwal, Gaurava   22:25
Live and basically updates on a weekly level, even a daily level to say what are the current PTO accruals we're seeing for our nurses across the system and that automatically refreshes in real time.

Jordan Cooper   22:37
Any final thoughts, Gee, to our listeners? Any recommendations? I'm sure many of the issues that you've raised today resonate with listeners to Healthy Data podcast. If they're intrigued and want to learn more, maybe they want to kind of put some of your ideas into practice to their own organization. What would you advise them? Or even what would you advise?Yourself in 2024, you know, a few years ago when you were earlier in this process.

Agarwal, Gaurava   23:01
I will say we've published the our methodology for how we created this data analytics team in the Rhode Island Journal. It's worth looking up so you can kind of see our process. We've tried to be very transparent on what we're trying to do. So if folks are interested in recreating that model, I think that our publication is out there.

Jordan Cooper   23:15
Mhm.And.

Agarwal, Gaurava   23:20
I think for me I am very interested in.Moving towards this model of objectivity around Wellness, I think it'll be taken seriously. Obviously during the pandemic, everyone cared about Wellness because it was pretty obvious as we move forward is going to take data just to remind people how Wellness is related to safety, quality.

Jordan Cooper   23:29
Mhm.Mm-hmm.

Agarwal, Gaurava   23:42
In all the metrics that we care about in healthcare and it's our it's incumbent upon us to use that data to tell those stories and figure out where we can improve that coal mine.

Jordan Cooper
Stopped transcription