Transcript:

0:0:0.0 –> 0:0:7.600
Jordan Cooper
We are here today with Doctor Rebecca Macheras, the chief Medical Information Officer and Vice President at Mass General Brigham Hospital, Mass.

0:0:7.610 –> 0:0:14.350
Jordan Cooper
General Brigham is a health system based in Boston, MA for 3300 beds in 16 hospitals and 15,000 providers are Rebecca.

0:0:14.360 –> 0:0:16.100
Jordan Cooper
Providers are Rebecca.

0:0:16.110 –> 0:0:17.440
Jordan Cooper
Thank you so much for joining us today.

0:0:17.450 –> 0:0:17.980
Jordan Cooper
How are you doing?

0:0:18.710 –> 0:0:19.250
Mishuris, Rebecca G.,MD, MPH, MS
I’m great.

0:0:19.260 –> 0:0:20.180
Mishuris, Rebecca G.,MD, MPH, MS
Thanks for having me, Jordan.

0:0:21.20 –> 0:0:21.870
Jordan Cooper
Yes, great.

0:0:21.880 –> 0:0:22.110
Jordan Cooper
Good.

0:0:22.170 –> 0:0:22.970
Jordan Cooper
Glad to have you.

0:0:22.980 –> 0:0:31.190
Jordan Cooper
So we’re gonna cover a wide variety of topics today, and we’re gonna start with discussion on clinician facing artificial intelligence tools.

0:0:31.360 –> 0:0:35.630
Jordan Cooper
AI has been in the news over the last year with the emergence of chat GPT.

0:0:35.670 –> 0:0:38.730
Jordan Cooper
Now there are large language models, also known as alms.

0:0:39.160 –> 0:0:48.800
Jordan Cooper
I’d like to start with the discussion about not only what the hype is and what’s in the news, but what’s actually going on with AI in health systems today.

0:0:50.260 –> 0:0:56.430
Mishuris, Rebecca G.,MD, MPH, MS
It’s a big topic obviously, and and you know, AI has been around in healthcare for a long time.

0:0:56.440 –> 0:1:3.610
Mishuris, Rebecca G.,MD, MPH, MS
I think the advent of generative AI is is truly novel though, and and new in the last year or so.

0:1:3.930 –> 0:1:13.130
Mishuris, Rebecca G.,MD, MPH, MS
What we’re seeing today is the use of generative AI in facilitating administrative tasks mostly is where we’re starting.

0:1:13.680 –> 0:1:16.930
Mishuris, Rebecca G.,MD, MPH, MS
Now, that’s not to be minimized at all.

0:1:17.40 –> 0:1:41.270
Mishuris, Rebecca G.,MD, MPH, MS
As clinicians, we are inundated with these administrative tasks, everything from writing our notes, which which actually has some real clinical value right to things like prior authorizations where reasonable people can disagree about whether there’s clinical value in those or not, um and and things that are both clinician facing and patient facing like patient portals and and responding to patient messages through the portal.

0:1:41.280 –> 0:1:43.70
Mishuris, Rebecca G.,MD, MPH, MS
And so we’re generative.

0:1:43.80 –> 0:1:47.730
Mishuris, Rebecca G.,MD, MPH, MS
AI is starting now is is trying to tackle some of those more administrative tasks.

0:1:47.960 –> 0:1:55.850
Mishuris, Rebecca G.,MD, MPH, MS
I think the where everyone’s hoping it will go eventually is to help us in our clinical work as well, but we are nowhere near that yet.

0:1:56.870 –> 0:2:6.920
Jordan Cooper
So before we go into a variety of different, I think we may go into a survey of of different sort of AI tools that are available in large health systems today.

0:2:7.430 –> 0:2:10.460
Jordan Cooper
I do wanna kind of backtrack and get context.

0:2:10.470 –> 0:2:24.980
Jordan Cooper
A lot of the promise of electronic health records had been to automate clinician workflows to free up their time to enhance the patient provider interaction and improve health outcomes.

0:2:24.990 –> 0:2:31.70
Jordan Cooper
And of course, the reality over the last 20 years of EHR implementation sometimes has not borne that out.

0:2:31.80 –> 0:2:51.530
Jordan Cooper
There are many clinicians have expressed that there have been longer working hours, more administrative time and more screens getting in between the provider and the patient interaction within that context, would you care to reflect upon the AI’s potential to actually deliver on the promise originally made by HR?

0:2:52.110 –> 0:2:57.840
Mishuris, Rebecca G.,MD, MPH, MS
Yeah, I I think it actually has huge potential in ways that we have not seen to date.

0:2:58.170 –> 0:3:11.640
Mishuris, Rebecca G.,MD, MPH, MS
So to date, we rely a lot on customization, personalization of workflows, making sure that staff are doing what they’re supposed to be doing within the EHR at the appropriate times, trying to get everybody to work to the top of their license.

0:3:12.90 –> 0:3:20.250
Mishuris, Rebecca G.,MD, MPH, MS
But what we’re seeing, as you correctly pointed out, is that there’s a lot of clicking, right, and it’s very different than if I were handwriting a note in my Manila folder.

0:3:21.210 –> 0:3:30.840
Mishuris, Rebecca G.,MD, MPH, MS
Umm Pre EHR and you know the the promise of EHR’s was not just to digitize my Manila folder and make my handwriting more legible, although it’s done that.

0:3:31.150 –> 0:3:33.670
Mishuris, Rebecca G.,MD, MPH, MS
But it was to be able to harness data, right?

0:3:33.680 –> 0:3:50.800
Mishuris, Rebecca G.,MD, MPH, MS
Be able to harness data to improve patient care and improve patient outcomes, improve safety, improve quality, improve equity and unfortunately we’re like bogged down in the minutia of clicking and screen switching and and task switching of the EHR.

0:3:50.810 –> 0:3:56.590
Mishuris, Rebecca G.,MD, MPH, MS
And we haven’t gotten to the point yet where we can actually harness all of that data and improve care outcomes.

0:3:57.430 –> 0:4:6.900
Mishuris, Rebecca G.,MD, MPH, MS
The fact that we’re starting with administrative burden with with generative AI is actually really exciting because it means we can finally get to the rest of those things.

0:4:7.230 –> 0:4:20.640
Mishuris, Rebecca G.,MD, MPH, MS
And so, you know, I think that we have been doing our best with trying to improve workflow within the EHR, improve training, improve optimization, you know enhancing the the technology itself.

0:4:21.220 –> 0:4:21.450
Jordan Cooper
Umm.

0:4:20.930 –> 0:4:26.420
Mishuris, Rebecca G.,MD, MPH, MS
But to really move the needle, I think we’re going to see over the next, let’s call it, three to five years.

0:4:27.80 –> 0:4:44.810
Mishuris, Rebecca G.,MD, MPH, MS
I’m a a shift in how we’re able to do that because of things like generative AI and large language models where they are going to be able to take a lot of the kind of more algorithmic work that we do out of our hands and and just be able to do it.

0:4:44.860 –> 0:4:50.410
Mishuris, Rebecca G.,MD, MPH, MS
I think right now the technology is new enough that no one’s quite comfortable enough to just let it fly.

0:4:50.870 –> 0:4:51.100
Jordan Cooper
Mm-hmm.

0:4:50.580 –> 0:4:55.220
Mishuris, Rebecca G.,MD, MPH, MS
And so we’re all piloting these technologies trying to understand the impact.

0:4:55.230 –> 0:5:4.580
Mishuris, Rebecca G.,MD, MPH, MS
We still very much have human in the loop with the technologies that we’re piloting today because we’re dealing with patients lives right and and we don’t want to get it wrong.

0:5:5.470 –> 0:5:17.420
Mishuris, Rebecca G.,MD, MPH, MS
But once we get it right, it will have changed the trajectory of how we’re able to address the administrative burdens that to some extent the EHR has imposed upon us.

0:5:17.430 –> 0:5:21.220
Mishuris, Rebecca G.,MD, MPH, MS
But but to some extent, we’ve actually done to ourselves in various other ways, right?

0:5:21.230 –> 0:5:33.80
Mishuris, Rebecca G.,MD, MPH, MS
So the EHR is often the face of a lot of other things that have gone on regulatory requirements, safety checks, things which people actually really should want, right, like we wanna be safe.

0:5:33.90 –> 0:5:34.910
Mishuris, Rebecca G.,MD, MPH, MS
We wanna provide high quality care.

0:5:35.0 –> 0:5:43.410
Mishuris, Rebecca G.,MD, MPH, MS
Some of those regulatory requirements have come down to ensure that we are providing high quality care and ensure that we are providing equitable care.

0:5:44.60 –> 0:5:53.420
Mishuris, Rebecca G.,MD, MPH, MS
Umm, but they are coming through the EHR and so people look at it as a an EHR burden rather than a regulatory burden or a safety burden.

0:5:53.750 –> 0:5:57.580
Mishuris, Rebecca G.,MD, MPH, MS
And and and I guess you know calling them burdens probably isn’t quite right either, right.

0:5:57.590 –> 0:5:59.350
Mishuris, Rebecca G.,MD, MPH, MS
Like we should wanna be providing safe care.

0:6:0.10 –> 0:6:0.330
Jordan Cooper
Mm-hmm.

0:6:16.380 –> 0:6:16.690
Jordan Cooper
Mm-hmm.

0:5:59.810 –> 0:6:31.370
Mishuris, Rebecca G.,MD, MPH, MS
And but it’s coming through the EHR and so to the extent that we can remove some of the administrative tasks that we’re doing or remove some of the things, you know some of the clicks were making to to be able to report to regulatory bodies that we’ve done something because now the AI can just troll the chart and figure out that I did it without me clicking a button that I did do it that will remove the barriers to providing care and the barriers to being able to get to the promise of the EHR, which is to deliver better care.

0:6:32.570 –> 0:6:35.980
Jordan Cooper
So when you’re, we’re talking about a few different topics here.

0:6:35.990 –> 0:6:44.100
Jordan Cooper
One is the potential for impact on wellbeing of the provider and as addressing provider burnout, but obviously also patient outcomes.

0:6:44.110 –> 0:6:55.760
Jordan Cooper
And you’ve been talking about how we may evaluate some of that, you said maybe by having an AI eliminate the need for a provider to click a certain button in order to demonstrate that a task was completed.

0:6:55.950 –> 0:7:1.980
Jordan Cooper
Can you speak more about the potential for impact on wellbeing of both the provider and the patient?

0:7:1.990 –> 0:7:4.990
Jordan Cooper
And from there, I’d like to pivot into evaluation.

0:7:5.0 –> 0:7:9.920
Jordan Cooper
Obviously you can’t improve what you can’t measure, so impact and well being.

0:7:10.410 –> 0:7:11.480
Mishuris, Rebecca G.,MD, MPH, MS
Yeah, absolutely.

0:7:11.490 –> 0:7:11.880
Mishuris, Rebecca G.,MD, MPH, MS
So.

0:7:11.930 –> 0:7:23.240
Mishuris, Rebecca G.,MD, MPH, MS
So for me as a CMIO, this is a space I live and breathe in the intersection between technology and the clinician experience, and the intersection between technology and the patient experience.

0:7:23.250 –> 0:7:28.800
Mishuris, Rebecca G.,MD, MPH, MS
And you know, we know that our providers are burnt out.

0:7:29.370 –> 0:7:33.190
Mishuris, Rebecca G.,MD, MPH, MS
We know that is increasing over time for many reasons, right?

0:7:33.200 –> 0:7:36.60
Mishuris, Rebecca G.,MD, MPH, MS
Not the least of which was the last three years of a pandemic.

0:7:36.220 –> 0:7:38.720
Mishuris, Rebecca G.,MD, MPH, MS
And there there are a lot of drivers of burnout.

0:7:38.730 –> 0:7:43.330
Mishuris, Rebecca G.,MD, MPH, MS
The EHR is often cited as like the top five reasons for burnout.

0:7:43.420 –> 0:7:45.290
Mishuris, Rebecca G.,MD, MPH, MS
It’s not quite true though, right?

0:7:45.340 –> 0:7:50.350
Mishuris, Rebecca G.,MD, MPH, MS
As I was saying before, the EHR’s, often the face of other things that may be contributing to burnout.

0:7:50.440 –> 0:7:58.890
Mishuris, Rebecca G.,MD, MPH, MS
We also know that things like, you know control over 1 schedule is a huge driver burnout or lack of control.

0:8:0.70 –> 0:8:0.250
Jordan Cooper
Umm.

0:7:58.900 –> 0:8:7.90
Mishuris, Rebecca G.,MD, MPH, MS
I should say and, but the EHR is certainly one of the, let’s say, called top 10 reasons for people to be burned out, and it’s not the EHR itself.

0:8:7.100 –> 0:8:21.780
Mishuris, Rebecca G.,MD, MPH, MS
It’s the fact that I have to click the button to say that I did something that I already did right and so and what we’re really interested in is whether the AI that’s coming in to alleviate the administrative burden is actually going to improve.

0:8:21.790 –> 0:8:22.380
Mishuris, Rebecca G.,MD, MPH, MS
Well being.

0:8:22.430 –> 0:8:26.620
Mishuris, Rebecca G.,MD, MPH, MS
Is it going to reduce how much time I’m spending it in the EHR overall?

0:8:26.630 –> 0:8:32.60
Mishuris, Rebecca G.,MD, MPH, MS
Is it going to reduce the amount of time I’m spending in the EHR in the evening when I wanna be home with my family?

0:8:32.630 –> 0:8:39.50
Mishuris, Rebecca G.,MD, MPH, MS
Is it going to actually help me improve the care I’m delivering?

0:8:39.60 –> 0:8:44.210
Mishuris, Rebecca G.,MD, MPH, MS
So I feel better as a clinician that I am providing high quality, safe, equitable care.

0:8:44.560 –> 0:8:47.730
Mishuris, Rebecca G.,MD, MPH, MS
And so those are the kinds of things that we wanna be looking at.

0:8:48.340 –> 0:8:56.50
Mishuris, Rebecca G.,MD, MPH, MS
In addition, you know, as a health system, we’re also concerned about how much these things cost and whether we’re going to be able to afford them.

0:8:56.60 –> 0:9:2.210
Mishuris, Rebecca G.,MD, MPH, MS
And so is there in addition to the clinician well being return, is there a financial return to some of these?

0:9:2.220 –> 0:9:13.120
Mishuris, Rebecca G.,MD, MPH, MS
Are we able to recoup more of the costs of caring for patients through using some of these tools on the patient side, right, there’s a I already in place, right?

0:9:16.950 –> 0:9:17.160
Jordan Cooper
Umm.

0:9:13.130 –> 0:9:22.710
Mishuris, Rebecca G.,MD, MPH, MS
Plenty of people interacted with chat bots, particularly over the last three years, trying to figure out if their symptoms were COVID or not, and whether they needed to go to the hospital or not.

0:9:22.790 –> 0:9:30.920
Mishuris, Rebecca G.,MD, MPH, MS
And so that’s an example of of AI in practice that was improving the interaction that patients were having with our health systems.

0:9:31.110 –> 0:9:38.60
Mishuris, Rebecca G.,MD, MPH, MS
In addition, we use, you know, umm voice recognition to route calls appropriately when patients call in.

0:9:38.530 –> 0:9:41.180
Mishuris, Rebecca G.,MD, MPH, MS
But there’s other things that we can do, you know, patients.

0:9:41.370 –> 0:9:42.900
Mishuris, Rebecca G.,MD, MPH, MS
I’m a primary care doctor, right?

0:9:42.910 –> 0:9:46.140
Mishuris, Rebecca G.,MD, MPH, MS
Patients ask me to fill out forms all the time.

0:9:46.890 –> 0:9:54.580
Mishuris, Rebecca G.,MD, MPH, MS
What if I had a mechanism to automatically fill out that form based off of information that was already in your chart, right?

0:9:54.650 –> 0:9:56.540
Mishuris, Rebecca G.,MD, MPH, MS
You would get your form filled out faster.

0:9:56.550 –> 0:10:3.490
Mishuris, Rebecca G.,MD, MPH, MS
You would get whatever you were trying to get through submitting that form, and I as the provider wouldn’t have had to spend my time filling out the form.

0:10:3.540 –> 0:10:8.430
Mishuris, Rebecca G.,MD, MPH, MS
From I could have spent my time thinking about your really complex medical history or or something else.

0:10:8.910 –> 0:10:9.220
Jordan Cooper
Mm-hmm.

0:10:9.30 –> 0:10:9.480
Mishuris, Rebecca G.,MD, MPH, MS
Umm.

0:10:10.50 –> 0:10:20.560
Mishuris, Rebecca G.,MD, MPH, MS
And so from a patient perspective, right, it’s about how can we improve the efficiency and satisfaction of your interaction with their provider and with the health system in general.

0:10:20.910 –> 0:10:26.300
Mishuris, Rebecca G.,MD, MPH, MS
I think also it has the opportunity to improve the equity of the care that we deliver, right?

0:10:26.310 –> 0:10:54.840
Mishuris, Rebecca G.,MD, MPH, MS
If we can and look at real world outcomes, the kind of tone of medical literature and identify true personalized medicine without having to know your entire genetic code, I can actually start providing higher quality, more equitable care for you, because I will remove the biases, the kind of inherent biases we have in healthcare within an individual within the system to try and address your needs in a more equitable fashion.

0:10:55.380 –> 0:10:55.670
Jordan Cooper
So.

0:10:55.470 –> 0:10:57.860
Mishuris, Rebecca G.,MD, MPH, MS
So there’s real potential on both sides of the coin, I think.

0:10:58.160 –> 0:11:2.710
Jordan Cooper
On that topic, Rebecca, on the topic of equity, I wanna ask you about the quality of the data.

0:11:2.720 –> 0:11:18.590
Jordan Cooper
There’s been some reporting that a lot of LLM S have been trained on Internet data or on data that’s reflective of certain races, predominantly and sexes, white men, which may have a different impact on minority patients.

0:11:18.800 –> 0:11:24.450
Jordan Cooper
What’s the effect of any inherent bias in the data that was used to train LLM.

0:11:24.600 –> 0:11:27.910
Jordan Cooper
When it’s integrated into the healthcare delivery, the delivery of healthcare.

0:11:28.550 –> 0:11:29.210
Mishuris, Rebecca G.,MD, MPH, MS
Yeah.

0:11:29.290 –> 0:11:43.130
Mishuris, Rebecca G.,MD, MPH, MS
So this is something we have to be really careful about and really guard against because you’re right, if we train models on outcomes that were inherently biased, we are going to perpetuate those biased outcomes, those inequitable outcomes.

0:11:43.140 –> 0:11:45.180
Mishuris, Rebecca G.,MD, MPH, MS
And that’s precisely what we don’t wanna do.

0:11:45.330 –> 0:11:47.60
Mishuris, Rebecca G.,MD, MPH, MS
And so we have to do a few things.

0:11:47.70 –> 0:11:52.890
Mishuris, Rebecca G.,MD, MPH, MS
One is to make sure we’re using diverse patient data to train the algorithms in the 1st place, right?

0:11:52.900 –> 0:11:58.880
Mishuris, Rebecca G.,MD, MPH, MS
So we can’t use data from a single health system in a single small town in one part of the country.

0:11:58.890 –> 0:12:5.130
Mishuris, Rebecca G.,MD, MPH, MS
We have to use diverse, really big data systems to to train the algorithms in the 1st place.

0:12:5.360 –> 0:12:11.870
Mishuris, Rebecca G.,MD, MPH, MS
The second is to look at the performance of the algorithms and make sure that the performance is not inequitable, right?

0:12:11.880 –> 0:12:23.310
Mishuris, Rebecca G.,MD, MPH, MS
So if if the algorithm suggests a certain intervention or a certain diagnostic study, is it driving outcomes that are, in the end equitable?

0:12:23.420 –> 0:12:25.870
Mishuris, Rebecca G.,MD, MPH, MS
Or is it perpetuating inequities?

0:12:25.880 –> 0:12:39.780
Mishuris, Rebecca G.,MD, MPH, MS
And if it’s perpetuating inequities, we’ve got a backup and relook at the algorithm and be able to adjust the algorithm to drive equitable outcomes, even if the algorithm wants to drive inequities because of the data that’s been trained on.

0:12:40.500 –> 0:12:47.710
Jordan Cooper
So bias is one of many different risks facing health care delivery systems looking to evaluate LLM.

0:12:47.720 –> 0:12:50.950
Jordan Cooper
S or other sort of machine learning type solutions.

0:12:51.120 –> 0:13:21.960
Jordan Cooper
Other potential threats are HIPAA violations, with an accident that release of personal health information while training the LM, sending out the wrong message without provider review and approval or alert fatigue is a problem where instead of ameliorating the number of clicks and there’s a potential that it may cause more clicks, can you speak about some of these threats and and how CIO or a CMIO listening to this episode may find an ameliorate those threads and his or her own institution?

0:13:22.820 –> 0:13:23.250
Mishuris, Rebecca G.,MD, MPH, MS
Yeah.

0:13:23.260 –> 0:13:24.970
Mishuris, Rebecca G.,MD, MPH, MS
And and it goes beyond that too, right?

0:13:24.980 –> 0:13:41.530
Mishuris, Rebecca G.,MD, MPH, MS
It there are beyond just HIPAA violations, other privacy violations or possible data, you know broad data releases is certainly a fear that we all have just transparency of the algorithm and trust of the algorithm is really important for adoption.

0:13:41.800 –> 0:13:48.370
Mishuris, Rebecca G.,MD, MPH, MS
If I can’t get a provider to trust what the algorithm is telling it, then they’re never going to use it.

0:13:48.380 –> 0:13:52.950
Mishuris, Rebecca G.,MD, MPH, MS
They’ll never pay attention to it, and part of that trust is knowing where it came from.

0:13:52.960 –> 0:14:4.0
Mishuris, Rebecca G.,MD, MPH, MS
Being able to know how the algorithm was built, what data was used to to build it, how it was tested, so having a transparent system and and and the list goes on.

0:14:4.10 –> 0:14:11.880
Mishuris, Rebecca G.,MD, MPH, MS
I think you know, as health systems are responsibility is to raise the concerns and then address them.

0:14:11.890 –> 0:14:46.850
Mishuris, Rebecca G.,MD, MPH, MS
And so if you know to have a framework whereby we evaluate AI and and really any technology, but in today’s world particularly AI against that framework where we’re going to look and say is this an equitable and it is this driving an equitable outcome is the algorithm transparent, are there sufficient data security policies and practices in place where I can be, you know, at least secure that that the data is safe, kind of whatever happens to it and and the list goes on.

0:14:46.860 –> 0:14:54.580
Mishuris, Rebecca G.,MD, MPH, MS
There are a number of professional and and national organizations, and even international organizations that are trying to come up with these frameworks.

0:14:55.0 –> 0:15:0.800
Mishuris, Rebecca G.,MD, MPH, MS
Everything from like the White House to Sam, try to NST.

0:15:0.810 –> 0:15:14.460
Mishuris, Rebecca G.,MD, MPH, MS
Right there, there are lots of organizations that are trying to come up with these frameworks and I think it’s incumbent on us as health systems to be looking at that literature the same way we look at the clinical literature to figure out what treatment is best for hypertension today.

0:15:14.590 –> 0:15:27.780
Mishuris, Rebecca G.,MD, MPH, MS
And we have to be looking at that literature and figuring out what systems and processes do we need to have in place at our own organizations to be secure that we are mitigating and and securing against any of those risks.

0:15:28.270 –> 0:15:36.0
Jordan Cooper
So Rebecca, a lot of our listeners may now understand that there are some sort of abstract framework that we brought a lot of concerns, a lot of things they should look for.

0:15:36.50 –> 0:15:40.540
Jordan Cooper
I wanna dive right into actual use cases, so mass general Brigham.

0:15:40.750 –> 0:15:42.340
Jordan Cooper
There you have different use cases.

0:15:42.350 –> 0:15:44.600
Jordan Cooper
The AI may be ambient listening.

0:15:44.670 –> 0:15:47.100
Jordan Cooper
Maybe portal messages summarizing the chart.

0:15:47.220 –> 0:15:50.260
Jordan Cooper
How do you evaluate what the lowest hanging fruit is?

0:15:50.270 –> 0:15:54.280
Jordan Cooper
Where do we start 1st and how do we pick a vendor or solution?

0:15:54.290 –> 0:15:55.40
Jordan Cooper
Or do we homegrown?

0:15:55.50 –> 0:15:59.630
Jordan Cooper
My solution is that we minimize these threats and maximize these opportunity gains.

0:16:0.800 –> 0:16:3.990
Mishuris, Rebecca G.,MD, MPH, MS
So it’s a great question and your comment earlier is absolutely right, right.

0:16:4.0 –> 0:16:12.800
Mishuris, Rebecca G.,MD, MPH, MS
I can’t change what I can’t measure and so and it also is going to be dependent upon your own organizations, priorities and strategy, right?

0:16:12.810 –> 0:16:29.950
Mishuris, Rebecca G.,MD, MPH, MS
So if your organization has a strategic goal of improving clinician well being, you may look at a different piece of technology than if your organization has a goal of improving patient experience or revenue generation and the list goes on.

0:16:29.960 –> 0:16:37.710
Mishuris, Rebecca G.,MD, MPH, MS
And so you really have to take it from your organization’s perspective about what’s really driving your strategy at your organization.

0:16:37.940 –> 0:16:43.310
Mishuris, Rebecca G.,MD, MPH, MS
That being said, generally the IHI quintuple framework is a good thing to start with.

0:16:43.320 –> 0:16:59.930
Mishuris, Rebecca G.,MD, MPH, MS
If if your organization doesn’t have its own strategy, I would start there and then it part of it is also being driven by what technology is available, right we this technology is new enough that there are a few spaces where vendors are focused.

0:17:0.260 –> 0:17:16.220
Mishuris, Rebecca G.,MD, MPH, MS
So things like robotic process automation, ambient listening that you mentioned drafting and denovo messages either to patients or summarizing a chart, there are lot you know kind of spaces within the clinical realm that that companies are starting in.

0:17:16.230 –> 0:17:31.150
Mishuris, Rebecca G.,MD, MPH, MS
And so it’s a it’s a triangulation really, of what’s your organizations priorities, what technology is out there and and what can you support as an organization moving forward as you move forward with these things, you then have to evaluate them, right?

0:17:31.160 –> 0:17:32.450
Mishuris, Rebecca G.,MD, MPH, MS
What impact are they having?

0:17:32.460 –> 0:17:37.10
Mishuris, Rebecca G.,MD, MPH, MS
Are they having the impact that you need them to have as an organization?

0:17:37.20 –> 0:17:40.620
Mishuris, Rebecca G.,MD, MPH, MS
And so, you know, paying attention to things like, is it safe?

0:17:40.630 –> 0:17:41.560
Mishuris, Rebecca G.,MD, MPH, MS
Is it transparent?

0:17:41.570 –> 0:17:42.470
Mishuris, Rebecca G.,MD, MPH, MS
Is the data secure?

0:17:42.480 –> 0:17:43.960
Mishuris, Rebecca G.,MD, MPH, MS
Those that’s table stakes.

0:17:44.70 –> 0:18:10.700
Mishuris, Rebecca G.,MD, MPH, MS
Then we have to figure out, is this actually improving clinician well being right our our my providers less burned out are they spending less time in the EHR after hours are returning messages around faster to our patients are the patients more satisfied with their experience with our health system and those are the kinds of questions you have to start asking as you actually implement these technologies to figure out if they are having the impact that that you intend them to have.

0:18:11.770 –> 0:18:15.740
Jordan Cooper
So you actually mentioned a lot of interesting KPI just now.

0:18:16.210 –> 0:18:26.0
Jordan Cooper
You you said specifically spending less time in the EHR after hours, our patients satisfied that’s a little bit more difficult, but you can have a fetish measure for that.

0:18:26.90 –> 0:18:27.920
Jordan Cooper
Are they getting faster responses?

0:18:27.930 –> 0:18:29.380
Jordan Cooper
Well, that’s very discreet.

0:18:29.390 –> 0:18:32.990
Jordan Cooper
What time did we receive the message and what time did they receive a response?

0:18:33.770 –> 0:18:37.240
Jordan Cooper
I guess I I think keep what?

0:18:37.250 –> 0:18:38.840
Jordan Cooper
Are there any other contributing?

0:18:39.10 –> 0:18:50.710
Jordan Cooper
Any other KPI is for clinician productivity, patient outcomes maybe you know population health management or is it all on an individual basis?

0:18:50.880 –> 0:18:51.290
Jordan Cooper
What?

0:18:51.300 –> 0:18:56.620
Jordan Cooper
What would you consider would be the most important KPI you can as we’re evaluating AI solutions?

0:18:57.290 –> 0:18:59.720
Mishuris, Rebecca G.,MD, MPH, MS
So so again, there’s table stakes.

0:18:59.810 –> 0:19:0.480
Mishuris, Rebecca G.,MD, MPH, MS
Is it saved?

0:19:0.490 –> 0:19:1.420
Mishuris, Rebecca G.,MD, MPH, MS
Is it secure?

0:19:1.430 –> 0:19:2.300
Mishuris, Rebecca G.,MD, MPH, MS
Is it equitable?

0:19:2.310 –> 0:19:3.120
Mishuris, Rebecca G.,MD, MPH, MS
Etcetera.

0:19:3.220 –> 0:19:11.170
Mishuris, Rebecca G.,MD, MPH, MS
And then we’re talking about impact and the impact that you’re going to be looking for is going to differ depending on what technology you’re implementing, right?

0:19:11.210 –> 0:19:17.500
Mishuris, Rebecca G.,MD, MPH, MS
It’s gonna be different if I’m implementing ambient listening versus a call routing voice recognition system.

0:19:18.90 –> 0:19:21.940
Mishuris, Rebecca G.,MD, MPH, MS
And so you really have to then pay attention to what’s the intent of the technology.

0:19:22.90 –> 0:19:28.400
Mishuris, Rebecca G.,MD, MPH, MS
If we’re talking about clinician well being, so we’re talking about something like ambient listening to support, note documentation.

0:19:28.870 –> 0:19:34.300
Mishuris, Rebecca G.,MD, MPH, MS
I’m interested in, you know what I’ve talked about the how much time are you spending documenting your notes?

0:19:34.310 –> 0:19:37.110
Mishuris, Rebecca G.,MD, MPH, MS
How much time are you spending in the EHR after hours?

0:19:39.60 –> 0:19:41.950
Mishuris, Rebecca G.,MD, MPH, MS
Can the are the notes understandable to our patients, right.

0:19:41.960 –> 0:19:43.670
Mishuris, Rebecca G.,MD, MPH, MS
Our patients are all reading their notes now.

0:19:43.680 –> 0:19:45.670
Mishuris, Rebecca G.,MD, MPH, MS
Can our patients understand the notes?

0:19:45.820 –> 0:19:46.930
Mishuris, Rebecca G.,MD, MPH, MS
What’s the workflow?

0:19:46.940 –> 0:19:53.680
Mishuris, Rebecca G.,MD, MPH, MS
Are you actually satisfied with the workflow of getting of going from the visit to the note and then things like throughput?

0:19:53.690 –> 0:19:58.430
Mishuris, Rebecca G.,MD, MPH, MS
And that’s where we’re gonna have potential impact to the bottom line, right.

0:19:58.440 –> 0:20:8.980
Mishuris, Rebecca G.,MD, MPH, MS
The revenue bottom line of the health system, so in urgent care, are you seeing more patients because you don’t have to spend as long writing your notes and so you’re able to see more patients in urgent care.

0:20:9.40 –> 0:20:11.910
Mishuris, Rebecca G.,MD, MPH, MS
And so that’s an impact to our financial bottom line.

0:20:11.920 –> 0:20:19.250
Mishuris, Rebecca G.,MD, MPH, MS
It’s also an impact to our patients, right, because now they can actually get the care that they came to see us for and they’re not waiting 6 hours to see us.

0:20:20.200 –> 0:20:30.150
Mishuris, Rebecca G.,MD, MPH, MS
Are we documenting better so that we within value based contracts are better representing the risk that we take on with the patient’s we’re seeing?

0:20:30.160 –> 0:20:36.790
Mishuris, Rebecca G.,MD, MPH, MS
So we’re better representing how sick our patients are through our improved documentation with ambient listening.

0:20:36.800 –> 0:20:39.990
Mishuris, Rebecca G.,MD, MPH, MS
And so those are some of the other things that we’re paying attention to as well.

0:20:40.440 –> 0:20:45.840
Mishuris, Rebecca G.,MD, MPH, MS
Again, we are very early stage in all of this and and piloting it and trying to figure out what the impacts are.

0:20:46.520 –> 0:20:49.220
Jordan Cooper
So we are approaching the end of this podcast episode.

0:20:49.240 –> 0:20:50.70
Jordan Cooper
I like that pose.

0:20:50.80 –> 0:20:55.590
Jordan Cooper
A final question to you, so many listeners may be thinking about timing.

0:20:55.640 –> 0:20:58.620
Jordan Cooper
You said we’re on the very, you know, we’re in early days here.

0:20:59.760 –> 0:21:6.270
Jordan Cooper
If I’m AS CMIO listening to this episode and maybe asking myself, alright, do I wanna go ahead now and be on the bleeding edge?

0:21:6.280 –> 0:21:14.260
Jordan Cooper
Or maybe I should wait a year or two so other organizations like Mass General Brigham can do the experiments, preview of the greatest value is, and I can just waltz in.

0:21:14.270 –> 0:21:21.830
Jordan Cooper
Once a bugs are fixed and just go in there later and not deal with all the Kingston and hassles right now, how would you answer a question like that?

0:21:22.980 –> 0:21:24.950
Mishuris, Rebecca G.,MD, MPH, MS
Umm do you wanna be an innovator?

0:21:24.960 –> 0:21:32.10
Mishuris, Rebecca G.,MD, MPH, MS
An early adopter or a late adopter right in the in the grand scheme of the technology wave, what’s your appetite for risk?

0:21:32.80 –> 0:21:34.120
Mishuris, Rebecca G.,MD, MPH, MS
What’s your resources?

0:21:34.130 –> 0:21:38.490
Mishuris, Rebecca G.,MD, MPH, MS
What are the resources you can bring to bear in a kind of innovative space?

0:21:39.0 –> 0:21:39.420
Mishuris, Rebecca G.,MD, MPH, MS
Umm.

0:21:39.840 –> 0:21:49.510
Mishuris, Rebecca G.,MD, MPH, MS
Or are you in a system that doesn’t have that kind of those kinds of resources, and you do need to wait for something that’s a little bit more proven because you can’t afford to get it wrong, right?

0:21:49.520 –> 0:21:59.950
Mishuris, Rebecca G.,MD, MPH, MS
I I have the privilege right now of being in an organization that has resources to bring to bear on innovation and and does that, you know, in the clinical space and also in the operational space.

0:21:59.960 –> 0:22:1.730
Mishuris, Rebecca G.,MD, MPH, MS
I come from a system, you know.

0:22:1.740 –> 0:22:7.590
Mishuris, Rebecca G.,MD, MPH, MS
I came from a safety net system where we didn’t have those kinds of resources and couldn’t necessarily be on the bleeding edge of technology.

0:22:7.960 –> 0:22:9.960
Mishuris, Rebecca G.,MD, MPH, MS
And so it’s not wrong.

0:22:9.970 –> 0:22:14.840
Mishuris, Rebecca G.,MD, MPH, MS
Either way you go about it, but it it really has to do with your appetite and your resources.

0:22:15.850 –> 0:22:20.480
Mishuris, Rebecca G.,MD, MPH, MS
We will all get there eventually, but it’s a matter of of when, not if, I think.

0:22:21.150 –> 0:22:25.220
Jordan Cooper
Well, thank you very much for our listeners who joined in the middle.

0:22:25.230 –> 0:22:32.110
Jordan Cooper
This has been doctor Rebecca Mishuris, the chief Medical Information Officer and VP of Mass General Brigham Rebecca.

0:22:32.120 –> 0:22:33.700
Jordan Cooper
I’d like to thank you very much for joining us today.