S4E16: Data Governance & an Automated KPI Dashboard (ft. Anup Palvia & Bridget McCormick, Cooper University Healthcare)

Anup Palvia   0:58
Yeah, absolutely. Formally speaking, on a formal data governance program aspect, looking at it in that way, I would say Cooper is still very early in his data governance journey. I think what has been very interesting in my journey working with Bridget

Jordan Cooper   1:12
Mhm.

Anup Palvia   1:19
is we developed an application and an organizational dashboard that essentially allowed us to set up a framework for data governance without officially launching data governance. And that's what we thought would be really cool and special to talk about today.

Jordan Cooper   1:37
Interesting. Okay, Bridget, would you tell me more about that framework and how it occurred to you to set it up, especially since, as you're saying, it is unique, you know, how did you come to this idea when your peers across the country may not have done so yet?

Bridget McCormick   1:52
Right. So like many organizations, we set goals every year based on what we're trying to achieve, and they're aligned to our overall strategic plan and objectives. And in order to get the results that we're looking for, we have a goal cascade process where Every leader receives goals that ultimately align to our organizational dashboard, and they cascade them down to the team member level. And this is a process that's put into place to make sure we're all rowing in the same direction, and we're ultimately impacting the goals of the organization.

Jordan Cooper   2:21
Yeah.

Bridget McCormick   2:31
So prior to connecting with Anup and his team, a lot of that tracking of goals and progress and metrics was very manual. You know, initially we started on Excel spreadsheets and then we even had a master Excel spreadsheet on a SharePoint site. for the organizational dashboard that multiple data stewards had to go in and update, and there was differences in when that got updated. And our leaders really had to go on a hunting expedition for the metrics for their goals that they were accountable to. And so a lot of time was spent and a lot of manual work in that process. So part of our goals process is we have operational leaders that set those organizational goals that we call pillar champions. And they're set around 4 pillars, volume, high reliability,

Jordan Cooper   3:14
Mm-hmm. Mhm. Mm.

Bridget McCormick   3:29
Cooper experience, education and research, and growth and financial stewardship. So we had somewhat of a governance structure set in place with our pillar champions that set the goals for the organization. The piece that was missing is how do we have one central place managing all those metrics and one central place for people to go and have transparency on how they're doing based on those metrics. Tracks.

Jordan Cooper   3:54
Got it. And would you say that you're the business owner and then you went to Anup and asked his team to build out this dashboard and framework for you?

Bridget McCormick   4:04
Yes, so I'm one of the pillar champions, but I also oversee our goals and objectives for our leaders here at Cooper.

Jordan Cooper   4:06
Yeah. Mhm. So, okay, so obviously every provider organization in America has a bunch of dashboards and goals that everybody needs to hit. And many of them use Power BI or different data visualization tools and dashboard. There are an abundance of dashboards across the country for many different vendors.

Bridget McCormick   4:33
Uh-huh.

Jordan Cooper   4:34
So you built your own framework, you developed your own application dashboard. Let's just dive in. What are some of the tactics, maybe Anup, that you can talk about that you've used to standardize your data quality, right? So Bridget comes to you and says, we need to hit these metrics on these dashboards. And you say, I'd love to, but there are different definitions. Maybe the data sources aren't reconciled. Maybe there's duplicative data. How did you go about standardizing your data quality?

Anup Palvia   5:06
Yeah, no, that's a great question. And that's really the secret sauce of this effort that we undertook. As Bridget mentioned, on the face of this, this was an operational problem. There was a lot of hours wasted, countless labor hours wasted, where people were trying to hunt and gather.

Jordan Cooper   5:18
Mhm.

Anup Palvia   5:25
For this information, by contacting different subject matter experts around the organization, the term "data steward" didn't exist at Cooper when we first embarked on this, but we realized that these subject matter experts were essentially acting as gatekeepers for this information, because they had that subject matter expertise.

Jordan Cooper   5:30
Mhm. Mhm.

Anup Palvia   5:45
And they were also manually pulling a lot of this. So they were wasting time and the leaders were wasting time trying to find it.

Jordan Cooper   5:51
Mm.

Anup Palvia   5:51
So... We set out first, the first major challenge was, well, what are all the metrics that really matter? And all this information was collected in a smorgasbord of different spreadsheets for different divisions, different parts of the organization. We decided to focus on 2 areas. First, the physician leadership, sort of like a lot of times when you prioritize, there's the high impact. For that, for us, that was physician leadership. And then there was a low-hanging fruit. And that was a lot of the other leaders who had a lot of commonality amongst the goals that they typically picked. I think for physician leaders, the work of standardization was very heavy. We had to go through metric by metric where there were similar sounding metrics that we had to demonstrate and prove that they were really the same thing.

Jordan Cooper   6:45
Let me jump in for a second, just ask a clarifying question our listeners may be wondering right now. So Anup, when you're focusing on these different metrics and trying to find out which metrics matter, and Bridget, feel free to jump in here, are you focusing on CMS defined measures, National Quality Forum measures, measures that you are paid for or dinged on?

Anup Palvia   6:51
Sure.

Jordan Cooper   7:06
or these internal process measures that are accountable to the C-suite, but really just live within Cooper University.

Anup Palvia   7:15
It is both. It's both. I think we have, as Bridget mentioned, there are four pillars. There's quality, finance, Cooper experience, and education and research. Amongst the quality measures, many, many of them are CMS driven. So there's these standardized definitions that exist outside. Even amongst those, you have to pick and not get confused.

Bridget McCormick   7:17
Mhm.

Jordan Cooper   7:17
Yeah. Mm-hmm. Mhm. Mhm. Right.

Anup Palvia   7:36
But yeah, there's a mix. So a lot of them are internal measures.

Jordan Cooper   7:41
Got it. Okay, so sorry to interrupt. So there's two areas here, physician leadership, high impact, and then leaders with a lot of commonalities where there's low hanging fruit.

Anup Palvia   7:51
Yes, yes, absolutely.

Jordan Cooper   7:53
All right, and Bridget, you were participating in the prioritization of which measures you were focusing on and how to clean the data?

Bridget McCormick   8:02
Yes, and working with our pillar champions and other subject matter experts that were responsible for this data, and our senior team who helps to approve what that final organizational dashboard looks like every year.

Jordan Cooper   8:08
Mm-hmm. So how did you, I'd love to hear how, when you have just hundreds of different measures that are applicable at Cooper Health, how did you prioritize which measures? I'd love that, I think our listeners would love to hear what tough choices you had to make and how you made those choices.

Anup Palvia   8:30
Absolutely, absolutely. Then there were quite a bit of tough choices.

Bridget McCormick   8:30
Uh-huh.

Anup Palvia   8:37
I think in the first iteration of this, we had about 300 metrics that came down to like 80, I think it was 87. And the way that we went about that was they sound similar, they're in the same division, talk to each person. We had to introduce a concept called scope type because...

Jordan Cooper   8:55
Mhm.

Anup Palvia   8:58
For a leader in the organization, there wasn't at the time, for those on the technical side, the concept of a dimension and a measure are part and parcel of the work that we do. But on the business side, that wasn't really how people were thinking. So they would have a different metric for a different location or a different specialty or a different area or a different. So we had to introduce this concept of scope type. and scope area and really train hard and educate the organization on that concept to enable us to decouple those two things, those two concepts, so that you could have a lot more.

Jordan Cooper   9:23
Mm-hmm. Mm-hmm.

Anup Palvia   9:29
Once you do that, a lot of these metrics were similar. And then at times it was just labeling and differences of that nature as well. So that's how we were able to bring it down to that level. I think one of the toughest decisions we had to make is we initially wanted to be. a little bit more aggressive in our data governance journey. We wanted to standardize. We wanted to use this tool not only to... reduce operational time spent on this, but also like when you have more standardized metrics, that's how you unlock the value of data itself, right? If people are using the same metrics, that's how you find the division that is excelling in something and can maybe help others out, et cetera. So we wanted to achieve that, but I think it was not possible to... To. make all those trade-offs or sell that, I think, all the way up front. So we ended up coming up with the concept of ungoverned goals and governed goals. And we built our application to allow for those ungoverned goals, but kind of there's a series of nudges where we try to...

Jordan Cooper   10:25
Mhm. Yeah.

Anup Palvia   10:37
incentivize selecting a standardized governed goal so that those are, you know, going to help us with the with efficient governance and automation for data delivery as well.

Jordan Cooper   10:49
So, Bridget, what were some of the specifications you asked of Anup and his team when you said, I want you to develop this application dashboard? What, from a business perspective, were you looking for, and how did you describe the problems that you were hoping to solve with this solution?

Bridget McCormick   11:05
Yeah, so first kind of shared a little bit about what our process looked like, current state. You know, we had all these different Excel spreadsheets. People were entering information differently, so there wasn't standardization. So first we wanted to not only come up with standard metrics, but then have a tool for our leaders to go in

Jordan Cooper   11:11
Mhm.

Bridget McCormick   11:25
and select the goals that they were going to be responsible for for that calendar year.

Jordan Cooper   11:31
Mhm.

Bridget McCormick   11:32
And that one central place would be where everyone goes to enter their goals. Their leader would have transparency to see what they've entered and could also see how they're doing based off of those goals. And then the big piece would be, as a leader, at the end of the day, I want to be able to go into this dashboard, filter down to my name,

Jordan Cooper   11:40
Mmh.

Bridget McCormick   11:52
see the goals that have carried and what's the current status and my progress on the performance of those goals.

Jordan Cooper   12:01
I want to kind of throw a curve ball and see if either one of you are interested. Can you take an example of 1 metric that applies to you that you select in your own dashboard and tell me, just walk me through how that measure is, I mean, affecting morale, your team. what data challenges there are to get that metric that you are measured on by whoever you report to. Either one, feel free.

Bridget McCormick   12:28
Yeah, so I can go. One of the goals that a lot of our leaders carry that's related to experience is an internal customer service goal. And so this is actually a tool that we've created for those departments that their customer is another area or department within the system.

Jordan Cooper   12:29
Uh-huh. Mhm. Mhm.

Bridget McCormick   12:49
Many of our departments carry patient experience goals, which is very easy to measure with our partner who does the surveys. But how do we have accountability for those departments that support those individuals that are delivering patient care? And so we actually developed a tool with the same team.

Jordan Cooper   13:03
Mhm.

Bridget McCormick   13:07
to have a standard survey that allows us to quarterly send out to leaders. And so if I work within the human resources team, how are we doing in supporting you and your team around accuracy, communication, ownership, and responsiveness? And then each leader, I get a goal based on how we did on that survey. And have a set. improvement that needs to be done by the end of the year based off of that. So because that survey is tracked now by Enterprise Analytics, every quarter I can go in, see the results of that, see how I'm comparing to my goal that was set based off of the beginning of the year, and I'm reviewing that information with my team.

Jordan Cooper   13:50
Mhm.

Bridget McCormick   13:53
What areas have we improved? What areas do we still need improvement on? Are there certain departments we need to round on and follow up because we've dipped in accuracy, for example?

Jordan Cooper   14:06
So. I love that. I think that provides more transparency and accountability and provides more visibility for leadership into what is actually happening in the organization. Anup, when you get a request from a leader like Bridget to say, hey, you know, I want this, you know, standardized metrics. I want to have select goals that I'm responsible for this year.

Bridget McCormick   14:08
Got. Right.

Jordan Cooper   14:30
and you have new data sources like surveys that are being standardized and sent out internally. What technical challenges do you have incorporating these new data sources into the application dashboard and how have you been overcoming those technical challenges?

Anup Palvia   14:48
I'm glad you asked that, Jordan. That's one of the decisions we had to make at the inception of this project was, was the scope of this project going to be... the metrics that are available automatically, which at the time was less than half? Or are we going to let the scope be all the metrics that these leaders need so that to the end user, this is a fully functioning product? And that the messy stuff, we can keep some of it behind the scenes. So we ended up with the latter approach.

Jordan Cooper   15:03
Mm. Mhm.

Anup Palvia   15:21
where we have the half of metrics, say about half the metrics that aren't fully automated. We had them coming through scaled manual process where the data stewards will enter them into a central repository, calculating them however they calculate them. They only have to calculate them once for a certain scope. that way. And it's still more efficient for those subject matter experts who had always been providing this on demand. But then for the end users, it feels like a fully automated experience. So that way, we were able to launch everything. And then as metrics became automated over time, Internal customer survey is a good example. That was automated pretty early on in the process. So we're able to just plug that in quickly. And then over time, as new metrics come up, data stores may get new work, but then we'll automate a certain number of metrics every year as well.

Jordan Cooper   16:06
Mm. So. I just want to hone in, Anup, and follow up on something you just said. I think you said data stewards send into a data into a central data repository calculating once. So my question is, is there some kind of portal that you built into your enterprise data warehouse that allows different metric owners to submit their own Like metrics or how are you ingesting that new data into your dashboard? There's a new numerator denominator data source.

Anup Palvia   16:41
What? We let them enter them into flat files and whatever data is convenient for them. So essentially flat files and their own, well, we have our web of scripts that will pull it from the right place that we've organized. And then over time, it gets pulled, put into proper structured data.

Jordan Cooper   16:56
Mhm. Got it. Let's shift back, Bridget, to some of the internal politics and cultural navigations that was required to have this project succeed. I'd love to hear you elaborate and you don't have to do everything, pick your poison here. How have you, how did you...

Bridget McCormick   17:10
Huh. Mhm.

Jordan Cooper   17:21
forge compromises across different service lines and work with leadership to align definitions of those ECQMs or recruit VP and other VP and C-level sponsorship of your work so that this is something that, okay, you know, it's not just Bridget's, you know, priority, but this is the organization's priority.

Bridget McCormick   17:40
Right.

Jordan Cooper   17:41
How did you kind of navigate all those different obstacles?

Bridget McCormick   17:44
Yeah, so I think the important thing is having strong sponsorship from our executive leadership team and also to our VPs and above. We also had them be kind of our first customers in using the tool, which I think really helped kind of almost create our own champions of the benefits.

Jordan Cooper   17:50
Mhm. Mhm. Mhm.

Bridget McCormick   18:04
that people will see in having all this being tracked in one central place. I think the other thing too that we did really well with Anup and his team was not progress over perfection. Like Anup said, we weren't going to wait till it was perfect before we started this work. And we brought people along.

Jordan Cooper   18:10
Mm-hmm. Mhm.

Bridget McCormick   18:26
and kept them updated. So for example, when we worked specifically with the physician leaders, we worked closely with our chief physician executive, our chief medical officer, and our VP of Medical Affairs who really oversee that process and stayed connected with them through and through and made sure we got their feedback.

Jordan Cooper   18:41
Mhm.

Bridget McCormick   18:45
Will this work for you and your leaders? And then based on feedback, we would adjust. And then we would even have meetings with the specific physician leaders and getting their feedback. So it was an iterative process that we continued to refine. One thing that Anup and I did really well too is we stayed connected, had regular check-ins.

Jordan Cooper   18:55
Mhm. Mhm.

Bridget McCormick   19:05
And if we didn't have a full agenda, then we only spent a few minutes connecting and checking in on our progress. But I think that really helped keep things moving in the right direction.

Jordan Cooper   19:18
Um...

Anup Palvia   19:19
Yeah.

Jordan Cooper   19:20
Yeah, Anup, did you want to comment on that?

Anup Palvia   19:21
Yeah, and I just wanted to add, I agree that those check-ins helped us course correct a lot. And I think the other thing that really helped with adoption of this was, and like Bridget mentioned, never let perfect be the enemy of good enough. That was there in the framework, as we mentioned earlier, that was also there in the deployment. Adoption of the tools sometimes wasn't as fast as we wanted as people that weren't used to using a friend versus a spreadsheet. It took a little time to get them in. Sometimes we would do it for them. Like if things, we were going to ensure tool adoption one way or the other. And we only had.

Jordan Cooper   19:54
Yeah.

Bridget McCormick   20:00
The.

Jordan Cooper   20:08
Cooper.

Bridget McCormick   20:19
Right.

Anup Palvia   20:20
As well.

Jordan Cooper   20:20
So we're approaching the end of the podcast episode, so I'd like to just ask a few more final questions. I think our listeners are interested. This sounds great. You build something great that works at Cooper Health. How successful was it? Is this something that I should be replicating at my organization? You know, how are you have metrics? Have you been able to tie your work to revenues at the organization? Have you seen that you've improved your 30 day readmission score that you're being paid on? Have you been able to improve staff retention? You know, do you actually have demonstrable KPIs of your ROI?

Anup Palvia   21:00
Yeah, I anticipated that question. And ROI is always tough for data analytics initiatives. I think the cost savings aspect are the most straightforward. We did labor hours calculations, thousands of hours you can associate with, you know, easily half a million a year.

Jordan Cooper   21:07
Mhm.

Anup Palvia   21:20
or more, but, and that's just without a full rollout. But I think with data governance. And this is why data governance is notoriously hard to sell. It's very hard to pinpoint the true ultimate value, which is decision quality and faster process improvement due to clearer comparison of metrics against clearer benchmarks because everyone's talking the same language.

Jordan Cooper   21:32
Mhm.

Anup Palvia   21:50
So I think. The leadership understands that. They feel that convenience, but it's not something that I'm able to pinpoint.

Jordan Cooper   21:52
Were you able? Were you able to retire any other data visualization or dashboard tools that the organization previously subscribed to?

Anup Palvia   22:06
Yes, yeah, yeah, the manual organizational dashboard spreadsheet. Um, Bridget was very happy to rip that one, I will say. So, there there are there are a few that were of manual efforts that we've been able to retire, yeah.

Bridget McCormick   22:17
Yeah. And I would say indirect return on the investment would be, you know, the transparency of just having those metrics in front of our leaders and they see the progress they're making really has, I think, helped us improve our overall performance on our organizational dashboard.

Jordan Cooper   22:30
Mhm. Mm-hmm.

Bridget McCormick   22:37
Before, it wasn't as easily accessible. It wasn't as transparent. And then I think that's even helped people also think about what are the goals I should be carrying that do align with what we're trying to achieve as an organization. And I would say that's kind of an indirect impact. That's maybe not as easily measured specifically, But just looking at our own dashboard and the goals we've been able to achieve, it would be hard pressed to say that the work that we've done around this governance, transparency, the data, doesn't have an impact on that.

Jordan Cooper   23:03
Mhm. Final question to each of you, and I'll allow each of you to close out with a statement here. Bridget, some people are listening and they're still, you know, their workflows are dependent on spreadsheets today. What advice do you have to them or to yourself a few years ago prior to creating your first spreadsheets? And your organization isn't doing well on metrics, you're being dinged by the payers, you're having a lot of pressure from the C-suite to do better, and there's just a lot of data confusion. And you're in spread, what advice do you have for them?

Bridget McCormick   23:51
I would say to think about, you know, rather than focusing on perfection or boiling the ocean, where can you start first to start to see the impact and where's your biggest opportunity? And start small, but then eventually you'll start to get the momentum and progress you need.

Jordan Cooper   23:56
Mmh. Yeah.

Bridget McCormick   24:10
more people on board. And, you know, don't get frustrated when people still want to go back to the manual process. Meet them where they are, like Anup said. Sometimes it takes a little more hand-holding, but eventually you get there.

Jordan Cooper   24:27
So, Anup, I want to ask you a different variation of that same question. I'd like to ask you to offer advice both to technical owners and business owners of related workflows. If there is someone listening who is also the VP of Business Analytics, or a business owner, for example, how would you tell them? Did you engage with the business owner early enough? Did you need to engage, get executive sponsorship before you began building this? Did you need to do something with budget in order to allow your team to have freed up time? Did you do staff augmentation? Like just What are some practical words of wisdom you can offer, one, to someone like yourself, who's a technical owner who has to actually deliver the product, and two, to business owners, maybe engage me sooner or something like that, who have this problem who you need to serve?

Anup Palvia   25:25
Yeah, that's a good question. I would say start small and start high impact. I think when we start, because we started with the physician leaders and a small group of people who had a very high shortage of time and a very high desire to save time, it was the right customer. So start with the right customer. And. Give them something tangible in a with a prototype as early as possible, because... Data governance is very ambiguous. If you talk about data governance, it doesn't resonate. If you talk about standardization of metrics, it won't resonate. Saving you time, so you don't have to hunt and gather for metrics, that'll resonate. Start with the right customer, give them something tangible as soon as you can. I think that made the big difference for us.

Jordan Cooper   26:15
All right. Well, I appreciate each of you. Thank you both for joining Healthy Data Podcast today.

Anup Palvia   26:22
Our pleasure. Thanks for having us.

Jordan Cooper   26:24
And for our listeners, this has been Anup Palvia, the Director of Business Insights and Analytics at Cooper University Healthcare, and Bridget McCormick, the VP of Organizational Effectiveness and Learning at Cooper University Healthcare.

Anup Palvia   26:38
All right. Thank you.