Bringing Clinical Data to Life with Simulation

III. Why Should an Electrophysiology Lab Manager Care?

1. Hospital administrators’ input

There is a lot of data here, but we have not yet really explored what this means to the hospital administrator. When I say ‘hospital administrator’, I mean the person in the hospital who is most closely managing the EP lab resource. [07.30] It was my responsibility to explore what this meant to the hospital administrator and how to show them the value. I talked to many hospital administrators around the United States. We had advisory boards where we brought dozens of them in and I went out and travelled, and talked to some of them at their hospitals.

2. Defining metrics

I learned that the evidence we had provided so far was not very meaningful to them. We had done things like trying to quantify the economic [08.00] value of this in cost per minute. Cost per minute is interesting and it tells you how you are spending your money, but they told me, ‘Reece, if this procedure ends sooner, I am still paying for that room and it is just sitting there empty. If the staff I have hired to perform these procedures are done sooner, I cannot send them home early and save money.’ The cost per minute was not a very meaningful metric.

As I already discussed, focusing on procedure time was not as meaningful for them. [08.30] Showing these averages and helping them to understand the averages without the variability was not very impactful. Then it was assuming only this procedure time data mattered and there was not anything else that mattered. I learned that the metrics that were important to hospital administrators in this case were things like avoiding overtime. If they had days where the procedures went so long that people were there past the end of their shift, that was [09.00] undesirable.

What was also desirable to them was, if cases ended soon enough that the room could be used for other cases, that was useful in terms of treating more patients and potentially having profitable procedures. The lab occupancy time meant more to them. Understanding the detailed distribution and the true variability of these procedures, and accounting for that, was important, as well as accounting for the real-world complexity, which I will talk about in a moment. [09.30]

3. Case study

Diving a little deeper into how we decided to use the information we gathered from hospital administrators, there was a connection between what they were asking for, the richness of data that we had and the modelling approach to transform that data and the way we were showing it into a new thing that mattered to them. We started by going back to the raw data that we got from the clinical study. [10.00] This is a histogram of procedure times for the cryoballoon technology, for AF technology and the two curves superimposed on each other.

We learned that not only did we have a 36-minute reduction in the average procedure time; it also appeared that the longest procedures were less with the new technology as well. [10.30] In fact, if you look at the 90th percentiles, there is about an hour difference. The worst cases would be about an hour faster with the new technology. That was impactful.

4. Real-world complexity

We tried to account for real-world complexity from the administrators. I talked to them over and over again and kept expressing what I thought their problems were. Then they would say, ‘Reece, there is one more you do not know. This is the new one.’ This is a good sampling of what those things are. [11.00] They would say, ‘Yes, your technology made me 30 minutes faster but my doctor will come 30 minutes late to his case and I have lost that.’

Sometimes their electrophysiologist would not come to the case on time, for good reason, but still late. Sometimes the patient’s lab results would be delayed and they would not be able to start the case until they were finished. Sometimes the room turnover would be complicated and that would cause problems. Sometimes the patient would simply not be able to get to the room because nurses were not available to push them there. [11.30] We took all of those different things and put them into a model so that I could tell the hospital administrators who asked me, ‘Reece, did you account for room turnovers’, ‘Yes, we have got them in here. This is how we put it in and this is how it behaves in the model.’ Eventually it got to the point where they said, ‘Okay, now I think you understand me and I am willing to look at the results of your model.’ [12.00]

5. Case scheduling and the model

This is a block diagram of how the model behaves. It is probably not the most complex model you will ever see, but at least it accounts for everything that was of interest to our administrators. What happens in a hospital is that, since these procedures take a couple hours, they will attempt to do two cases per day. They do them according to a block schedule, so they will start the first case at a fixed time in the morning and, even if they finish that case early, [12.30] they will not start the second case until its block schedule defined time. If that first case goes on longer, it will push the second case out.

In the model, the patient first arrives at the hospital according to the block schedule. The patient may get there on time or may not. They may go through certain delays before they are available to be in the operating room. [13.00] As a separate part of the process, the EP comes in at the beginning of his shift. He will start doing things that he thinks are useful. He does just not run in, stand next to the operating table and wait for the patient to come. He is busying himself with what he thinks are useful tasks, so I call these ‘task delays’ and they may go anywhere from five to 60 minutes.

Once the time has come for the procedure to happen, the EP will check [13.30] to see if the patient is there. If the patient is not there, he will go back and do some other things. He will not sit there and wait. Even if the patient were to come one minute after the EP checked, he will still finish whatever task he went off to do, and that is the source of why the EPs sometimes come late to procedures. Once all this has sorted itself out and they are both there together, they will go into the procedure room.

We will pull a procedure time from these [14.00] distributions, for which SIMUL8 has some really nice capabilities. These particular curves are best fit as gamma curves, so we put one into SIMUL8 and it pulls random procedure times off those curves. Then the patient exits and the doctor goes back to his loop and waits for that second case of the day to occur.

IV. Operation of the Model

1. The simulation in action

Hopefully that is a reasonably clear understanding of how the model is supposed [14.30] to operate. I have actually captured a brief video of the model in operation. Now the model is going to start working. I have the time turned down really low so you can see [15.00] the workings of the model. This is the RF line. The patient got to the RF case first and the cryoballoon case started afterwards. Now it has completed that case and that doctor is off to his tasks. The cryoballoon case ended.

Now we go on to the second case of the day, which started with RF as well. [15.30] What we find here is that, when that day goes onto completion, the second RF case took longer and they ended up in overtime that day. The cryo second case went quicker and they did not end up in overtime.

Now the model is speeding up. It is repeating this over and over again, with days of two ablation cases. We see the accumulation of days of overtime, hours of overtime [16.00] and also days where both cases got done and there was room for another case. When I say ‘room for another case’, I mean there was at least an hour left that day in that lab. They do not attempt another ablation case but they can go in there and do something that is quicker to do, such as a pacemaker implant, an ICD replacement or some other case. Most of these cases are margin-positive for the hospital, so they want to do more of them if they can. [16.30]

2. Validation

Now that you have seen that, there is another thing that is important to do when you are building models and that is validation. In other words, is the model predicting something that I did not build into it to predict? One of the things we did to validate this model was to look at the time that patients exited the EP lab after the second case. It is a marker for how long both cases took.

We put in how long individual cases would take [17.00] and all the operation of when patients and doctors arrive, room turnover and all those things. We did not necessarily put in when the second case should end, as that was something that the model needed to predict. This shows raw data from the clinical study for when that second case, the afternoon one, would end during a day, and the version of the data from the simulation. [17.30]

3. Variabilities

First, we eyed this up and said, ‘Well, that looks very similar. We are feeling good about the model. It was not biased above or below what was predicted in real life.’ Then we did some statistics on it, to look at the mean and the variability, and it also looked very similar to the real data. That gave us more confidence that the model was properly accounting for all of the different variabilities.

I am making it sound cleaner than it really was. When we looked at the very first version of the model, [18.00] these were off from each other and we explored why this was. We talked about it and found, ‘Oh, there was something about the variability of how the doctor acted that I did not get the first time around in conversations with physicians’, so we went in there and got that fixed.

4. Results of the simulation model

a. Overtime

The simulator predicted a meaningful improvement in the metrics for [18.30] efficiency that the hospital administrators indicated were meaningful. The first thing to look at is days of overtime, or overtime avoidance, as I would call it. Overtime avoidance is not necessarily the biggest direct money-saver for hospitals, but it is recognised as something that improves staff satisfaction. You can tell them they save money and that gets their attention, and then they come to this conclusion [19.00] that one of their biggest problems is staff going into overtime.

They genuinely worry, if they overwork their staff, that they will lose them. Nurses have options: they can go and work in another part of the hospital or at a different hospital. Because of the technical difficulty of these procedures, it is very difficult to find a nurse who can support an EP lab procedure. They are hard to replace and to train, so that is important to the hospital administrators. Even so, the costs do [19.30] accumulate meaningfully over time and we show them, by looking at it annually or over a longer period of time, that this adds up as a meaningful cost as well.

b. Incremental procedures

The second metric which has been impactful with our administrators is days with an incremental procedure. If you use RF technology, that happens upwards of 20% of the time. If you use cryoballoon technology for both cases, that happens almost two-thirds of the time. [20.00]

c. Presenting to hospitals

When we talk with our customers about these data, we combine them with outcomes research data into hospital financial databases to estimate hospital costs, revenue and margin per case. These data can be combined with these simulation results to enable estimation of annual financial impact of the choice to use cryoballoon over RF for a specific hospital [20.30] in the United States. We use this sort of presentation a lot at hospitals where the physicians already desire to use cryoballoon technology but have run into a barrier at the administration level in investing in it.

When they need to do these ablation cases, there is a capital outlay at the beginning. There is a generator, as it is called, which is a fairly substantial piece of equipment that sits in the EP lap and generates the energy that allows the ablation to happen. That is a six-figure investment [21.00] for the hospital. When we reach a situation where the physicians believe it is a good clinical choice for their patients and the hospital administrators understand there can be real economic advantages coming from efficiency, then we can overcome these barriers to adoption.

d. Publication

Finally, since we completed this model, started using it with our customers and having some success with it, we went on to publish the output [21.30] of this simulation model in the Journal of Invasive Cardiology last year. This lends further credibility to the localised hospital presentations when we do them.

With that said, I have completed what I wanted to say about my project and I am very interested in having an interaction with you over any questions you might have.

Questions and Answers

Claire Cordeaux

Reece, thank you so much. That was a fascinating study. [22.00] Thank you so much for putting it together for us and taking the time to talk to us about it. A couple of questions have come through, so I am going to put them to you on behalf of everybody else.

One is about the impact that that has had on adoption. Can you say anything about that?

Reece Holbrook

Yes, it is good. I am not a field person, so it is rare [22.30] that I get to go out there and observe. By the time it comes here, I have enabled our field to apply these data, but I do hear anecdotes. I heard a handful of anecdotes when we first released this. Oftentimes, the field person would be so excited about the way it went, because it was a new thing that they had not done before, that they would write me an email to say, ‘Wow, this data really helped me.’ I had a hospital administrator who was completely against [23.00] making the capital investment to allow their physicians to do the first cryoballoon case even though the physicians wanted to. When we presented this data, that enabled them to relent on their position, to go ahead and invest in this and to allow the first cases to be done.

After some months, it became [23.30] less novel to our field and they stopped telling me those stories, but I assume that that continues to go on. It continues to be an important part of the tools that our field uses when they are trying to sell this new technology.

Claire Cordeaux

That is good to know. Do they actually take the simulation out themselves and talk to people about it, or do they just show your slides? [24.00]

Reece Holbrook

That is an excellent question. At the beginning, I had imagined that I would be flying out to hospitals, pulling up the simulation, saying, ‘Tell me your average times’, and running the simulator for them to see it in action. It turns out that, as long as you tell the story in a credible way, the hospital administrators do not necessarily need to see it in action. We do not actually run the simulation in front of them.

In one case, [24.30] we took in some specific hospital data, reran the simulation for them and provided the results for them with their own localised data, but in general they seem willing to accept these generalised results that are published. For the most part, they will accept the simulation data as it is. The thing that we always localise to the individual hospital is the costs, revenue and margin results [25.00]. They are then applied to the more generalised outcome from the simulator.

Claire Cordeaux

Thank you very much.

Another question has come in about whether it is normal practice for you to look at the wider context around your procedure. Could you say anything about that?

Reece Holbrook

Yes. [25.30] I would say it is something we are evolving. I would say that, 10 years ago, we would not have imagined doing anything like this. I am an engineer; this is a very technology-focussed company; we are making the greatest new thing. Our founder, Earl Bakken, always said, ‘You need to walk around in hospitals. You need to watch them do things. They cannot always tell you what is important to them; you need to see it for yourself.’ We did have that part as well, but in general, [26.00] we felt that, if we made a technology like this that we knew made procedures faster, we had done our job and the world would come beating down our door to use it.

As time goes on, we find ourselves really understanding the whole situation and trying to figure out how they would measure their success, which does not end at the thing that we at Medtronic imagined we could measure. This is a thing that our current CEO, Omar Ishrak, is really encouraging the whole company to do. [26.30] If you ever encounter him, and he is often on the news or speaking at conferences, he always says this. We need to be able to show the economic value of our technologies and we need to do it – he would use the words – ‘to express the quantified financial benefit to the target customer.’ It is a very loaded phrase where it has to be quantified so you need a number, it needs to be financial and it needs to be [27.00] a benefit to a target customer.

In the case of what we talked about today, it is the hospital administrator who is most specifically responsible for managing the EP lab resource. It forces us to go out there and say, ‘Who cares about this? Whose life does it affect? Can we get to a point where we show them how it changes dollars in their budget?’ It is an evolution but we have come a long way from where we used to be.

Claire Cordeaux

Yes, it sounds like that is the case. [27.30] Thank you very much.

Another question, if you do not mind, is this: are costs and revenue applied after the simulation or does your model include costs, some of which will be stochastic?

Reece Holbrook

That is an excellent question. When I first started using SIMUL8, I had another project in mind that I wanted to do, which was much more complicated. [28.00] I thought that, as I was learning how to use it, I would do something relatively simple, so I built this model and did not really put the quantified dollars into the model itself. I could have and I have since done that, but this particular model comes out with the result you saw, with days of overtime and days of incremental procedure.

Then, separately, I have a simple spreadsheet calculator that I built that has the results of the simulation in it. Then we go into the outcomes [28.30] research databases. I have taken all this capability and given it to someone in our field organisation, who does it themselves now. If we are doing a specific hospital presentation, they will pull out the specific finances for that hospital and put them in the spreadsheet calculator, and that will generate the final output.

If I was doing it today, I would have to meditate over whether I would want to build that into the [29.00] simulation or if I would do that in a calculator outside, but I think both methods are viable.

Claire Cordeaux

Thank you very much. It is interesting that you outline it in that way. I am guessing that every one of us is on a simulation journey. As you learn more about what you can do and how you can solve problems, you can get more and more complex in the way in which [29.30] you run a model, but actually you have a publication and some real value out of a relatively simple model.

Reece Holbrook

Yes, but one thing I have seen is that, internal to Medtronic, if I build dollars into my simulation, eyes get wide open and they pay better attention to it. With this model, people said, ‘That is neat, Reece’, but it was only when I got it quantified to the dollars [30.00] that I really got attention. I have been leaning a little more towards putting those dollars right in the simulation.