Speaker 0: Walkout.
Speaker 0: And so he was supposed to be on a panel
Speaker 0: at some conference, and of course, he he wasn't physically
Speaker 0: there, so they had him on the big screen, but
Speaker 0: they put a black turtleneck and sport coat on the
Speaker 0: back of a chair, so it looked like he was
Speaker 0: there.
Speaker 1: That's funny.
Speaker 1: This meaningful use.
Speaker 1: Still, I know I wrote some papers some time ago
Speaker 1: and meaningful use.
Speaker 1: Is that still a jam?
Speaker 1: Is that still an important thing?
Speaker 1: Or we passed it supposedly ended this past year?
Speaker 0: You know, it got subsumed by some other metrics and
Speaker 0: health system health information systems still need to be certified
Speaker 0: using a similar certification process.
Speaker 0: And.
Speaker 0: Yeah, I got a lot more complicated.
Speaker 1: There's still money connected to high tech and the affordable
Speaker 1: care.
Speaker 0: Yeah, that's what I mean.
Speaker 0: No, that all that'll end it now.
Speaker 0: OK, now it's just, you know, you just have to
Speaker 0: do those things to get your money from CMS.
Speaker 1: Thank you.
Speaker 1: Well, we passed the top of the hour, and so
Speaker 1: I think we'll go ahead and get started and thank
Speaker 1: you, everyone.
Speaker 1: It's my distinct pleasure to introduce Dr. Jim McClay.
Speaker 1: Jim is an emergency room physician and clinical informatics with
Speaker 1: a lot of expertize.
Speaker 1: Dr. Jim Campbell and I recruited Dr. McClay in 2001
Speaker 1: to help.
Speaker 1: Are at that time, health informatics program.
Speaker 1: Jim has been very active in Nebraska Medicine, helping getting
Speaker 1: our epic set up.
Speaker 1: He has been very active in April seven and a
Speaker 1: number of organizations, and recently Jim has accepted a new
Speaker 1: position and is professor of health management and informatics and
Speaker 1: chief research and informatics officer for the University of Missouri.
Speaker 1: And what he promises as part of his job to
Speaker 1: keep with the Center for Intelligent Health Care.
Speaker 1: So Jim, thank you very much, and I'll turn the
Speaker 1: table over to you.
Speaker 0: OK, good.
Speaker 0: Can you see my screen?
Speaker 0: Yes, I successfully shared it, and I'm sharing the slides,
Speaker 0: not the speaker notes.
Speaker 0: OK, well, so a lot of you know me, I'm
Speaker 0: Jim McClay.
Speaker 0: I am in the Department of Emergency Medicine at UMC,
Speaker 0: and I was going to talk to you about improving
Speaker 0: data literacy and quality in order to develop our learning
Speaker 0: health system and give you some examples of what we've
Speaker 0: been able to achieve with our work over the past
Speaker 0: few years.
Speaker 0: So let's see here.
Speaker 0: Next slide So just some disclosures.
Speaker 0: Yes, I've been here.
Speaker 0: I came in 2001 and I have been somewhere along
Speaker 0: the line I inherited being chair of the Biomedical Informatics
Speaker 0: Graduate Program.
Speaker 0: And with the help of Jim Campbell, we have set
Speaker 0: up the clinical research analytics environment, which we'll talk a
Speaker 0: little bit more about, and that provides the core for
Speaker 0: our participation in the Patient-Centered Research Network.
Speaker 0: And important thing to know, John Wendel, is that I'll
Speaker 0: be one of the instrumental people in the Greater Plains
Speaker 0: collaborative.
Speaker 0: Which is the research network that we work in, and
Speaker 0: it's it's based in Missouri, at the University of Missouri
Speaker 0: and UMC and University of Utah, both collaborate on that
Speaker 0: network.
Speaker 0: So we all work together already.
Speaker 0: And then I've been part part of the Greater Plains
Speaker 0: Biomedical Informatics and Cyber Infrastructure Corps for our CTO.
Speaker 0: If you don't know about the CPR will point to
Speaker 0: that a little bit, too.
Speaker 0: And then I've been involved at a health level seven,
Speaker 0: which is the standard body for the past 15 years
Speaker 0: or so.
Speaker 0: Somehow, I just kept going after everybody else left and
Speaker 0: been representing you and NZ at that organization.
Speaker 0: So what I wanted to do was take a little
Speaker 0: bit of a controversial comment here, and we had the
Speaker 0: opportunity at UMC to participate in the creation of a
Speaker 0: maturity model for assessing the success of research.
Speaker 0: Informatics at medical centers and stones was published as the
Speaker 0: Research Informatics Open Science Maturity Model.
Speaker 0: And I was going to pull up the the model
Speaker 0: for you so you can see that it's relatively straightforward,
Speaker 0: easy to use tool with five questions.
Speaker 0: And actually in this website, they actually made available some
Speaker 0: other assessment models one that might we might all be
Speaker 0: interested in health care analytics adoption model, which is from
Speaker 0: health management systems society.
Speaker 0: But in the research informatics survey, it's a five question
Speaker 0: survey with the typical comments about leadership and engagement.
Speaker 0: And if we go through and we we select August,
Speaker 0: of course, go through and select the top one for
Speaker 0: all the laughs you get done and get to.
Speaker 0: Oh.
Speaker 0: I think I put too many things.
Speaker 0: When you read the results and it shows you where
Speaker 0: you stand in comparison to the other institutions that have
Speaker 0: filled the shop and.
Speaker 0: So we just go to the next slide, these are
Speaker 0: the main questions that are asked and it's about leadership,
Speaker 0: engagement, informatics and data governance.
Speaker 0: How important informatics is research informatics is to the value
Speaker 0: of the company or of the institution.
Speaker 0: So if you're at University of Utah, one of which
Speaker 0: was the first informatics program in the country and it's
Speaker 0: been forever known for for their ability to develop their
Speaker 0: own information system and decision support tools, so big reputational
Speaker 0: value there and then asking about opening, supporting open science
Speaker 0: and sharing, and then developing the resources of the institution
Speaker 0: to develop, to share, to motivate innovation and sharing of
Speaker 0: these resources.
Speaker 0: And so if we think about that and what we've
Speaker 0: done here at our institution over the years is to
Speaker 0: work, to overcome the barriers, to reusing our data that
Speaker 0: we collect for research.
Speaker 0: And her you know, it's it's a bit of a
Speaker 0: whack whack a mole kind of situation and that these
Speaker 0: barriers continually arise and we figure out how to overcome
Speaker 0: them.
Speaker 0: But if for any of our researchers on campus, it's
Speaker 0: not unusual for them to deal with problems of getting
Speaker 0: access to data.
Speaker 0: We've had two conversations today at other meetings talking about
Speaker 0: getting access from a public health department and from other
Speaker 0: institutions.
Speaker 0: And then you kind of see how to integrate the
Speaker 0: data that you've collected from other from multiple sources and
Speaker 0: then figure out how to share.
Speaker 0: So whenever we set up sharing between institutions, we have
Speaker 0: to set up a day to use agreement and a
Speaker 0: data sharing agreement that a business associate screaming and screaming.
Speaker 0: And then there's the ongoing concern about the security and
Speaker 0: confidentiality and preserving part of it.
Speaker 0: So these barriers are well-recognized, continually being dealt with, and
Speaker 0: I think we've made some significant progress.
Speaker 0: And so one of the things that really helps is
Speaker 0: to have a good data governance process at the institution
Speaker 0: or the organization and to recognize that we're never going
Speaker 0: to be perfect at it.
Speaker 0: Whether it's just a continuous process in our in our
Speaker 0: goal of data governance is to improve data access and
Speaker 0: use, making the data that's available accessible in a usable
Speaker 0: fashion and then to promote data literacy, which is a
Speaker 0: concept that is being recognized now that we are flooded
Speaker 0: with data as an important factor in growing this as
Speaker 0: we discover that we're really not very data literate and
Speaker 0: then improving the data quality so that it can actually
Speaker 0: answer questions in a reliable fashion.
Speaker 0: And the Center for Intelligent Health Care has a major
Speaker 0: concern there as we work towards developing intelligent algorithms of
Speaker 0: mine.
Speaker 0: This data to help us make decisions and of making
Speaker 0: sure that the data reminding is of good quality, but
Speaker 0: also avoiding biases and ethical issues.
Speaker 0: That's right.
Speaker 0: So data literacy, one of those things that I'm only
Speaker 0: now beginning to realize is what I struggle with when
Speaker 0: I'm just journaling.
Speaker 0: Working with organizational leadership is, is that this?
Speaker 0: And we need to develop a recognition that we have
Speaker 0: to be able to understand what we're doing with the
Speaker 0: data that we need to know how to access it,
Speaker 0: assess what it is that we're accessing and how good
Speaker 0: it is and interpret it, manipulate, manage, summarize all of
Speaker 0: the data omics kind of comments that we make.
Speaker 0: So when we work with our researchers, we go through
Speaker 0: a process which is to some extent reflected in what
Speaker 0: you could imagine for developing data literacy or helping somebody
Speaker 0: who isn't as literate about the data to a point
Speaker 0: of applying.
Speaker 0: So at the end of the presentation, I'll point you
Speaker 0: at our clinical translational research resources.
Speaker 0: And one of them is the cycle of going through
Speaker 0: this process.
Speaker 0: So when there's a question to be answered, we need
Speaker 0: to determine what data it's going to take to answer
Speaker 0: the question and understand what data we need, which data
Speaker 0: sources are appropriate and what types and formats those data
Speaker 0: come in.
Speaker 0: It's a it's one of the one step in here
Speaker 0: is to really do a careful assessment of the quality
Speaker 0: of the data and and I'll talk about that in
Speaker 0: a minute.
Speaker 0: But so when we work with our teams, we're working
Speaker 0: to kind of continually work through this cycle and develop
Speaker 0: more and more educated.
Speaker 0: Researchers and clinicians and data users to really understand how
Speaker 0: they can take the data and apply it to their
Speaker 0: issues, whether they're trying to learn or make decisions or
Speaker 0: or generate evidence or solve problems.
Speaker 0: And so the concept would be to go from what
Speaker 0: we frequently see now, which is a environment where data
Speaker 0: is.
Speaker 0: Are kept in silos generated and non standardized formats maybe
Speaker 0: even kept on paper and and surrounded by mythology and
Speaker 0: concern about security and privacy and what we can and
Speaker 0: can't do with it from a regulatory standpoint to a
Speaker 0: to really using that data to grow us as a
Speaker 0: learning organization.
Speaker 0: And so we really are trying to develop tools to
Speaker 0: access and analyze the data for results.
Speaker 0: And Center for Intelligent Health Care, right, it's not about
Speaker 0: just having the data, it's about doing something useful and
Speaker 0: then knowing how to actually query data to get answers
Speaker 0: and building tools to automate the process.
Speaker 0: And developing a standardized platform where data is either physically
Speaker 0: or virtually stored and accessed in a common format that
Speaker 0: makes it easier for people to use.
Speaker 0: So I've really been kind of struggling down that road
Speaker 0: over many years and and one of the big concerns
Speaker 0: that we hear repeatedly is that the data, the real
Speaker 0: world data that we collect in the course of our
Speaker 0: general care processes, you know, the health care as a
Speaker 0: as a national complex system collects myriads of data of
Speaker 0: all different kinds and all different formats.
Speaker 0: But there's this big concern about the quality of the
Speaker 0: data.
Speaker 0: And how do you how do you draw a conclusion
Speaker 0: from this data that's that's suspicious for its quality?
Speaker 0: You can imagine how many transforms.
Speaker 0: Any any raw data has to go through before it
Speaker 0: can actually be incorporated into a research study and at
Speaker 0: every step of the way, we get a little concerned
Speaker 0: about loss of fidelity of the data.
Speaker 0: So we actually divide up the assessment of the quality
Speaker 0: of data into categories and then develop metrics around those
Speaker 0: categories.
Speaker 0: So we're really continually striving to have a complete set
Speaker 0: of data that is, in the case of the clinical
Speaker 0: environment, fully describes the patient and that there is no
Speaker 0: drop out of the data that might be available.
Speaker 0: And then we have consistent data and that in many
Speaker 0: cases, it's how it's stored and indexed and linked together
Speaker 0: so that it's good as data accrues over time, it's
Speaker 0: synchronized and it's not conflict.
Speaker 0: We're also complying with standardized formats for our data and,
Speaker 0: you know, working in the health level seven world or
Speaker 0: working with Jim Campbell, that standards are very important.
Speaker 0: They allow us to both measure and share the data
Speaker 0: in a meaningful fashion without having to make things up
Speaker 0: as we go along.
Speaker 0: And then, of course, you know, reflecting the real world
Speaker 0: that the data that you have accurately reflects so.
Speaker 0: These these categories are all things that we work with
Speaker 0: and massage and try to improve, and we have a
Speaker 0: pilot project that we're just getting ready to launch with
Speaker 0: a collaborator.
Speaker 0: San Antonio has 63000 automated data quality checks for electronic
Speaker 0: health data we have in our in our data part.
Speaker 0: So so that's a big enough problem that I served
Speaker 0: as a National Quality Forum on Technical Expert panel looking
Speaker 0: at how we can assess and improve the data that
Speaker 0: we get from our electronic health records for for research
Speaker 0: reporting quality metrics.
Speaker 0: And that report is what's published.
Speaker 0: Oh, well, about a year and a half ago and
Speaker 0: really kind of makes, you know, it's mostly about making
Speaker 0: recommendations for policymakers and federal agencies and how they can
Speaker 0: improve data quality.
Speaker 0: What kind of processes shouldn't go?
Speaker 0: So that's trying to fix things at the policy level.
Speaker 0: And then we work as part of the Patient-Centered Research
Speaker 0: Network, which we've talked about before.
Speaker 0: I can talk about it more in a different setting
Speaker 0: if you want to know more about it.
Speaker 0: Cornett is a it represents about a hundred million patient
Speaker 0: lives across the country, and the data is extracted in
Speaker 0: a standardized format kept locally at our at our individual
Speaker 0: data marts and then queried through a federated system.
Speaker 0: And that data set is carefully evaluated for quality on
Speaker 0: a quarterly basis.
Speaker 0: So we go through a extensive process of extracting the
Speaker 0: data and putting it into a standardized format and then
Speaker 0: running a long list of data, completeness and quality of
Speaker 0: metrics against it, making sure that the data conforms to
Speaker 0: a standard format.
Speaker 0: There's a number of plausibility steps in there.
Speaker 0: You know that you know you don't have, you know,
Speaker 0: a nine foot tall patient who weighs 15 pounds and
Speaker 0: things like that.
Speaker 0: And that data is complete over time, and we don't
Speaker 0: have dropouts in the data that we develop.
Speaker 0: And then that we try to keep our data up
Speaker 0: to date.
Speaker 0: Those of you who have ever worked with federal databases
Speaker 0: and data from CMS, you know that it could be
Speaker 0: a couple of years old.
Speaker 0: So we try to keep our data down to about
Speaker 0: 30 to 90 days old at the maximum, and then
Speaker 0: we use this data to to participate in research studies.
Speaker 0: So a lot of talk about those because I think
Speaker 0: it's important to see that it's a it's a continuous
Speaker 0: cycle.
Speaker 0: We we create a data mart.
Speaker 0: We work across the country to make it work.
Speaker 0: We then try it out, see where the gaps are
Speaker 0: and we go back to the to our source data
Speaker 0: and we try to improve things and we go back
Speaker 0: and actually put new measures in that we're collecting and
Speaker 0: then we keep going.
Speaker 0: So I'll start with the idea that we can actually
Speaker 0: use the data that we collect from our electronic health
Speaker 0: record systems.
Speaker 0: This is real.
Speaker 0: This is data from real world care.
Speaker 0: Nothing special is collected and essentially once it's collected and
Speaker 0: put into the data datamart, it is essentially free.
Speaker 0: We just discover this data.
Speaker 0: It's like mining or from from your, your iron mine
Speaker 0: or something like that.
Speaker 0: I grew up in northern Michigan, where copper was so
Speaker 0: common that you could pick up boulders of copper on
Speaker 0: the shoreline.
Speaker 0: And and so we're looking for that kind of data
Speaker 0: so well.
Speaker 0: So the first thing is we'll just try that out,
Speaker 0: and we did a demonstration study that I'll describe to
Speaker 0: you, which is looking at comparative effectiveness of different bariatric
Speaker 0: surgery metrics.
Speaker 0: And then once we recognize we could do that, the
Speaker 0: next question is, well, can we can we use these
Speaker 0: tools that we have to recruit patients in the studies
Speaker 0: and the tools that we have?
Speaker 0: Or, you know, we we have these patients come to
Speaker 0: our clinics and we collect data on them and we
Speaker 0: can we can query that data and we can say,
Speaker 0: Hey, you fit the metrics for the study and then
Speaker 0: we can just point them at a website where they
Speaker 0: can be.
Speaker 0: Both consented and randomized and then followed over time without
Speaker 0: any special effort.
Speaker 0: And then we'll talk about using the H.R. data for
Speaker 0: ongoing disease surveillance.
Speaker 0: So bariatric study was a question for that after we
Speaker 0: sort of surveyed the the investigators around the country for
Speaker 0: good ideas.
Speaker 0: This one came up as a relatively straightforward comparative study.
Speaker 0: Corey McBride, our.
Speaker 0: Bariatric surgeon was one of the lead scientists on this
Speaker 0: project, and it was to compare three surgical procedures for
Speaker 0: weight loss and then look at outcomes over a five
Speaker 0: year period.
Speaker 0: And so it's a common procedure and it makes a
Speaker 0: significant difference.
Speaker 0: And so.
Speaker 0: The model was, OK, we have a number of health
Speaker 0: systems that are reporting their data in a standardized format,
Speaker 0: and so you can see we start with our at
Speaker 0: the bottom there in the diagram with our source data,
Speaker 0: and we go through the process of cleaning that up
Speaker 0: and combining it and creating a data repository that we
Speaker 0: then put it into the format for use by the
Speaker 0: coroner.
Speaker 0: And then each individual site that participates receives a query
Speaker 0: against that data in that fast and fast format.
Speaker 0: So SAS is one of the statistical packages and and
Speaker 0: so since everybody's data is in the same format and
Speaker 0: well-characterized, we only have to create one query against all
Speaker 0: of that.
Speaker 0: Now the question becomes, well, yes, but is it if
Speaker 0: you're if you're sending in aggregate data from each site,
Speaker 0: is that the same as having the data all in
Speaker 0: one place?
Speaker 0: And.
Speaker 0: And so we were able to start with the coronet,
Speaker 0: 100 million patient records and forty one health systems and
Speaker 0: come up with a way.
Speaker 0: We assessed this tool and this was the methodology paper
Speaker 0: that we did where we actually looked at testing the
Speaker 0: the difference between starting with aggregated data and starting with
Speaker 0: this federated query, and was able to show that the
Speaker 0: results were identical.
Speaker 0: We had, you know, Dr. Tello is a statistician and
Speaker 0: did some manipulation there, but it demonstrated that this was
Speaker 0: a feasible way of doing things, and this is the
Speaker 0: the basic source of data that we started with over
Speaker 0: time.
Speaker 0: So we have a 10 year time period where we
Speaker 0: can because we have data from our chargers going back
Speaker 0: that far, we can track patients over that longer period
Speaker 0: and then the occurrence of these different procedures.
Speaker 0: So you can see this the bottom one and the
Speaker 0: yellow is called the gastric banding, and that helps has
Speaker 0: fallen out of favor, as you can see over time.
Speaker 0: And then it used to be a relatively invasive procedure
Speaker 0: where they would go open up the belly and replace
Speaker 0: the stomach and the small intestine, and they call that
Speaker 0: a ruined Y.
Speaker 0: That's the R W R Y and then somebody, some
Speaker 0: brilliant person came up with a what was called a
Speaker 0: sleeve gastrectomy, where they could shrink the size of the
Speaker 0: stomach without doing anything so invasive, and that became a
Speaker 0: more predominant procedure.
Speaker 0: And so the graphic there just shows you by the
Speaker 0: size of the circle.
Speaker 0: How many are in that here?
Speaker 0: So when you get all done with the analytics, we
Speaker 0: are able to look at about sixty five thousand procedures
Speaker 0: following now over five of the outcomes over five years.
Speaker 0: And the zero point on this graph is the point
Speaker 0: at which they get their procedure.
Speaker 0: And we actually have data from the year before.
Speaker 0: So we know their weight prior to getting the procedure
Speaker 0: at a point in time when they get the procedure
Speaker 0: and then we send short patients that we had data
Speaker 0: on for five years and followed them over time to
Speaker 0: find out that the patients that had the ruined whi,
Speaker 0: which is the bottom line there had the highest percentage
Speaker 0: of total weight loss and that it persisted over a
Speaker 0: long period of time, and the colored bars are the
Speaker 0: confidence intervals on that data.
Speaker 0: So there's pretty strong confidence intervals there on these outcomes.
Speaker 0: And and then we as a secondary study, we looked
Speaker 0: at diabetes remission on these patients because that's the other
Speaker 0: goal.
Speaker 0: These people get diabetes and we want their diabetes under
Speaker 0: control.
Speaker 0: And so you can see that the outcomes for the
Speaker 0: rule when sleeve gastrectomy are essentially the same with overlapping
Speaker 0: confidence intervals for the adjustable gastric banding wasn't that good
Speaker 0: at it.
Speaker 0: And I think you could make the argument here that
Speaker 0: there was good reason for the reduction in the amount
Speaker 0: of gastric banding, but this is a really good reason
Speaker 0: to pick one or the other two.
Speaker 0: So now we've demonstrated that we can we can mine
Speaker 0: this data and actually follow patients over time and come
Speaker 0: out with comparative effectiveness research.
Speaker 0: So what if we added one more piece to that?
Speaker 0: We said, Well, now let's see if we can actually
Speaker 0: enroll patients in a prospective.
Speaker 0: Study using a similar model, and so the idea there
Speaker 0: and this was a real struggle for our IRP is
Speaker 0: just that the patients, instead of having to be face
Speaker 0: to face, would be referred to a web portal where
Speaker 0: they would they would go through and get there.
Speaker 0: They would be consented and they would be provided with
Speaker 0: information and queried about their understanding of the information.
Speaker 0: And then based on the results of that, they would
Speaker 0: be enrolled in the trial and they would be randomized
Speaker 0: to that point.
Speaker 0: And then we can follow them over time because they
Speaker 0: are they have.
Speaker 0: We have both a web portal where they'd be queried
Speaker 0: on a regular basis for outcomes, but also we're tracking
Speaker 0: them through the chart for these visits.
Speaker 0: So that model was applied to a study that looked
Speaker 0: at.
Speaker 0: So this is the process looked at aspirin dose for
Speaker 0: patients with a preexisting cardiovascular disease.
Speaker 0: So, you know, a few hundred thousand people are running
Speaker 0: around, haven't had a prior minor stroke or or an
Speaker 0: MRI or other vascular disease that they quite a while
Speaker 0: ago recommended that everybody take aspirin.
Speaker 0: But the problem was there was equipoise about whether they
Speaker 0: should take a baby aspirin or they should take the
Speaker 0: full size of aspirin.
Speaker 0: And why does it matter?
Speaker 0: Is that larger doses of aspirin make it more likely
Speaker 0: to get bleeding complications.
Speaker 0: Lower doses of aspirin don't work as well.
Speaker 0: And so the question was well, for those people that
Speaker 0: are at risk, what did I take?
Speaker 0: Simple question.
Speaker 0: Nobody would ever fund a study like nobody would know.
Speaker 0: Pharma companies interested in that and the National Institute of
Speaker 0: Health.
Speaker 0: This is not an innovative thing.
Speaker 0: They weren't really interested in, either.
Speaker 0: But when it came to actually testing this process and
Speaker 0: the size of the patient potential patient population, we actually
Speaker 0: got funding from Korea and the NIH to run this
Speaker 0: trial.
Speaker 0: And with the effort with an intent to enroll 15000
Speaker 0: patients in a study where they would be identified through
Speaker 0: the air and contacted, we contacted them with letters and
Speaker 0: emails and notifications through the patient portal.
Speaker 0: And then they would get a address to a website
Speaker 0: and they would be given their golden ticket number, which
Speaker 0: is their their identifier that they would use to track
Speaker 0: themselves and randomize them to the two choices after the
Speaker 0: enrolled follow them up over time.
Speaker 0: And we were able to engage 40 centers into this
Speaker 0: process across the country with about and we were able
Speaker 0: to approach four hundred and fifty thousand people to participate
Speaker 0: in the study.
Speaker 0: And you know that one of the barriers to doing
Speaker 0: any kind of study like this is getting the just
Speaker 0: even identifying patients that would participate.
Speaker 0: And we had thirty two thousand of them go and
Speaker 0: visit that portal and find out about the study and
Speaker 0: fifteen thousand enrolled to participate.
Speaker 0: And they were randomized into the two doses.
Speaker 0: And basically, the idea was just, you know, instead of
Speaker 0: when they went down to the pharmacy, they'd buy the
Speaker 0: eighty one or 325 mg bottle.
Speaker 0: And then when they went back to the doctor, they
Speaker 0: would say, I'm taking this dose of battles.
Speaker 0: And so the endpoint there is looking at outcomes.
Speaker 0: There are two outcomes, one being bleeding and the other
Speaker 0: being a recurrence of the other disease.
Speaker 0: And you can see from this from this result that
Speaker 0: there was no essentially no difference in recurrent hospitalization for
Speaker 0: or hospitalization or death from their MI or stroke over
Speaker 0: time.
Speaker 0: And depending on the dose, which which is the dose
Speaker 0: doesn't matter.
Speaker 0: And then the question about bleeding is, well, OK, then
Speaker 0: if the dose doesn't matter for outcomes, then are there
Speaker 0: going to be more likely to get bleeding?
Speaker 0: And then also, that matters.
Speaker 0: So the conclusion is it doesn't matter.
Speaker 0: And and that that's not a really exciting conclusion, it
Speaker 0: wasn't a really exciting study, what was really exciting was
Speaker 0: we were able to enroll 15.
Speaker 0: We were able to access all those patients in the
Speaker 0: role of 15000 patients, Rs 20000 here because that's where
Speaker 0: we started and really only spending about eight hundred and
Speaker 0: fifty dollars per participant enrolled instead of study costs that
Speaker 0: are that I have comparable study costs for something kind
Speaker 0: of similar where there was a pragmatic trial called promise
Speaker 0: for coronary artery disease and that thirty one thousand one
Speaker 0: hundred dollars per purchase and another study on anticoagulation for
Speaker 0: thirty eight thousand dollars per participant.
Speaker 0: So we demonstrated that we could use this mechanism and
Speaker 0: and now that mechanism is being used for a very
Speaker 0: large trial of statins in the elderly because the possibility
Speaker 0: of taking cholesterol medication when you're older, it may reduce
Speaker 0: likelihood of getting to actually do it in the aging,
Speaker 0: but that's going to take a while to study so.
Speaker 0: But we're using this same platform to do that kind
Speaker 0: of a couple of other things that we've done along
Speaker 0: the way.
Speaker 0: John Windell might be interested in this one.
Speaker 0: This is just a a simple oral one by one
Speaker 0: of our investigators who is retrieving data on acute kidney
Speaker 0: injury to see if they could do some machine learning
Speaker 0: and actually identify who's at risk.
Speaker 0: And and predictive variables are things, you know, so acute
Speaker 0: kidney injury happens to people that are critically ill.
Speaker 0: What we do contrast imaging and certain medications.
Speaker 0: And so trying to figure out what the what are
Speaker 0: the risk factors there.
Speaker 0: So this is an ongoing study that we're all waiting
Speaker 0: to see what the results of that are.
Speaker 0: And then now that we have all that data, can
Speaker 0: we can we can we create cohorts of patients that
Speaker 0: are in a sort of a registry kind of a
Speaker 0: model or are being surveilled long term?
Speaker 0: And one of the things that when COVID came along,
Speaker 0: we were all geared up and ready to go and
Speaker 0: it took us about a month and a half to
Speaker 0: spin up the national COVID Cohort Collaborative, which is a
Speaker 0: system where the participants in these research networks who are
Speaker 0: already have their data already curated and are extracting it,
Speaker 0: are now extracting a subset of the data and putting
Speaker 0: it into a secure enclave called the National COVID cohort
Speaker 0: collaborative.
Speaker 0: And I'm not sure exactly.
Speaker 0: I don't have the latest statistics.
Speaker 0: There's a dashboard.
Speaker 0: I could have downloaded today's statistics, but at the time
Speaker 0: I did this and we were sixty seven sites reporting
Speaker 0: and what we report is not our entire experience, but
Speaker 0: all the patients who have been tested and are COVID
Speaker 0: positive.
Speaker 0: And then a two to one propensity match on patients
Speaker 0: who are not COVID positive.
Speaker 0: And that creates a dataset where we can track many
Speaker 0: of the COVID activities because we have data before patients
Speaker 0: are diagnosed.
Speaker 0: I think we go back to 20 beginning of 2019.
Speaker 0: We can find patients that have been exposed to medications
Speaker 0: or various disease processes and look at the correlation of
Speaker 0: outcomes over time.
Speaker 0: And one of the earlier things were to look at,
Speaker 0: you know, there's lots of patients who are hydroxychloroquine.
Speaker 0: You remember hydroxychloroquine.
Speaker 0: It was all the rage back in about a year
Speaker 0: and a half ago.
Speaker 0: But everybody who has some kind of rheumatoid arthritis, arthritis
Speaker 0: or other autoimmune disease is probably already on hydroxychloroquine.
Speaker 0: And so we could look at those patients and we
Speaker 0: could say, Well, you know, I mean, how would they
Speaker 0: do with COVID?
Speaker 0: Because if it's protective, they they should look better than
Speaker 0: other conditions.
Speaker 0: And so there's been publications on that.
Speaker 0: There's a publication that we that was let out of
Speaker 0: and see looking at transplant patients and their outcomes know
Speaker 0: there are the immunocompromised patients.
Speaker 0: And so the B and 3C takes advantage of the
Speaker 0: process that we've been describing where we extract that data
Speaker 0: and check the quality and put it into a standardized
Speaker 0: format.
Speaker 0: But the patient centered research network is not the only
Speaker 0: research network that does that.
Speaker 0: There's another one which is a network, an international network
Speaker 0: led by.
Speaker 0: Columbia, and they use a data model that I'm never
Speaker 0: going to remember all the acronyms, but they use a
Speaker 0: different data model.
Speaker 0: And then there's the national niches, and caps has a
Speaker 0: different model they use.
Speaker 0: I called the accrual the clinical trials network.
Speaker 0: And so what we did was what we did.
Speaker 0: What what some really ingenious implementations did was said, Well,
Speaker 0: we'll just we'll just convert all of those different information
Speaker 0: models into one.
Speaker 0: And they adopted the OMAP information model.
Speaker 0: So all the data is it's all clean and standardized,
Speaker 0: and they just convert it to OMAP.
Speaker 0: It put it into the into the enclave for analysis
Speaker 0: and use the tools for that.
Speaker 0: And so when when an investigator wants to to use
Speaker 0: that in any investigator on our campuses who have executed
Speaker 0: the data used are the data use agreement, which I
Speaker 0: think all of our campuses in the CPR group have
Speaker 0: can apply to get a seat at the enclave and
Speaker 0: go to the you're going to check in and then
Speaker 0: they walk you through the process of onboarding.
Speaker 0: You'd have to make sure that you have your certifications
Speaker 0: and that you take the NIH privacy and security training,
Speaker 0: which is not too bad.
Speaker 0: And then when we're in there, there's this huge community
Speaker 0: of experts, from clinical experts to statisticians to the data
Speaker 0: experts and the there's a lot of pre-made materials that
Speaker 0: are being built up.
Speaker 0: So we have subject matter experts that review of medication,
Speaker 0: which so you can find of all the steroid medications
Speaker 0: and you can find all the, you know, medications that
Speaker 0: are cardiovascular Typekit and then those are made available and
Speaker 0: you can browse all those things.
Speaker 0: You can use tools and then you can start work
Speaker 0: with of the implementations to begin to build your extraction.
Speaker 0: Britain, the number of manuscripts that are coming out is
Speaker 0: increasing.
Speaker 0: Faster than our journal editors can keep up, so a
Speaker 0: lot of these are just going to be published in
Speaker 0: the online archives for review so we can get the
Speaker 0: ideas out there prior to publication.
Speaker 0: But that's an example of how having gone through this
Speaker 0: process in advance when the opportunity came up, we were
Speaker 0: immediately ready to go with creating a the and three
Speaker 0: c and think I have a link here for four
Speaker 0: participants and you can contact us and we'll walk you
Speaker 0: through the process.
Speaker 0: So that's that's a registry idea.
Speaker 0: But there's another one that we've set up, which is
Speaker 0: to use this information model to track blood pressure control.
Speaker 0: Crawl across sites and over time.
Speaker 0: And this is the blood pressure control laboratory.
Speaker 0: It's being run.
Speaker 0: Well.
Speaker 0: And one of the.
Speaker 0: Back over time for each.
Speaker 0: We're doing site level tracking, so the participating sites are
Speaker 0: clinics.
Speaker 0: So health care organizations and clinics, and then there are
Speaker 0: patient populations of each clinic.
Speaker 0: And so it's tracked at the population level.
Speaker 0: We're looking at the metrics quality metrics for blood pressure
Speaker 0: control and and then the data is adjusted and cleaned
Speaker 0: up and then represented to us in a dashboard.
Speaker 0: And this is just one snapshot of the dashboard looking
Speaker 0: at each of the colored blocks is a facility in
Speaker 0: there.
Speaker 0: We don't know whose facility is whose.
Speaker 0: We're just given our phone number and when we.
Speaker 0: And so if you look up at the top right
Speaker 0: there, there's a little box or some kind number facility,
Speaker 0: 12 and I put 12 in there.
Speaker 0: It'll highlight my facility's results in comparison to all the
Speaker 0: other facilities here.
Speaker 0: And so what you're seeing here is overall blood pressure
Speaker 0: control, meaning the number of patients in the cohort at
Speaker 0: that that is served by that clinic that has their
Speaker 0: blood pressure under control.
Speaker 0: And I think it's the 120 over 80 now because
Speaker 0: it used to be a little bit higher.
Speaker 0: And so you see, you know, the bulk of the
Speaker 0: facilities, it's about 60 percent control.
Speaker 0: And then they also provide us with changes over time.
Speaker 0: And we're actually have a manuscript we're putting out looking
Speaker 0: at the impact of COVID on blood pressure control and
Speaker 0: showing that, you know, as people stayed away from their
Speaker 0: health care provider, blood pressure control got a lot worse.
Speaker 0: And and so, you know, kind of a secondary outcome
Speaker 0: of of the lockdown was reduced blood pressure control.
Speaker 0: So there's a number of different ways we can cut
Speaker 0: this up and looking at whether how artists from the
Speaker 0: patients high blood pressure notice to time that they've got
Speaker 0: it change in their blood pressure medication, if that's so
Speaker 0: that's that's another approach.
Speaker 0: So, you know, that's that's just showing that once we
Speaker 0: kind of say, Hey, we're going to do this and
Speaker 0: we're going to go through the process of developing data
Speaker 0: literacy and improving our data quality and then using it,
Speaker 0: we get better and better and it becomes a more
Speaker 0: natural process.
Speaker 0: We've done all the legwork to be ready to participate
Speaker 0: in various studies and then we've gone through and actually
Speaker 0: demonstrated the viability of this approach to generating out of
Speaker 0: that.
Speaker 0: And so this is supported, for the most part by
Speaker 0: our our clinical and Translational Research Program in Nebraska, which
Speaker 0: is funded by A.J. Mouse.
Speaker 0: And we have informatics core and a bio bio, bio
Speaker 0: statistics and epidemia epidemiology and research design course.
Speaker 0: So the statisticians can help you with figuring out your
Speaker 0: analytic approach.
Speaker 0: And the informatics team can help you with getting access
Speaker 0: to data and determining the data that you need, and
Speaker 0: then going through the cycle of coming up with an
Speaker 0: idea of having access to the data, improving your question
Speaker 0: refinement and running queries, and getting to the point where
Speaker 0: you can publish a paper or get a grant.
Speaker 0: And so encourage everybody to if they're not already to
Speaker 0: participate in the Great Plains of CPR and take advantage
Speaker 0: of the resources that are rolling out and being available
Speaker 0: here.
Speaker 0: We uh, we have a link to these resources on
Speaker 0: the website, the CPR.
Speaker 0: You don't have CW biomedical informatics core, and we also
Speaker 0: have on that site the data dictionary for the data
Speaker 0: that we have available and detailed data dictionary and then
Speaker 0: a browser to allow our investigators to better.
Speaker 0: Aren't they have to be on our campus, but to
Speaker 0: actually look at the.
Speaker 0: Types of data and the volume of data that's in
Speaker 0: the data warehouse, I typed up data and and codes
Speaker 0: and so, you know, this is now a resource that
Speaker 0: has been demonstrated in all these different formats and remains
Speaker 0: active and live.
Speaker 0: And as Tammy demonstrates, the value of developing the skill
Speaker 0: set, the literacy and understanding the use of the data.
Speaker 0: So that's that's the end of my talk.
Speaker 0: So now we're open for questions.
Speaker 1: And thank you very much.
Speaker 1: That was fascinating and I really appreciate all the you
Speaker 1: know, you've condensed.
Speaker 1: Two decades of work into the one thirty six flight
Speaker 1: deck, and I don't think people realize how much work
Speaker 1: that goes into, but how once you set that up,
Speaker 1: how it really is accelerating.
Speaker 1: I actually have two questions for you.
Speaker 1: So Rob Caliphs is being nominated to go back to
Speaker 1: to head the FDA.
Speaker 1: And obviously, Rob has had a lot to do with
Speaker 1: the Duke Clinical Research Institute, etc..
Speaker 1: Where do you see this model and the FDA lining
Speaker 1: up?
Speaker 0: OK, so for those who don't know, I think Rob
Speaker 0: Campbell actually came out in an interview to be our
Speaker 0: chancellor at some point, but he's been a real friend
Speaker 0: of this project, and the initial architecture of the coronet
Speaker 0: was based on FDA's Mini-Sentinel architectures.
Speaker 0: For those who don't know, the FDA does post post
Speaker 0: marketing, monitoring for adverse events and that sort of thing.
Speaker 0: And they called that process sentinel.
Speaker 0: And a number of the sites are just directly part
Speaker 0: of the Sentinel Network now, because of the the models
Speaker 0: the same, the there's a slight difference in the information
Speaker 0: models that the FDA wants, but it's mostly the same.
Speaker 0: And it's mostly the same because Rob Keller in his
Speaker 0: vision, said, You know, at some point the FDA is
Speaker 0: going to want to be able to use this, this
Speaker 0: research network.
Speaker 0: Jim Campbell might want to comment on the fact that
Speaker 0: the FDA is coming to Nebraskans because apparently we're the
Speaker 0: only ones that have the brilliance to do a particular
Speaker 0: project for them.
Speaker 0: So, yeah, so the thought is that Rob Califf will
Speaker 0: go back to the FDA and will help sort of
Speaker 0: beef this up as it were.
Speaker 0: How's that?
Speaker 2: May I comment, Jim?
Speaker 0: Yeah, please do.
Speaker 2: Yeah.
Speaker 2: You know, for those of you who sniff the, you
Speaker 2: know, federal announcements and so forth, you may be aware
Speaker 2: of the fact that a few weeks or a month
Speaker 2: or two back FDA published guidelines for use of real
Speaker 2: world data and regulatory activities for the FDA and the
Speaker 2: project.
Speaker 2: The request for proposals that we have been asked to
Speaker 2: respond to basically looks at the application of data like
Speaker 2: our crime data is that in the, you know, real
Speaker 2: time, if you will, use of real world data, clinical
Speaker 2: data for moving our laboratory test devices approval forward to
Speaker 2: full medical authorization.
Speaker 2: So I think that, you know, there is a very
Speaker 2: close alignment developing John between fed the Fed thinking on
Speaker 2: what is the relevance of what we are doing and.
Speaker 2: You know, I'm glad to say I think we're getting
Speaker 2: some recognition for it.
Speaker 1: Congratulations.
Speaker 1: It is still tentative, but Rob has tentatively agreed to
Speaker 1: come in the fall to our clinical informatics conference in
Speaker 1: October as our keynote speaker.
Speaker 1: That's pending his approval and his approval to be able
Speaker 1: to come out here.
Speaker 1: But that seems very positive.
Speaker 1: The second question I have for you, Dr. McClay, is
Speaker 1: how is Crane going to deal with an LP in
Speaker 1: the medical record?
Speaker 1: How do you see?
Speaker 1: Do you have a plan already in place or how
Speaker 1: exactly forward?
Speaker 0: Yes, we do actually have a plan in place, and
Speaker 0: it's it's a two step process.
Speaker 0: So.
Speaker 0: What Dr. Wiggles referring to is extracting information from a
Speaker 0: prose text, natural language processing, and that can be instruction
Speaker 0: reports like things we get from pathology or unstructured reports
Speaker 0: like the narratives that clinicians dictate and the the idea
Speaker 0: is that there's two steps there.
Speaker 0: One is in order to do privacy preserving research, we
Speaker 0: have to remove all the identifiers.
Speaker 0: Now that's really hard when it's just natural language.
Speaker 0: And so there are algorithms in place that will strip
Speaker 0: identifiers and replace them with tags and in your text.
Speaker 0: And so, for instance, if it's a if it's a
Speaker 0: proper name, they'll just put in brackets, proper name and
Speaker 0: dates and that sort of thing.
Speaker 0: And then once that once the IRP accepts that, that's
Speaker 0: kind of a US institution by institution negotiation, then then
Speaker 0: the data can be made available for your routine natural
Speaker 0: language processing engines.
Speaker 0: And and that is.
Speaker 0: And so that's the second step.
Speaker 0: And so we would be able to strip out text
Speaker 0: for our investigators to look at.
Speaker 0: One of the things that I always think is really
Speaker 0: useful is can we can we can we sift through
Speaker 0: that data in advance and do what our document integrity
Speaker 0: team does?
Speaker 0: For those of you in clinical care is out on
Speaker 0: the floor where you know we're taking care of patients.
Speaker 0: There's a team called the Document Integrity Team and they
Speaker 0: have blue shirts.
Speaker 0: Are they used to have blue shirts?
Speaker 0: They go around and they're they're nurses and clinicians, and
Speaker 0: they and they read the chart and say, Look, doc,
Speaker 0: you know, you've recorded the fact that this person's blood
Speaker 0: pressure's 210 over one 130, but you haven't written down
Speaker 0: that they have hypertension.
Speaker 0: So if you write down that they have hypertension, then
Speaker 0: we can put that in as a diagnosis and and
Speaker 0: it becomes a fact in the record.
Speaker 0: And so that's called document integrity.
Speaker 0: Well, you know, I've always thought that, you know, that
Speaker 0: we could automate a big portion of that and feed
Speaker 0: that back to our our clinical staff while we're also
Speaker 0: recording in our data warehouse.
Speaker 0: And so that's sort of the end goal there.
Speaker 0: It's closing that loop.
Speaker 1: Thank you.
Speaker 3: Jim has the great it actually impressed with how fast
Speaker 3: through the DNA that approaches are and the quality of
Speaker 3: the data so forth.
Speaker 3: And you kind of hit on this a little bit
Speaker 3: sort of the learning health system side of this with
Speaker 3: pragmatic trials during the big ones or learning health system
Speaker 3: may be a bunch of smaller, pragmatic trials that have
Speaker 3: a continuous cycle and so forth.
Speaker 3: What's once you have a good data like this?
Speaker 3: But what's missing to really accomplish that sort of full
Speaker 3: virtual cycle learning health system?
Speaker 3: Well, what are the gaps to taking the research infrastructure
Speaker 3: and really plugging it in and affecting the quality of
Speaker 3: care in a place like yours?
Speaker 3: We have certain gaps here as well.
Speaker 0: Well, that's a lot to unpack for.
Speaker 0: I think that a robust data governance team that includes
Speaker 0: both the clinical side of the research side is important
Speaker 0: and developing that level of understanding and trust in the
Speaker 0: results of these trials are test to the point where
Speaker 0: they would be useful for clinical decision making.
Speaker 0: And one of the one of the things there is,
Speaker 0: you know, the data that's generated out of these research,
Speaker 0: things are not always actionable at the point of care.
Speaker 0: So you got to figure out how to how to
Speaker 0: close that loop so that the results of studies are
Speaker 0: actionable point of care.
Speaker 0: It's nice to know that you know this person's risk
Speaker 0: is 16 percent and this person's eight percent.
Speaker 0: But what do I do with that when I take
Speaker 0: care of a patient?
Speaker 0: And so I think that's a that's a gap that
Speaker 0: it's going to be a little tough to close, but
Speaker 0: really important as more and more data is collected and
Speaker 0: digested.
Speaker 0: You know, there's there's this conceptual model in there that
Speaker 0: it takes so many data points to make a decision
Speaker 0: in health care.
Speaker 0: And when we were, John and I were starting out,
Speaker 0: maybe that was, you know, five or 10 things that
Speaker 0: you needed to know about a patient and you could
Speaker 0: kind of keep that all in your head at the
Speaker 0: time.
Speaker 0: But now there's hundreds to thousands of data points that
Speaker 0: you need to sift through.
Speaker 0: In order to create the database, you need to make
Speaker 0: a wise decision about somebody's health care.
Speaker 0: And we do a lot to digest that.
Speaker 0: But with all the instrumentation we have, it's just a
Speaker 0: firehose of data sifting through that, and I think it's
Speaker 0: probably the biggest thing.
Speaker 0: 558.
Speaker 1: Any other questions?
Speaker 1: Jim, thank you.
Speaker 1: And I look forward to continuing to work with you
Speaker 1: in the future.
Speaker 1: I think we'll have a lot of fun, John.
Speaker 1: I think so.
Speaker 1: That's the whole goal.
Speaker 1: Thank you very much, folks.
Speaker 1: We'll see you next month.
Speaker 0: All right, take care, everybody.
Speaker 0: You.