web viewwe’ve been working with both on our side with the servers and getting some aws ......

Meeting Minutes

Subject: Notes from BioSense Governance GroupDate: March 16, 2017, 12:30 pm – 2:00 pm EasternFacilitator: Stacey Hoferka

Present on Call:Associations:ISDS: David SwensonNACCHO: Bill StephensCSTE: Jim Collins ASTHO: Jaime Howgate

Federal:CDC: Roseanne EnglishCDC: Cassandra DavisCDC: David WalkerCDC: Alan DavisCDC: Aaron Kite-Powell CDC: Max WorlundCDC: Rosanne EnglishCDC: Anthony WilsonCDC: Kim RaymondCDC: Sue SwensenCDC: Umed AjaniCDC: Mitch MorrisCDC: Farah Naz (contractor)DoD: Paul LewisVA: Mike Holodniy

State Public Health:Stacey HoferkaCaleb WiedemanNatasha CloseJeff Lee

Local Public Health:Holly WhittakerJessica White

Non-Public Health:

Additional Support Personnel:Hither Jembere (ASTHO)Mary Ann Cooney (ASTHO)Tim Carney (ASTHO)Camille Fields (ASTHO)Roland Gamache (ASTHO)Becky Lampkins (CSTE)Amy Ising (ISDS)Emilie Lamb (ISDS)

Non-BGG Public Health:Krystal CollierRosa ErgasOscar Alleyne

MEETING NOTES:

February meeting notes are being prepared and will be out for review.

© Association of State and Territorial Health Officials 2013 2231 Crystal Drive, Ste 450, Arlington, VA202-371-9090 www.astho.org

Meeting MinutesStacey added functional BioSense Requests to the agenda to be discussed after ASTHO updates.

General CDC Updates

Aaron Kite- Powell gave an update on CDC’s current work. There are a number of ongoing activities as I think has been mentioned in past calls that one of

our main priorities is to optimize the system, make sure it’s running well and functioning as expected and try to improve performance of users’ experience. We’ve been working with both on our side with the servers and getting some AWS functionality to do better monitoring of things up and running. Moving forward, assuming budgets, we want to have enhanced servers, more powerful servers that we plan to bring online in the coming months. That should add resources for users so that ESSENCE can run more quickly and process a larger number of users.

A few users over the last few weeks have put in JIRA tickets that have been really helpful in highlighting when the system’s not responding properly. One of the solutions is to try to automate pinging of the ESSENCE environment to see if it’s going to respond and that is an ongoing activity.

Under optimization, one of the options moving forward for converting free text queries into actual indexable fields has to do with creating what we call CC and DD categories. Right now in ESSENCE there are three of them. So if you log into ESSENCE and you go to the CC and DD category field on the left hand side for available query fields, you’ll see there’s visits of interest, there’s a Nora Virus in Version 1 and there’s a foreign travel query. We want to try to use that environment for building keyword queries that users find particularly useful. It can be something that’s specific to a given site and we can name that query something specific for a site with initials, for instance, and then a descriptive name of what the query is for. We’re gradually building that capability internally creating a process for translating the ESSENCE syntax into a SQL syntax that we then build into the back end of ESSENCE so that it’s selectable for users moving forward. So anytime we can convert a free text query, which is by nature a slower running query, into a query that can run against an index field and perhaps even a field that’s cubed at some point, then we’ll improve the performance on the user side.

There’s a number of things that are being worked on by J2APL. One of them here has to do with removing some additional joins on the national data sources. So if you log into ESSENCE and you click on Data Sources, then you’ll see there are two that say Limited in the name. Those data sources currently look for additional tables to join to as part of the query processing under the hood. We’ve determined with J2APL that that’s probably unnecessary and if we flatten that out a little bit more by adding columns essentially, then we don’t have to do those joins and therefore, we don’t have to slow down a query. So that’s something that is being worked on right now as well. So once that’s done that should speed up queries on that data source. It should also speed up other queries in general because if you have a lot of people running queries against those data sources, then it’s going to slow down other people’s experience at the same time. So it should have downstream effects. Some longer term investigations include how to optimize free text search. It’s such an integral part of the way users use the system, but unfortunately, currently anyway, on such large amounts of data, it is also very slow, particularly in the way that we allow it because you can do wildcarding, you can do partial words, and you can use regular expressions. That sort of flexibility on a large database has a bit of a cost in terms of performance. Also related to that additional work for looking at optimization


Meeting Minutesopportunities has to do with how we can perhaps separate queries across databases or even across servers so that we can utilize additional resources for running the queries more quickly rather than being stuck within one really, really large database, but that, too requires some additional research and additional development work to know how to do it properly. So that’s a brief update on the ESSENCE optimization side of the house. I don’t know if we want to do questions now or wait till everybody goes through their various updates, but I’m open to questions.

o Roland: I think we take a couple questions nowo Stacey: I got a couple. So one of them is that you mentioned about the earlier stuff with

best practices with queries. That maybe bad queries are hanging things up and people are putting zero tickets in so you’re learning a lot from that? I’ve been learning a lot, too, from just Wayne’s webinars where he’s like, this is an example of what you shouldn’t do. Have you been compiling any kind of, even if it’s a rough, informal version of best practices for bad queries?

o Aaron: there is a bit of a list of that and it’s something we’ve talked about putting out in one of the newsletter articles. Some of the examples of those and the problem is some of these are not across the board, but I think we can say in general for many of the queries it does help if you also include your site name in the query rather than just relying on access control to limit you to what you want. It depends on the query to some extent, but it probably won’t hurt to include your site name in the query along with whatever else you’re doing.

o Holly: Just with that there are some queries that we’re trying to build to be accessible and useful across sites. So we’re looking for other ways to optimize that so that when somebody else tries to run the query they can’t edit it. I think that there are some limitations there

o Aaron: You can run the query and then change the site name. You can change the site name as you need to. Even if you included when you share it across sites, they can run it. That first run won’t show anything, but at that point they can edit the query in configuration options. There is an extra step there. I agree. I’ve talked to Wayne about other ways one might edit a query that’s shared with you across sites without having to run it first. So again, that would be a future development item, but it’s something that I recognize is an extra step that really probably shouldn’t be necessary. So that’s something we’ve thought about and shared some.

o Natasha: I have a question about if we include the site name, but if we want to build dashboards or something that would prevent people from filtering down to their region, right?

o Aaron: No, you could add other geographical limitations. It really depends on the purpose of the query

o Natasha: That we learned that the dashboard filter wouldn’t work if you use anything other than patient or facility region

o Aaron: Right. Right now the dash boarding, changing in geography is always limited to that. That’s another development item where it’d be nice to be able to change dashboards to other sites too

o Roland: So I’ve heard three people who are users of the system talk about some of these issues with, Aaron, the developing of the query and sharing the queries and you


Meeting Minutestalked about indexing. Is this a major problem or is it just a minor slow down problem? The question I’m really asking is this an issue for a policy development of some kind that we should be discussing outside of this particular update or not?

o Aaron: I don’t know if that’s an answer that I can give. I’m not sure if it’s a policy related optimization and making sure the queries are running the way people expect is of prime importance for us as we move the ESSENCE environment to running on a national system where it’s typically seen and safe. So whether that’s a policy discussion, I’m not certain. I know that based on looking at query logs, the last time Johns Hopkins University Applied Physics lab looked at a large history of query logs, over 98 percent of all the queries are running in less than two minutes, completing in less than two minutes. It’s really just those queries that are taking much longer and some of them take quite a bit longer where we’re trying to figure out better ways of either running it under the hood so that we can optimize or create new methods for free text querying; those kinds of things that we’re really focused on most.

o Roland: Jessica, you’re on the call. Is this an issue with your local health department as well?

o Jessica: Yes, it is. We have been working very closely with Aaron at NSSP and also with our state partners to try to create a couple of different queries. So we’re looking at opioids and we’re looking at heat-related illnesses; two queries that are free-text based rather than built into the system already. So it does take quite a bit of time, as Aaron’s well aware. And we’re trying to share with other partners like in Region 9, so we are interested in how to best share these types of queries; that sort of thing.

o Roland: Stacey so do you think this is something worth looking at a policy level or not or others on the Governing Board?

o Stacey: So, at least based on some of the things that Aaron outlined as in our future development items, I think if this is a little bit more in line with what we have done in the past with prioritizing functional requirements and on different occasions we’ve reviewed the list of other outstanding issues of things users have requested that they say they need in the system and tried to rank them or prioritize them. Some of the things Aaron described sound like the nice to have later or they’re working on it. I think later we’re going to talk about this a little – have a process where we routinely review open items and get a sense from PDC how much longer it would be and then get a sense from the users how important is it. It might take three months. Do we really focus on this because a lot of users need it?

o Oscar: Do you have a timeframe that you would like us to or I should say focus on how to approach that effort? Meaning should we set a recommendation for three-month, six-month review for setting those requirements or reviews of the requirements or technical suggestion points

o Stacey: Hey, Oscar. So it didn’t make the agenda because some of this stuff came in over the last week or so, but I want have a discussion before I get to stakeholder engagement, I wanted to do functional requirement gathering and I laid out a process, which I’ll go into later, but I was thinking quarterly unless needed more often. We could potentially post something like all the open JIRA tickets. Put them out there for users to comment on over the course of like two weeks, seek some CDC input about how feasible those requests are and then get a back and forth between CDC and the


Meeting Minutesgovernance to say yeah, this seems like this is great or wait, there’s something really needed to be prioritized.

o Roland: This may be along the same line again as what you’re going to talk about, but I was wondering, Aaron, if optimization and performance also includes the data quality end. I was wondering if you were seeing a lot of missing data or quality issues in the data that you can identify with some of the routines you’re going through already.

o Stacy: I think there are some data, yes, data quality fits in with that. I think we have a couple people on the call today who could talk about specific data quality issues around chief complaint that I would like to loop into that conversation of how do we prioritize those needs in terms of functionality that might affect data flow and quality. Yeah, I think those are tied together.

Roseanne provided an update on the Legacy portion of the CDC update.

So as you well know, we’ve been full force on working on getting the Legacy data from the past into the new data flow and into the future that we have now set and into ESSENCE. This is deemed as one of our top priorities and we have as many hands on deck focusing on this. I thought I would first just briefly go through the process we’re using to give you an idea of the steps involved.

The first thing we do is identify and confirm requirements with the sites as far as the Legacy data that they would like to move over from the old world into the new world and equally important, what they do not want to move over. We also review the facility identifiers that are in the old Legacy data and just verify that, assuming again they want move the data over, that the IDs have been registered in the new master facility table and if they have not, we work with the site to determine if this was just an oversight when the MFT was built and we need to add them or do they need to rethink whether they want the data or not. In addition, in some cases the IDs from the old world may be something they want to change. So in short we work with them on any necessary master facility table updates and crosswalk updates.

There is support any special requirements that the site may have, especially sites that are intimate with some of the data quality issues from the past, whether that was due to incoming data that had since been corrected or due to maybe interesting ways the data had been processed in the past. We very thoroughly document those special requirements to ensure that we have them right and can pass them to developers to code accordingly. So once all that’s said and done and we have confirmation and closure from the site on the requirement, we do a test. We grab about six months’ worth of data from the Legacy world, process it through our staging environment and into staging ESSENCE. That gives an opportunity for internal operational staff here to take a look at the data first to make sure we didn’t mess anything up before we pass it onto the sites for their review.

Once the sites have had some time to look at the data and staging and in ESSENCE and hopefully provide a blessing or perhaps they provide some input that requires us to go backwards and make some changes. In either case, when all is said and done and we have that approval, however long it may take, we then flip it over to production. I just wanted to point out that the process is thoughtful, intentionally thoughtful. We’re not just pushing a button and letting it flow through because we want make sure we get it right because we don’t want to do it again.


Meeting Minutes At this time we have almost 4 that have officially gone into production and we have 12 that are

in the site is reviewing state where they’re looking at it and verifying that its data is as expected. As soon as we get their approval we can move them on and push them through into production.

We have 14 sites where we are in that let’s make sure we have your requirements correct mode and are working hard on gathering that information. The other thing to point out is the numbers that I’ve just cited are tied to sites that have sent data through the SFTP mechanism. I won’t get into the gory details of that, but in the past and today as well there were two ways in which data could be routed to the platform. When routed through the SFTP and the old Legacy flow, the data landed in a separate site dedicated database. So the numbers I just cited were for the ones that landed in the separate site dedicated databases. In addition, in the past there was also this PHIN MS routed data and decisions made in the past were such that all the data routed through that mode landed in the same database that I call the PHIN mothership. There are 14 sites that have a footprint PHIN MS in our database. Fifty percent of those also have a footprint in this dedicated SFTP database that is they may have sent through FIN at some point, but then flipped over to SFTP. So we have to ensure that for the 50 percent that are in FIN and also have a footprint in their dedicated data in the Legacy world that we get those together so in the end we have them all in our new site data in the new world. At this time there is almost 350 million rows in the PHIN MS data. Based on the projections, if we change nothing the processing time involved is intimidating at least.

So we’re now looking into possibly splitting this off onto a separate temporary server, the FIN MS processing to try to get that going and not have it be causing any conflicts with our current processing or any conflicts with our ability to move on these SFTP sites that I mentioned. We can go ahead and get those done and in parallel, have this separate island where we’re converting the Legacy data. We’re having ongoing discussions about that and we’ll keep you posted about it, but we’re really excited about closing this door and leaving the past behind so that we can get all this data over into the new world and on into ESEENCE.

We appreciate all the help from the sites that we’ve interacted with so far and working with us to make sure we’ve got the requirements correct, working with us to check out the data, provide some feedback and continue this back and forth until you all are satisfied with pushing the button to go to production. We really appreciate it because we need your input and your help in identifying things that we may have overlooked.

Roseanne also provided an update on the New Sites

Mike asked me to speak about this as well. In brief, he just wanted to let everyone know that we recently held, it was earlier this week, an informational webinar for new sites that may have expressed interest in joining NSSP and there were seven sites that showed up, which was great. It was a great webinar and we’re looking forward to working with the sites as they determine their readiness for providing data to the NSSP platform. As we work with sites over this onboarding period we’ll provide orientation sessions to introduce them to such things as the master facility table and the data flow and our tools and, of course, BAMC and ESSENCE. So he just wanted to let you know that we had this webinar and we had seven folks or sites I should say, express interest and join. Thank you.

o Natasha: I don’t really understand. Are the new sites, sites that have formed a DUA or are they sites that had a DUA but they had never submitted data because I’ve heard


Meeting Minutesseveral times they’re going be bringing on new sites on some sort of quarterly basis or something, but I’ve never really understood what that means.

o Roseanne: So the onboarding windows are intended to at least have two of them throughout the year where we can focus on any new sites of interest. To answer your question, the DUA is among the things that are discussed in that orientation webinar and the importance of that DUA with ASTO and having to get that done. The sites in question, some of them have never really to my knowledge at least, worked with ESSENCE in the past and with NSSP. So some of this may be very new to them, which is, again, why we’re hosting these orientation sessions and providing support contacts so they can address questions. We also are holding weekly calls with the sites to address questions through a forum like that. so I think it’s mainly sites that haven’t had this established DUA yet or maybe have expressed interest last year but weren’t quite ready and with that started working on the DUA, but weren’t ready to send data to us yet.

o Natasha: So they’re new, new sites. They haven’t really had a previous relationship with you guys.

o Roseanne: Right. I will say that some of these that showed up at the door have data from the far, far past that we’re seeing in Legacy, which may have predated even the BioSense 2.0 model. Of course we’ll work with them to see if there would be any reason they would want to keep that data as part of their new site data. Of course we’re not throwing anything out. We’re just making sure we triage any Legacy data to the right place and also don’t triage it if they don’t want it

o Natasha Close: Thank you.o Roland: Any other questions?o Holly: Roseanne, this is Holly. With regards to the Legacy data, how far back are you

seeing data or people asking to send over data from? Four or five years or longer?o Roseanne: Well of course it does vary. There is data as early from 2013 that we see that

folks have wanted. What I did not point out that there were eight sites that certainly want their Legacy data, but they have decided that they would rather work with their facilities to have the facilities send back messages for retrospective visits. I don’t know that they would go back four years or so, but that is something else that is part of our “Legacy conversion” is supporting sites that are opting to resend us messages instead of opting to have their Legacy data that was processed in the old world migrated over, but the data can go as far back as 2013.

Cassandra Davis gave an update on the Evaluation Feedback

We are planning to do an evaluation of the BioSense platform post-transition. So this evaluation is really intended to, after some discussion with internal stakeholders, decided to focus this evaluation on understanding how well the platform and its tools are really functioning and really able to meet the users’ needs, how well you guys are able to use the platform and its tools to be able to conduct your syndromic surveillance activities, make sure the tools are functioning as they’re intended and what type of improvements may be needed. So it was good to hear in the beginning when Aaron was talking about a lot of the things that we’re doing around system optimization and a lot of the things and issues that you guys are already seeing. We would want


Meeting Minutesto be able to collect a lot of that feedback also through this evaluation as you guys think about and use the system to be able to conduct your syndromic surveillance activities.

An evaluation plan was drafted and sent out to Stacy and Tim last week I believe. So we’re looking for feedback from the governance group on this particular plan and what we would like for the group to focus on is to really let us know if the purpose and focus of the evaluation is also in line with the things that you guys are also thinking and the questions that we would like to get answered from the evaluation. Are they appropriate and are they relevant for where we are at this point in time with the BioSense platform.

So we’re looking for that feedback to be able to come back to us by March 21st, if that is a feasible timeframe. Then we can be able to proceed with incorporating the feedback into our plan and moving further with the next steps and being able to refine our data collection tool that will be sent out to site administrators and local users that have currently transitioned and are using the BioSense platform.

o Cassandra, this is Tim Carney. I had it on the agenda in our section to talk about this as well and I’m glad you already covered it, but maybe you can talk the group through the timeline and also the method at which you would like to receive the feedback. Do you want it to be sent directly or to us or how would you like that handled?

The timeline, as I stated, so we’re looking for feedback from the group by March 21st and I think if the feedback could be compiled by maybe Stacie and Tim and then emailed onto me that would be great instead of receiving individual emails or feedback back that way. Then once we receive the feedback, then I’ll take a few days to incorporate that into our plan if there are any major changes that need to be made particularly around our evaluation questions or any of the indicators that we’re looking at collecting for the evaluation. Then be able to incorporate that into our plan. The survey that is being developed, that will be sent out to the users to be able to incorporate any of that feedback or make any changes that may affect some of the questions that we’re asking there as well. So we’re looking at probably about the end of March, so to speak, to get that feedback in, make some necessary changes and then send that in addition to a draft of the survey out to the group again for a final check and then it will go through our internal LMB process, which shouldn’t take very long since this is more of a customer service kind of feedback survey. So we’re really anticipating about a couple of weeks. So about the first two weeks in April we’re looking for that to be approved with the survey being able to go out to the users by mid-April with two to three weeks for the users to be able to complete that survey and submit the information to be back to us to be able to analyze by the end of April. So that’s the timeline that we’re looking at right now and trying to stick to.

o Tim: Excellent. I can definitely help on that. I didn’t know if you wanted us to send out – maybe I can email you if you wanted an updated timeline because I know that on the document that we have you wanted the pilot to go out on the 17th. Sounds like that’s going be pushed back a little bit.

o Cassandra: Yes, that’s going be pushed back a little bit. I can definitely update the timeline and send that back out to you before you send it out to the rest of the group if that’ll work.

o Stacy: I did look it over. I thought it sounded like it was laid pretty well. Seeing the questions they’re pilot testing, it would certainly be a really good way for me to think about giving more feedback. Like saw what the analysis was and what the data metrics were. I can wrap my head around specific questions when we see them and see if that


Meeting Minuteshelps. I would just say two things. Maybe one, what are you thinking would be the response rate of the two weeks? I would think you’d probably want to really encourage almost all sites to respond. That you wouldn’t probably cut it off with a low response rate only to make sure it’s representative as possible with all kinds of users. Did you have a response rate in mind?

o Cassandra: Ideally we would like to get a response from all of the sites that have currently transitioned and are using the platform, but with surveys like this, if we can usually get a good 60-70 percent response rate, then I think that we are doing pretty well. So definitely shooting for 100, but if we can get around 70 then that would be good.

o Stacy: I think that would be good. I think that if you’re promoting it as much as possible or making sure everyone gets to it sounds good and then just feedback on it. I’m sure you guys will be giving that out, but anything you capture in the free text field and stuff it’s always nice if you can share in some way. What one user shares and other people don’t think about mentioning or whatever might be valuable, but I have no other comments right now. I look forward to seeing the pilot survey.

o Cassandra: We are going to put in the newsletter that’s coming out in April about this evaluation and letting people know that it’s coming and be on the lookout for it. So that’s one of the ways that we’re trying to promote this.

o Krystal: Do you have plans to maybe give an update on the surveillance COP calls that ISDS is coordinating because that may be another way to inform the users of what you’re trying to do with your timeline. I think it’ll be towards the end of April, but it’s another place that we can promote it and emphasize how important it is to get this feedback. I guess one of the questions that we see, too, is that once you do receive the survey results, how are you going to share some of that information or do you all have plans to share some of that later with the greater COP?

o Cassandra: We do have plans to share all of the information that we get back. How we’re going to do that, we still are thinking that through. We want to make sure that the information we present is viable and easy to understand as possible. A lot of this information will definitely do some type of report, a more in-depth report that will be used internally, but then also we will do some sort of report out for the general public to know what came out of this and what’s been done because something like this is going to be done periodically as the system continues to evolve, new features and functions are added. We’ll want to continually assess where we are and how well these tools are functioning and meeting people’s needs. So we definitely intend to do a report out that will probably happen over the summertime. So somewhere around there.

o Tim: Will that report be focused on just the members of this group or did you want a more widely distributed strategy on that as well?

o Cassandra: I would probably say a more widely distributed. I think it would be helpful for people to know what’s going on and where we stand with the system as it is to date. So it would probably be on a wider form. Definitely to this group, but then also as Holly mentioned, to the COP and others.

Max gave an update on AMC


Meeting Minutes I’m the IT project manager for the NSSP BioSense platform. So we’ve been working on a number

of different things for the AMC. The one that’s coming out most immediately is we are done working on the user groups that had been under development for quite some time. The plan with the user groups is to deploy it to staging in the next week and let a subset of folks know about that it’s there. We’d like to get your feedback and let y’all see what we’ve done with the user group. So we’re going to leave it in staging for an extended period of time instead of rolling it out quickly. You’ll see the communications come out I believe tomorrow. We invite you to go take a look at the groups, look at the functionality, give us feedback on does it do what you thought it was going to do, is the user interface workable for you and that type of thing.

You’ll be able to create custom user groups and associate groups with rules. We hope that streamlines some of your work processes for granting access to data. So that’s one item that’s coming out in the very near future.

We are currently getting ready to bring to the table to discuss with you and show you the work we’ve done to bring the master facility tool into the AMC. So stay tuned for an invitation from us and a presentation. So I know we’re still working on some of our materials for those presentations. Those are the items I think that are most accurately right now coming up for new development on the AMC, the access and management center.

Another item that is that we are adding a site contacts tab to allow you to manage your site contacts through the AMC as well. So you can enter technical contacts, as well as maybe feed or other data transmission types of contacts. So that work will be specked out and we’d like to engage you and how does that help you meet your needs as far as interacting with the access and management center. Those are the big development items. We had a couple things we worked on last week with Arizona and a few of the other sites related to access rules and how to work with access rules. So we got a little bit of development we’re going to address in the access rules to help people understand I think that the more complicated rules you build, they do impact your performance and sometimes in ESSENCE.

We did a review of everybody’s rules and there were a few sites that we worked with through the application to help streamline the rules a little bit. So we’re evaluating what we could do possibly on the back end to improve the user interface there to help people understand when you build a rule in AMC how does that really translate into ESSENCE because every rule you put into AMC does write over to ESSENCE, but the impact in ESSENCE is not always clear.

If you’ve got any questions or comments or concerns from AMC, I’d be willing to take those now.

o Roland: Natasha, you recently worked on this. Is there anything specific with that policy change you want to bring up?

o Natasha: You’re talking about the language about the national view?o Roland Gamache: Yes.o Natasha: I don’t have anything. I actually was going to bring up to the group Hither had

sent out what she thought was the final language for me to review. We got some communication from NSSP that was sent out to the whole community I think that had very similar language. So I wasn’t really sure if that was even something we were still pursuing or working on. I kind of’ thought when I saw that that we were done, but for some reason NSSP had decided to take that on and send that out.

o Stacy: I think CDC has given Hither and ASTO the list of site admins, but Hither, I didn’t see the final email go out yet. Is the correct or has it gone out and I missed it?


Meeting Minuteso Hither: The final email has not gone out yet, no.oo Natasha: Hither hasn’t sent it out. I was just saying that NSSP sent out a

communication. I don’t know. It was a month or two ago that had very similar language saying that this functionality had been added and changed and to look at your data controls. So I wasn’t sure if we still needed to do this additional messaging.

o Roland: I think we should as a governance group just to make sure it’s formalized.o Natasha: It sounds like other people don’t remember that communication anyway. o Krystal: I as a site admin, Natasha, I’d appreciate some communication coming out from

the governance group on that just to give us some better background on what we should be doing and see that connection in relationship over stuff of what we’re trying to achieve. So if you guys send it out we’d appreciate it.

o Stacy: I think we definitely need to. Hither, do you have a timeline yet for that?o Hither: It’s ready. I’d sent it to Natasha so she can just look over it since she’d written

the original one, but I can send it out right after the call. o Stacy: Is that that good with you, Natasha?o Natasha Close: Yeah, that’s fine.

Sue gave an update to on Communications

We wanted to get some feedback on the portal and on the newsletter. I think that the information that we’re putting out is what you want, but I would like to know if there’s any improvements that we can make or if I can get some feedback. Is there any feedback on the portal at all?

o Natasha: I’ve used the portal a little bit. I haven’t had any complaints about it so far. Everything has been positive in my experience so far.

o Sue: Okay, good. I’m so glad that the knowledge repository is ready to link to. So we’ll be calling attention to that beginning of April.

o Stacy: My feedback on the portal is also that I like it. So far I haven’t had any complaints. I think having the newsletter’s really helpful, too.

o Sue: whenever you have ideas on what information you want us to get out or something you think would be helpful, please do that. What I’d also like to start doing is on the newsletter under the community practice section, work with Emily and Mark to cover topics that might include like what the steering committee is doing and try to get some feedback from the community that we can also share with them. Maybe find out what kind of topics are trending; that kind of information.

o Krystal Collier: Krystal with the EHR vendor work group. I did have a question because I was trying to find out if some of the work groups they have completed a project and would like to share and get the message out that we’ve done something, here’s where we are and make sure the COP knows the kind of things that we’re doing or if we’re looking at feedback for something. Is that something we can work with you to have maybe an update featured in the newsletter?

o Sue: Definitely. Yes. I’m very open to that. I can either link to something, but yeah, we can call attention to it. I like the idea.


Meeting Minutes

ASTHO Updates

Governance Group Nominations (Hither Jembere)Hither Jembere gave an update on the governance group nominations process. The nominations. . The call for nominations has been open since March 1, 2017 and had a closing date of March 15, 2017.

There are 5 seats that are up for election this year: o Two representatives from state and territorial public health

Stacey Hoferka Caleb Wiedeman

o Two representatives from local public health Jeff Lee Holly Whittaker

o One representative from non-public health data providers (hospitals, electronic health records vendors, etc.)

Laura McCrary Currently 2 nominations have been submitted one for local public health, which is Holly

Whittaker who currently holds a seat on the group already and one for state, which is Jill Barber, who is an epidemiologist from North Dakota.

Other updates regarding the nomination process Bill Stephens will be stepping down as a representative for NACCHO

o NACCHO has chosen Biru Yang as his replacement. o Biru is the Informatics Manager at the City of Houston Health Department

The nomination submission will be extended for another week for more nominations to be submitted with the new closing date being March 24, 2017 and the voting following immediately after the nominations closing date.

Discussion o Stacy: I wanted to say I know that there are a few people who have indicated to me that

they’re planning to be nominated. So I’m at least confident we’ll have an additional state nominee and one data provider nominee while we don’t even have the minimum for local health department nominees and it would be ideal to not just have the minimum number of slots with the number of candidates to fill. I want to say we’re getting a few more people in by extending the deadline for a week. So if governance group members could likewise seek out some of the individuals you think might be very interested in these roles, please get a little bit more proactive to encourage them or nominate them so that our list will be filled out at least with the number of candidates that we need and then plus a few additional choices. I think we need local health departments to be reached out to and get some nominations in that respect as well. I’ve reached out to Florida, I always reach out to Arizona who’s in part on this call so I would encourage, again, maybe an Arizona representative. Otherwise, I’ll brainstorm some more and see who else I can come up with.

o Jamie: I’ve been giving it a little thought so I think we are due for a call anyway on the ASHTO side. If I can join that discussion let me know.


Meeting Minuteso Stacy: As of right now the data provider would be someone from Cerner. So that’s a vendor

representative and certainly other data providers should be encouraged to be nominated as well.

o Jessica: Is it true that locals can only nominate locals and states can only nominate states? Is that the rule here?

o Stacey: I think it’s the rule, but if you’ve got someone who’s in a state that needs to be nominated you could probably forward them around. I was trying to look for more data providers and I would have just sent my suggestions to another state to nominate them on my behalf. So probably a work around if you want to recommend someone to be nominated by the appropriate level jurisdiction. You can try that. Send it maybe if you have to ask though if you can’t find someone to do the nomination, but technically, yes, that’s the rule, but if you find someone that’s interested just find an appropriate jurisdiction to nominate them

DUA (Mary Ann Cooney)

Mary Ann provided an update on an ongoing BGG topic.

I’m going to report on something that was brought up at the last governance call. I know many of you had brought up the concern about the expiration dates of the data use agreements that we hold that the states and jurisdictions all hold with ASTO and I’ve looked at the data use agreement and also sent it to our legal advisor.

There is a clause within the DUA under Term and Termination and there is no termination to the data use agreement unless either or both parties want to end the agreement. That is a rolling then agreement that is valid until said time that someone does choose to terminate. I hope that clears things up for everybody. I know CDC is also looking at the data use agreement in general and whether or not there should be further definition about term and termination, but at this time I think we’re all set with the current data use agreements that everyone has. Does anyone have any questions?

Discussion o Stacy: I just want to put this out to the governance group kind of as an awareness piece

then. So the ASTO DUA is in part tied to the cooperative agreement that you guys currently have for being the cloud steward. So as long as that agreement’s in place, the DUA holds. If in 2018 anything changes or if you weren’t the cloud steward, does that terminate the DUAs and would we the governance group or would jurisdictions need to be thinking about what to do at that time? We have a long time to go before that happens right now, but it should at least be on our radars that can the DUAs be extended indefinitely if the cloud steward piece changes or are jurisdictions be – or is there going to be any kind of limbo for them if they have the ASTO DUA, they need a DUA, but the ASTO one’s not applicable anymore. I don’t know if that’s a jurisdictional issue to think about. Like every jurisdiction can decide for themselves what they need or if governance needs to weigh in on that. Again, don’t need a decision immediately, but if it were to come up we would want to make sure that all the users and all the data use is


Meeting Minutessustained in the BioSense platform. So I’d like to at least know that that might be down the road something we have to think about

o Mary Ann: I think that if it came to time that there was going to be any changes with ASTO’s contract or relationship with the cloud, then obviously we would have to bring up the data use agreements and let folks know that ASTO was no longer going to be involved.

o Stacy: Is there anyone on the governance group right now that would see concerns or anything that would need to be thought of in advance that would sustain the cloud and the data we submit?

o Mary Ann: Obviously we would let people know as soon as we became aware that there could be any changes in that regard.

Functional BioSense Requests

Stacey introduced the functional BioSense requests discussion.

There have been a few issues that have come up lately and they’ve been in part very important issues for data quality and syndrome analysis related to chief complaints. I specifically wanted to make sure Bill had a chance to speak to that. Jessica had a chance to speak to that and likewise, Natasha and I are in agreement about some time zone issues, but they might also be things that we need to look out from that broader perspective of how to prioritize things for request. So I will speak to that after I turn it over for a few minutes to Bill and Jessica and see if you guys want to talk about the points that we’ve made with CDC over the last few weeks.

o Bill: Thanks Stacey, for bringing that back in. As we’re getting further and further into the data quality and some of the, as I call it, reconciliation between the BioSense platform and the EESSENCE regional system that we use here, we’ve been looking more and more at the data quality reports and trying to correlate that with some of the results that we’ve been getting on queries and have identified a couple of areas really that have been dramatically impacting the query results that we get primarily using the built-in queries, the national language processor queries that ESSENCE normally runs for centers and sub-centers. We zeroed in on what we thought was a chief complaint discrepancy in that one of the criteria for selecting chief complaints is that the chief complaint that’s actually put into the ESSENCE database is basically selected as the first non-null chief complaint that comes through from the data provider. We were looking at that as a possible cause of a discrepancy because we don’t use that. We actually have been using for quite a number of years here, not only the first one, but any subsequent updates that the providers may provide as the chief complaint goes through. At any rate, some of the things that we’ve discovered about that because we also use not only the chief complaint, but the admit reason. These are the OBX and the PV HO7 segments for those of you that are tracking that kind of detail. What we’ve discovered however, is that aside from those is that there are also some other elements that can dramatically alter the query results here, such as zip code that determines location of patient records giving rise to other variations. I would say we’re in the middle right now trying to determine which affects here, either the chief complaint definition that’s used by the system for actually identifying that versus some other missing or invalidated elements


Meeting Minutesthat may be influencing dramatically the query results that we’re seeing. We’re still trying to resolve that. Been working with Roseanne and her team and we’re making some good progress along those lines.

o Stacey: Thanks, Bill, for that. I know that that’s going to be something we have to discuss and come to a resolution and we take a closer look at it. At least I know from my understanding from parts of the problem is it resembled a lot some of the conversations that we had when we initially had the data flow discussions and I can’t say for sure how it’ll end up, but some of the solutions that they might pose that will help address this seem to align very well with things that the data work flow group was amenable to. I can very clearly remember conversations that do us just take chief complaints.

o I think to come to resolution, we can share it with the user community and see if there’s any new concerns, but so far I think this will all be very familiar since myself, Natasha, I think Roseanne has worked on that when we did data flow discussions. So I’m not surprised it’s an issue and definitely think there’s resolution that we can come to. So that’s all good.

o Bill: I think Roseanne’s already come up with some pretty good alternate methods there that will allow us to choose between the chief complaint and the admit reason as we do in the logic at the front end without requiring any other massive changes to the data flow logic that’s already in place.

o Natasha: I was just going to say I think some of the problem with this is that one size doesn’t fit all because I was excited that they were only using the first---because we specifically had one facility that was testing on all the diagnoses once they became available. So our chief complaints were getting huge and there was all this stuff that really wasn’t all that relevant. In noticing that, there was a glitch where they weren’t keeping the first complaint or the messages weren’t being processed in the right order. That glitch has since been fixed, but the data that has been processed that way, they’re holding off to re-process until we figure out what we want to do with chief complaints because at that point there will probably be re-processing of all the data. If anyone wants to clarify if I stated that wrong.

o Roseanne: The issue was if messages were coming in where a latest message date actually preceded a message that came in that had an earlier message date. Let’s say a message date was on March 14th and that’s the latest one so far and then the next one came in and it was March 10th. In that glitch on the ESSENCE side, they were using that last message that arrived even though the message date was earlier. They have fixed that so that prospectively it’s back to the way the requirement had originally specified process it based on the message date time; not the physical order of the records. As for the chief complaint, and thank you, Bill, for taking time the other day to talk with me. I guess there’s two things. One is balancing the volunteers group input from last year on keep the chief complaint fewer and per the guide use the OBX chief complaint and deal with the nuances over the years of OBX chief complaint coming in as coded versus text or coming in as coded, but it looks like text and all of that jazz. The other debate is this first/last thing. My hope and I don’t know, Bill, if this would work out, but if the first non/no admit reason is meeting your need, then the addition or potential addition of admission into the chief complaint processing may not be as big of a change as trying to change it where no, now we’re going to use the last one, you see, but I don’t want to get


Meeting Minutesinto the details. I know that we don’t have enough time for that and nor this may not be the right forum for it, but we’re looking at it and Michael asked just for us to check it out and come up with some potential ideas to share back with him and the BGG. Thanks again, Bill, for your time.

o Jessica: I don’t have too much to add and don’t want to extend this conversation too much, but we would like to be on those conversations. We are finding very similar things that we’d like changes to.

Within the discussion on BioSense Functional Requirements Natasha talked about Time Zone

Natasha: This came up at the NSSP grantee meeting when Roseanne was going over the various reports they put together and the timeliness report. At that time it was stated that they were either Eastern Standard Time, but it’s arrive date time that they’re using, which is in Greenwich Mean Time. Is that correct, Roseanne? So at least for specific time zone regions that adds on eight hours to every message that comes in. This means it’s automatically eight hours old when it arrives. So there was some discussion about whether there’s something we could be using instead. The problem with the messages is that they don’t have a time zone stamp on them so you don’t necessarily know if they’re using the local time of the facility or whether they’re using some other time. In our experience, I assume that it’s going to be in our local time and I think that that’s correct maybe five percent of the time. I’m not actually aware of anyone that’s not using our local time zones. I could foresee if they have a national organization and there’s a centralized source that might not be in their local time zone, but I think that would be accurate more often than just leaving it as GMT for everyone. So I communicated this to Roseanne already and she said she’s got ideas about what can be used across the board. I’m open to hear them. So put that out there.

o Roland: Thank you. Questions? o Stacey: I want to see if people are okay with this. My thoughts were how to capture

these kinds of feedback and when to discuss them to the extent that governance maybe see them pushed very strongly for certain ones being prioritized or needing to be reconciled against conflicting interests of different user sites. This gets us back to that old functional requirements gathering process that we had way back when we reviewed 600 old tickets and tried to figure out what was most needed. Then we got ESSENCE and it picked like 550 of them. So we can start over with a smaller number I think. But I wanted to propose something like this outline, which would be some frequency as needed. My really initial proposal would be to quarterly post a geo report or more specifically like out of the current weekly technical geo report. I think there’s a really good section called the Community Enhancement Request that if quarterly that section could maybe even be expanded with a little detail about what people are looking for and posted somewhere for user community review and input. Hopefully, on the ISDS website somewhere or the NSSP portal. So this is our quarterly review of user requirements that have been put into GRS that haven’t been addressed, how important are they, what do we do first. That sort of discussion could take place with CDC then giving a little bit of feasibility back saying, “I know that users want this. This is going to take six months’ development and all our resources versus these ten things that we can do in a week.” I think it needs to just be a process of on a regular basis looking at what


Meeting Minutesusers are asking for, seeing if they have any comments on a regular basis to review so they know what other users are asking for and can weigh in, getting CDC input and then going over to the governance group for approval if everything’s cool and CDC’s recommendations sound like they’re aligned with what they are capable of doing and what’s important. But if there’s something that comes up that’s critical and it can’t get resolved, then the governance group would probably have to work through a process of saying, “Well what do we do? We have these conflicting needs. Both jurisdictions need this. How do we resolve this?” That would be something that would resign the governance group to work through processes we’ve already had in place or refine this better if need be.

o How do people, what do you react to this? Is this a process you think would work? Does it even make sense to everyone? Are there things we should change? I don’t want to put people on the spot to approve it today unless everyone thinks it’s totally great, but let me see where we’re at. I just want to make sure we tell CDC there’s some process in place to do this on a regular basis going forward.

o Natasha Close: I’m excited that we’re going to setup some sort of formal let’s review what people are asking for and prioritize them. I don’t know that we want to review all open items. I think your suggestion of narrowing down the certain tier categories for the functional – it may show up as people report bugs or something. Maybe we can boil those down somehow because I know those open lists get pretty long of just all the JIRA tickets.

o Stacy: 600 was a lot. That’s not ideal. I would hope maybe if it happens on a regular basis then a lot of the requests might fold into something we’ve seen before. Once this kicks off it may not be as intimidating. Until we know how big it is or how intimidating, I don’t even know how to boil it down or categorize it yet, but I would agree with that. Make it manageable. We had some pretty broad categories and then ended up with 50 categories that got prioritized, which didn’t actually sound that bad at the time

o Natasha: There’s probably not that many requests coming in. I know I put in a fair amount, but once you look at all the other open items, it’s kind of’ hairy, but we could take a look

o Stacey: It would help me remember to put them in and also see how my requests fit other requests that have already been made. I would love this myself to keep me in good habits and stuff and understand everyone else’s needs.

o Roland: We’ll take a look at this at ASTO and see how feasible it is. We’ll talk with CDC and with other people. Definitely with you on the call as well and see what we can do to get this refined and talk about it at the next meeting. Is that fair?

o Stacey: It is. Yeah. Maybe by next meeting hopefully you can put something out there for the governance group members to possibly approve officially. That would be what I would really encourage to see happen maybe by next meeting. Officially send something over to CDC suggesting a process, please.

o Roland Gamache: I’d like to talk with them, too. You mentioned a few things about what things are easy to do right away and that’s what I’d like to get some feedback on from them as well.


Meeting Minuteso Stacy: I think that means we’re trying not to get in the middle of their flow of their

normal function. I know they must get things every day and they should keep doing that, but what they can’t do should be what we look at.

o Max: If you’ll reach out to me via email I’ll get an account setup. I know Mark Sum had an account setup that obviously turned off when he rolled off. So I don’t know if ASTO has a current login to JIRA, but there is a report that Mark was running and that we can help you get a report generated that you can run on your own if needed.

o Roland: hank you Max. Appreciate that.

Stakeholder Engagement

Roland Gamache provided an update on the stakeholder engagement workgroup

We had a meeting of the group earlier this week actually. I think the big thing I want to talk about are the objectives that came up from this are to improve EHR data quality. From the information that we’re getting from the EHR, how to get the feedback back to the EHR to improve the quality of some of the information we’re getting for the syndromic surveillance information, looking at intelligence on community health. Part of the end of the line point of this was to get sustainable funding for BioSense by engaging these stakeholders and by adding product value for the community and support for the maintenance and quality for quality improvement.

Stakeholder membership is Casey Hughes, Natasha Close, Brian Dixon, Stacey Hoferka, Holly Whitaker, Jessica White, Laura McCrary and the ASTHO team.

We’re working on that as a group as well. So we’ll have something where people from the governance group can go to look at the progress that we’re making. We have a list of vendors that we’re going to send out but waiting for the team to vet it first before we formalize that as a team, but we have groups in the hospital, like the ED, emergency management, infection control, poison control centers and so forth. Health educators is one that came up that I thought was really good. Looking at including vendors, local public health providers, the DOD and VA were ones that came up as well. Then CDC, of course.

We had some possible cross-cutting youth cases to look to see if this was a youth case we put into place with this type of stakeholder group what we’ll be able to do and does it make sense as far as engaging these stakeholders in a common issue, a common problem. Things like opioid use, community outbreaks, chronic disease and just health intelligence and analytics in general. Community health assessment came up as well.

Data Quality Committee (Krystal Collier)

Krystal gave an update from the group. You already got my first announcement, we have decided to transition our EHR vendor work

group to the data quality committee based on what the steering committee has defined as what a committee should be. We’re still going to add some more information about what that’s going


Meeting Minutesto mean in the future so I’ll keep that short and keep you guys informed of what our progress is and what the group will be doing.

Stacey has asked the past two meetings about where the Meta Data Visualization app is and from Caleb and Harold and Jill and Serena there’s been some great progress on the code and sounds like they’re finalizing some things. We’re going to start looking for options about how to demo this and share with the different groups where we’re at and get some feedback on some things and some direction for where we’d like to go for next steps. I did want to see that if I can arrange a demo if the governance group would be interested in seeing something like that on a future call.

I would like to thank all the governance group members that were able to attend the urgent care webinar that we had at the end of February. We really appreciate the support. I think it was a wonderful call and there was some great feedback that we had from partners and different people working together

Discussion o Stacey: I know we’re really low on time, but can you just mention very quickly. I think for

your April call are you planning to reach out and engage EPIC directly in a discussion?o Krystal: I think what we’re gone do first is there’s quite a few concerns that have come

up about working with EPIC and trying to understand some of the questions to ask and get some information about it. So I think it would be good to meet with jurisdictions first and find out the best next step. I would like to reach out to EPIC and ask some of these questions directly, but I think I want to get a better understanding of the needs of jurisdictions so we have a strategy in place for what we’d like to do because we haven’t engaged that group the same kind of way we had done when there were issues with Cerner. I’d like just to have a plan in place and know exactly what we’re trying to achieve before we just say, “Hey, there’s some issues and people are taking notice.” I did send one of the people that I have contact for and ask or just give some ideas about some of the work that we’re doing for the Meta Data Visualization app that’s just a heads up. By the way, people are looking at the data, we’re talking about some things. If we would like to contact someone in the future are you the right person. I haven’t heard anything back, but I am trying to find the right person that we can start a conversation with after the April call.

o Stacey: Sounds good. I think it’s a good approach, too. The members talking before you engage the vendor, but I think the vendor engagement’s really exciting. It’s important. How it goes and what you guys do will be great for everyone to be aware of. So thanks.

o Krystal: You’re welcome. I think it’s something, too, that would be nice to hear what happens from your group that’s working on this cause I think there’s ways that there are many vendors that we haven’t talked to at all. I think it would be nice to approach them and get some ideas about what they think working with syndromic. Just some feedback. Then hopefully some of these issues we all have of not understanding for the right questions to ask or some of the issues that come up. Just have some better direction because syndromic isn’t new anymore. We should have some people that are familiar with what’s going on and we should have some better bridges built with that group.

o Stacey: I think we probably should do as much as we can to cooperate on that. Digital Bridge is setting a good standard, if you will, for how to get the vendors and the


Meeting Minuteshealthcare professionals and public health all talking together. So to keep an eye on that seeing how successful collaborations go with the vendors. Clearly it’s an indication that vendors will work with public health given whatever the right motivations are

NSSP Community of Practice Update (Emilie Lamb)

Emily gave an update on ISDS’ work on the CoP. The two biggest things for the community of practice are the surveillance knowledge repository

was released yesterday. So that site is now live. The URL for that is www.surveillancerepository – all one word – dot org (www.surveillancerepository.org ) and that’s the repository of various surveillance related resources. There are lots of functionalities included in the repository that I won’t go into on this call because of time, but I would encourage everybody to take a look at that site.

If you are having difficulty accessing it from CDC, we’ve been working. For some reason it was on the blocked sites list. I’m not entirely sure why, but I think it’s been resolved now. I heard that from one of our CDC folks that you should be able to access the site now.

o You don’t need a login to access any of the resources on there. If you want to submit a comment on a resource, you will need to create a member account on the knowledge repository. So there’s no cost associated with it. You simply need to create a user account so that we can know who’s submitting the comments. We don’t allow anonymous comments on the resources.

o So there’s a registration page to register an account with the knowledge repository. You simply provide just some really basic information and then you should be able to create an account, but again you don’t need an account to access any of the resources. It’s simply a way if you want to submit a comment.

The NSSP COP steering committee has released a call for nominations. We currently have a steering committee of six members and there is a place for two additional members. So if you know somebody who would like to be part of the steering committee, please nominate them.

o You can simply send the information about the nomination to my email address. We don’t have a form for you to fill out. You simply provide a copy of the nominee’s resume and a short justification for the nomination. That call is open until the end of the month.

o We waited for the BGG nominations calls to be closed so that we wouldn’t get confused, but I just heard you guys are keeping it open for one more week. So hopefully people won’t get confused between the call for the BGG and the call for the NSSP COP steering committee.

The next surveillance COP call is on Tuesday, March 28 at 3:00.

Next Steps ASTHO to send out the post transition evaluation plan for feedback. Stakeholder Engagement to meet


http://www.surveillancerepository.org/

web viewwe’ve been working with both on our side with the servers and getting some aws ......

Documents