transforming ehr interactions using voice assistants...1 transforming ehr interactions using voice...
TRANSCRIPT
1
Transforming EHR Interactions Using Voice Assistants
Session 164, February 13, 2019
Yaa Kumah-Crystal, MD MPH
Timothy Coffman
Vanderbilt University Medical Center
2
Yaa Kumah-Crystal, MD MPH
Discloses the following relevant relationship with commercial interests:
• Research partnership with Nuance Communications
• Use of NLP libraries for research and development
Conflict of Interest
3
Timothy Coffman
Discloses the following relevant relationship with commercial interests:
• Research partnership with Nuance Communications
• Use of NLP libraries for research and development
Conflict of Interest
4
• Principles employed in creating a voice user interface for the electronic health record
• Challenges faced in data capture
• Challenges faced in data presentation
• Assessment
• Implementation and future plans
Agenda
5
• Discuss the value of artificial intelligent (AI)-enabled natural language processing and understanding technologies in the advancement of healthcare
• Discuss the limitations of current keyboard and mouse workflows in the EHR
• Describe the usability design considerations, information request mapping and artificial intelligence/natural language understanding platform built to process voice requests
• Discuss the provider cohort experience and outcomes around satisfaction
Learning Objectives
6
Informal Poll
Own a smart phone
Use your smart phone voice assistant
• Siri, Google assistant, Bixby
Own a voice assistant device
• Alexa, Google home, Apple Homepod
7
Why Voice?
8
Documentation Modalities
Writing
~13
WPM
Typing
~40
WPM
Speech
~150
WPM
9
Why not Voice?
10
Macros
Physical exam
Insert normal physical exam
11
Vanderbilt Innovations Portfolio Research
12
Given
• Installed EHR
• EHR vendor has existing partnership with natural language
services provider (including dictation)
• Existing in-house innovation capacity– Infrastructure supporting custom development
– Team in place with necessary SW development skills
– Mission to develop expertise in new ways to interact with the EHR
Decision
• Develop new software to integrate EHR with additional natural
language services
Solutions
13
Well really, how how hard can it be?
13
14
Oh...
15
Demonstration
16
17
VEVA Processing
NLP
User Query
Speech to text
Intent Fulfillment Text to speech
Results Display
What is her A1C?
Sally’s A1C is 8.6
18
VEVA Architecture
19
• EHR’s FHIR implementation does not yet support all desired data
• Additional development required for each new query
• New phrasings of questions requiring updates to language model
• Microphone and speaker issues in a virtual desktop environment
• Latency
Technical Limitations
20
Focus on developing demonstration of capabilities supporting
• One (1) specialty clinic
• Handful of general responses
• Aligned with FHIR (DSTU2) resources
• Some EHR-proprietary services
Build in capability to extend with
• Additional capabilities for other specialties
• General responses, discovered through user feedback
• FHIR (STU3) and FHIR (R4) resources as available
• FHIR-based alternatives to proprietary services
Planning for Extension
21
• MVP
• Rapid Iteration
• Vanderbilt as a Design Shop
Technical Considerations
22
Integrating voice response involves real-time services
• Audio streaming capabilities
– Virtual desktops introduce challenges
• Automatic connections
• Latency
– Varying hardware installed at point-of-use
• Individual laptop microphones
• High-end dictation microphones
• All of the above
Infrastructure Challenges
23
• “While voice is a natural interface for humans, this may be the first
time we are using it as an interface to have a computer
communicate back to us like this. Different rules may apply.”
Design Considerations
24
• Brevity vs. Clarity
• “Blood pressure is 120 millimeters of mercury over 80 millimeters of mercury ”
• “Blood pressure is 120 over 80”
• Uncanny Valley of Words
• I ate three big red apples
• I ate red big three apples
• Quantity, Opinion, Size, Age, Shape, Color, Material, Purpose
Design Considerations
25
Use Cases
26
• Preparing to see a patient in a provider workroom
– Large screen
– PHI private
Use Case A
27
• Reviewing information with patient present
– Large screen
– PHI with patient 3rd party
Use Case B
28
• In transition in a private environment (commute)
– Small Screen (mobile)- limited visibility
– Primarily verbal navigation
– PHI private
Use Case C
29
• In transition in a public environment (hallway)
– Small screen (mobile)
– PHI considerations
(aloud only with headphones?)
Use Case D
30
Usability Study
31
• Setting: Vanderbilt University Medical Center
• Cohort: 14 Pediatric Endocrinology Providers
• Female (10) | Male (4)
• Mean age 39, range 29-65
Study Design
32
• Usability Assessment
• 6 Skills Evaluated
– Training: Hello / What can you do
– Summary
– A1C
– Weight
– Blood Pressure
– Health Maintenance
Study Design
33
• Value of information communicated
• Length of content communicated
• Value of graphical display
• Overall thoughts about voice interactions
Feedback
34
System Usability Scale: Positively Worded
Strongly Disagree
Mean 4.12 (SD .46) 3.93 (SD 1.20)
4.50 (SD .52)
3.79 (SD .70)
4.00 (SD .79)
4.43 (SD .51)
Strongly Agree
35
System Usability Scale: Negatively Worded
Mean 1.67 (SD .66)
Strongly Disagree Strongly Agree
1.36 (SD .84)
1.71 (SD .83)
1.79 (SD 1.19)
1.64 (SD .74)
1.86 (SD 1.03)
36
• Mean System Usability Scale score 80.7 (highly usable)
• Scores of >68 are considered above average
• Summary
• Relevant medications, other problems
• Weight
• Growth chart
• Blood pressure
• Percentile information
User Feedback
37
Willingness to Use System
36%64%
NoYes
4 count9 count
2
1
1
38
• Incorporation of User Feedback
• Desktop Time Comparison
• Scalability
• Alerts and Decision Support
Next Steps
39
• Understand and measure provider frustration with EHR
• Create the business case for optimizing their experience and saving providers time within their EHR workflow
• Assemble a cross-functional team to build the overarching model taking into consideration the providers’ pain points, workflow, and information needs
• Build and iterate a voice assistant prototype while users test and provide feedback
• Understand information theory and map queries and content to satisfy users’ needs
Best Practices/Recommendations
40
An Interesting Study About Voice
Nationwide Children's Hospital in Columbus, Ohio
41
It looks like you have a question.
Yes Go away Clippy!
Questions
Please remember to complete the online session evaluation
42
Contact Us