Simcoach Standard Patient StudioA new kind of Virtual Patient

Thomas B. Talbot, MDMedVR GroupInstitute for Creative Technologies

2

Overview

$8M of DoD Advanced Technology R&D Project

A large public trial and distribution of the technology over 4 years

Ambitious scope, yet uses mature technology to mitigate risk

Usable, refined tools for the public– Free – Goal is to create a “critical mass” with the medical education community

Serious emphasis on student assessment

Designed with ‘lessons learned’ from prior work in mind

Desire to replicate the “gold standard” or surpass it

3

What it is:

Conversational Virtual Standardized Patient (The “Clinic”)– Includes plethora of ICT technology:

Avatars, Natural Language Understanding, Artificial Intelligence Dialogue System, Automated Non-Verbal Behavior Generation, Emotional Expression, Trust-based Responses

– Works through web-browsers & tablets

Online Authoring System (The Studio)– Guided authoring system targeting medical educators

Leverages prepared ‘personalities’ that are modified Learner ‘questions’ are mostly pre-packaged because they are the most

difficult aspect of patient authoring Two-phase “open questioning” / “review of system” model Simple assessment authoring integrated into tools

– Shared assets with forum and user rating system

4

The Standard Patient Studio Team

ICT Teams– MedVR Group

Rizzo, Talbot, Williams and others Screenwriting Team

– Simcoach Group Eric Forbell

– Integrated Virtual Human Team Arno Hartholt Includes Art Team

– Assessment Group Chad Lane

External Partners– Breakaway Ltd,

Serious game development company, Timonium, MD

– USC Keck School of Medicine Dr. Win May and others

– Uniformed Services University School of Medicine

National Capitol Area Simulation Center Gil Muniz, Alan Liu, et al.

5

What it is not: (lots of things are called “Virtual Patients”)

Online Case Presentation– Those depend mostly on pattern recognition

VR Surgery Simulation– Examples: Immersion Medical, Simbionix

Avatar-based Scenario Simulation– Examples: Pulse!!, HumanSim, others…– Procedurally oriented systems

6

Human Standardized Patients

The “Gold Standard” dialogue-based training system– Rely on student initiative to drive encounter dialogue, ask questions,

dynamically follow leads– Close approximation to actual patient encounters– Genuine physical exam and findings

Disadvantages– Expensive, difficult retention– Some pathologies & ages unavailable– Few opportunities to access them– Uncontrolled variation, subjective assessment

There is no desire to replace or eliminate HSPs– HSPs can be retargeted where inherent advantages lie

7

Problems with previous “Virtual Patients”:

Lack Dialogue

Not Readily Authorable

Emphasize things that computer sims aren’t good for

Lack critical interactive technologies

Avatar is often superfluous

Lack automated & well designed assessment

Lack breadth

Are inflexible

Lack of content ‘critical mass’

Target populations that do not need the training

Too expensive, too proprietary

Do not emulate gold-standard approach

Attempted things that were too complicated

8

SPS: Achieving A Gold Standard Interaction

1. Reading the chart

2. The patient interview (VSP session)1. Greeting the patient

2. Obtaining the chief complaint

3. Establishing Rapport

4. Open questioning

5. Specific questioning (Review of Systems)

3. Patient Examination (single text page or VSP session)1. Physical Examination

2. Review Laboratory or Radiology information

4. Make a Diagnosis and Treatment Plan (single menu page)

5. Counsel the patient on the diagnosis and treatment plan (VSP session)

6. Assessment (visual summary of skills and progress)1. Errors of commission and omission

2. System’s estimate of progress on learning skills and sub-skillsGREY: Lower Emphasis Areas BLACK: Higher Emphasis Areas

9

A Variety of Encounter Types

10

The Patient Interview

Virtual Human Encounter– Full ICT Virtual Human Capabilities– Typed Input, Verbal & Nonverbal Output

Natural Language Recognition Speech recognition year 3

On-Screen Educational Guides– Assessment Indicators

Rapport, Achievements Not used for evaluation oriented patients

– Virtual Attending Physician (VAP) Automatically asks a question when

pressed

Conversation– Free-text Question/Answer model

Patient may ask a multiple choice question– Open to Closed Questioning– Answers rapport dependent

11

Virtual Standardized Patient Interview Details

Based on pre-authored “Personalities”– Will create a very well-built out patient with negative/normal responses to a wide variety of

medical questions– At least five personality variants will be created– Most educator-authored patients will be based off these personalities

Variety of appearances– Educator will be able to select sex, face, skin, habitus, clothing, etc– Non-verbal tone and baseline rapport is selectable– Respiratory rate/depth can be set or based upon physiology engine data at authoring

Guided authoring by medical educators– First, enter chief complaint, diagnosis– Second, enter patient descriptive dialoge in response to open-ended questioning– Third, select items from “Review of Systems” and history lists, alter responses for

appropriate items and select items for evaluation scoring

12

Patient Interview Editor (Open Phase)

13

14

Virtual Attending Physician (VAP)

A Virtual Human Encounter– White Coat Attending– Simulated Socratic Session

Multiple-choice INOTS-like encounter– Choices with consequences– Unique feedback based upon learner response

Multiple Uses– Patient diagnosis & evaluation– Teaching points– Test for understanding

15

Physical Examination Phase

Different than other physical exams– Features are resource constrained– Will not attempt to do things that can now be done better via other means.

Multimedia interface– Pictures, sounds, videos or animations can be loaded– Think “X-Rays, rashes, auscultation, etc”– Lab/rad results available (physiology engine derived or author entered)

Available via menu or command line

Unique Virtual Human Encounter– Non-verbal, non-dialogue– Responds to commands to perform maneuvers

Neurological & Musculoskeletal focused “Stand on one leg and close your eyes”

– Easy “check the box” based authoring

16

Patient Diagnosis & Ordering

Simple One Page Multimedia Form– Select diagnosis from choices– Select options to order for patient plan

Auto-populates labs/radiology from physical exam phase

17

Patient Counseling

Virtual Human Session– Dialogue Based, Multiple-Choice or both? (undecided)– Resource Constrained– Optional

What the learner should do– Explain the diagnosis & treatment plan– Answer questions– Assess for patient understanding

18

Assessment Screen: Robust & Graphical

– Interview Phase Rapport & Trust Open/Closed question ratio Must Ask Items (critical) Nice to have asked items (secondary) Acts of commission (bad things you said) Efficiency

– VAP Encounter Score-based feedback or no Feedback

– Physical Examination Phase Critical & secondary items selected Efficiency score

– Counseling Phase Undetermined (limited assessment in this version)

– Diagnosis & Treatment Direct feedback on selections made vs. optimal

19

Initial Use Cases: “Bread & Butter” Medical Student / Intern Cases

20

Development Plan

Development Phase (18-24 months)– Construct tools, interface– Build prototype baseline-personality– Initial cases built & secondary personalities authored– Research-based question database optimization

Public Use Phase (24 months)– Open to second-level partners for first three months– Open publically– Public case authoring, student use, forum community– Public presentations & seminars at national events

Reporting

21

Research Questions

Standard Patient Encounters– How useful are virtual standardized patient encounters?– What are the comparative advantages and disadvantages of VSPs compared to human

standardized patients?– What kinds of patients, diagnoses or other characteristics are best/worst for VSP

encounters?– What phase of the clinical encounter is preferred by educators for a VSP encounter?– How do students respond to the presence of a virtual attending physician (VAP) quizzing

and guiding them? Is it useful?– How can VSP encounters best be blended with live standardized patients?

What are the cost implications? How can use of VSPs free up human resources to expand availability of human

standardized patients (HSPs)? If so, what will HSPs being doing more of?

22

Research Questions

Authoring– Can medical educators successfully author compelling VSP interactions when this

formerly required teams of experts?– What are the unexpected uses and author populations for this technology? Nurses, allied

health, dentists, pharmacists, sales, customer service?– What are the limitations for educator authored VSPs?

What features requires more sophisticated authors? How can authoring be improved to extend these domains to less sophisticated

authors?

Assessment– Are VSP encounter assessments equal or superior to traditional HSP encounter

assessments? How?– Is there value to offering a third person perspective replay?– How can an automated assessment be successful in graphically depicting a concise

assessment of student performance that has utility for guiding future student performance?

– How does VSP encounter performance change with repetition? What is the optimal number of repetitions?

– Are VSPs and the assessment good enough for use in state-mandated testing?

23

Research Questions

Capabilities– What capabilities are in greatest practical use?– What capabilities are not being used? Why?– What are the additional capabilities desired by the medical education community?

Other Results– Number of cases and patients authored– Student assessment and performance data– Author and student feedback– Project impact results

24

OPTIONS

Advanced Physical Examination Package ($4M, 2-year effort starting in year 2 or 3) - This option creates a sophisticated physical exam capability that is beyond the scope of the primary effort. Capabilities include:

– Large inventory of physical movements triggered by student commands, including sophisticated musculoskeletal and neurological examinations

– Advanced graphical ear/nose/throat. Includes advanced exams of the eye, retina, ear canal, eardrum, nares, throat and mouth.

– Advanced auscultation system – Use of large scale multifocal recording arrays will truly replicate listening-site specific heart and lung auscultation in real time to include anatomically appropriate and referred sounds.

– High fidelity skin pathology system – ability to show a variety of rashes and lesions all over the body. The skin should also alter color based upon anemia. Leverages ICT’s advanced graphics leadership.

– Haptic nodule / lymph node palpation – Works through acoustic or physical feedback, depending on device

– Physical manipulation and prodding – simulated palpation and reflex hammering– Physical performance maneuvers – simulated orthopaedic tests– Genitalia Examination – surface and external exams only.

25

OPTIONS

Extended Ethnicity, Physical Impairments, and Pediatric Package ($2.5M, 1-year effort starting in year 3 or 4) – This option adds models of different ethnicities, physical impairments, and children of both genders. Capabilities include:

– Multiple ethnicity options for all SSPS character archetypes– Multiple physical abnormality options across SSPS character archetypes– Infant Examination – features mother for interview, standard infant examination– Toddler – simulated experience with a toddler in a room– Early and middle childhood school children - dialogue personalities– Early and mid adolescents– Pediatric Physical models- includes age appropriate anatomy, selectable Tanner staging, acne and

common rashes– Parent in the room- features two virtual humans in the encounter– Child voices and sounds

26

OPTIONS

Advanced Assessment, Counseling and Decision-making Package ($1.5M, 1-year effort starting in year 3 or 4) – Capabilities include:

– Counseling Editor Wizards – create a variety of counseling type interactions– Counseling Assessment System – detailed assessment reporting for counseling– Cost & Resource Assessment – educational feedback on cost and resource efficiency, including

educationally-focused cost accounting for student decisions.– Third Person Perspective Replay- with non-verbal behavior depiction of student doctor via avatar– User state sensing – employing Kinect or a webcam sensor to provide assessment data on student

arousal, eye contact, facial expression, etc.– Additional Research Year –increases the total SSPS research time and public availability by 50%.

Additional data collection & content creation Additional content authoring Prototype for state level VSP testing Demonstrated integrated VSP/HSP curriculum

27

OPTIONS

Pan-National Cooperative Medical Content Consortium ($2M, 2-year effort starting in year 2) - Capabilities include:

– National Virtual Standardized Patient Consortium – a number of medical schools, and perhaps nursing or dental schools from across the country will join forces to create a critical mass of content as a national VSP curriculum.

– Twelve New Content Creation Awards - A solicitation will be put out for non-USC schools to participate in $100,000 sub-awards to create standardized patients and contribute to the research effort. Available content is expected to increase 8-10 fold.

– National Convention – This will establish a national VSP curriculum that is coordinated across the country.– Expanded Research Report - The report will include more expanded results from the now-larger and

broadly based research effort.

28

SUMMARY

The Simcoach Standard Patient Studio uses ICT’s unparalleled technology

The impact of SPS will be revolutionary

SPS will be a special national resource

SPS will closely approximate HSP encounters, but with some advantages

SPS will be useful for both learning & evaluation

SPS will provide objective student data