denis gouin valérie lavigne alexandre bergeron-guyard innovative interfaces and interactions group...
TRANSCRIPT
Denis GouinValérie LavigneAlexandre Bergeron-Guyard
Innovative Interfaces and Interactions GroupIntelligence and Information SectionDRDC Valcartier
KSCO 2012February 2012
Human-Computer Interaction with an Intelligence Virtual Analyst
2
iVAC initiative
• Intelligent Software Assistant (ISA) identified by MIT’s Technology Review as one of 2009’s most promising emerging technologies
• Conversational, computer-generated characters capable of providing guidance to a user in the conduct of his tasks
• Intelligence Virtual Analyst Capability (iVAC) R&D initiative at Defence R&D Canada
• 3-year initiative started in 2011
• Knowledge System with important HCI component
3
Virtual Assistant / Virtual Advisor
IBM's Watson supercomputerDARPA’s CALO Cognitive Assistant
that Learns and Organizes
SIRI Apps on IPhone 4s
4
DARPA’s Personal Assistant that Learns (PAL)
http://www.youtube.com/watch?v=BF-KNFlOocQ&feature=youtu.be
Video Clip presentation
5
Cognitive Assistant that Learns and Organizes (CALO)
6
HCI Requirements - General
• HCI dialogue between iVAC and the analyst(s)
– How to input knowledge into the system
– How to raise questions
– How to task the IVA
– How does the IVA ask for precisions on the questions and the way of presenting information
• Optimization of the presentation of the results
• One and/or multiple virtual analysts can serve multiple users
• Virtual analysts can interact with each other within and across meeting spaces
7
Enabling HCI Technologies
• Smart Room Environments
• Multimodal Interaction
• Information Presentation and Adaptive Interfaces
• Augmented Cognition
• ISA Representation / Avatars
• Storytelling
8
Smart Room Environments
• Instrumented environments (e.g. cameras, microphones, biometry)
• Ubiquitous computing. The user can access the system from various locations
• Track users location
• Track users activities
US Navy Command Center of the Future (CCoF)
DSTO LiveSpaces
9
Multimodal Interaction
• Allow the user(s) to employ multimodal interaction (e.g. voice, pointing, gesture, eye/gaze, neural/brain interfaces, emotion detection)
• Resolve ambiguities
– These are my priorities
– We should move that milestone back 6 months
Xbox Kinect
AFRL Interactive Data WallEmotion Detection, Valenti et al. 2007
10
Information Presentation
• In support of various tasks
– Present tools and information
– Answer questions
– Ask for clarification
– Remind the user of some tasks or procedures
– Propose alternative possibilities
– Give a briefing
11
Information Presentation
• Means
– Voice output and/or information display
– Highlight information elements
– Display information in a new window, organized as the user wants to see it
– Gather a set of documents
– Filter information based on user requests
– Capture information about a meeting (automatic transcription and semantic analysis of multiparty meetings )
12
Adaptive Interfaces
• Customized to the role and current tasks of the user
• Present tools and information based on the user preferences
• Suggest sequences of events based on learned tasks and preferences
• Understand the context / tasks
• Observe what the user is doing and his mental state
• Do not disturb
• The user is recognized using biometry
• The interface adapts to the distance of the user to the (large) display (text fonts and granularity of information)
13
Augmented Cognition
• Track the sensory and cognitive overload of the users
• Use of neural/brain interfaces, emotion detection
• Employ computational strategies to restore operational effectiveness
– Intelligent interruption to improve limited working memory
– Attention management to improve focus during complex tasks
– Cued memory retrieval to improve situational awareness and context recovery
– Modality switching (i.e., audio, visual) to increase information throughput
– When not to disturb
DARPA (2011a), DARPA Defense Science Office, Improving Warfighter Information Intake Under Stress (AugCog), http://www.darpa.mil/dso/archives/warfighter/index.htm
14
ISA Representation / Avatars
• Various representations
– Speech output (voice)
– A simple icon
– A two- or three-dimensional representation of a character (avatar)
PAL
15
ISA Representation / Avatars
• Adaptation of the avatar to the task and socio-cultural context
– Facial and voice features of the avatar (serious / smiling, tone of the voice ) should communicate emotions, danger, risk, uncertainty
– Consider socio-cultural factors (Gender / age / ethnicity / profession)
– Use different avatars to support different ISA tasks (weather prediction or course of action)
• Embedding avatars in a video conferencing
• Be careful of the Spectre of the Uncanny Valley
• Need for experiences with avatar representations
DSTO’s FOCALwww.postgenomic.com/faces
16
Storytelling
• Guide audience through a preconstructed narrative
• Use of tangible graphics and videos (TV news like)
• Richness of the storytelling approach
• Convey situation awareness about complex events
Logo – VisWeek -Telling Stories with Data
17
ISA for Coalition Operations
• Improved interoperability between coalition forces in terms of disseminating information
• Translate information between languages
• Share differences in tactics, techniques and procedures (TTPs)
• Synchronizing coalition activities
• Improve cognitive assistance by providing shared coalition awareness and task support
• Enable coalition organizations to learn by managing a knowledge base
• Provide better and faster decision making through access to comprehensive knowledge developed through time
18
Conclusion
• ISA, a knowledge system component
• Solve the human cognitive overload
• Conducting a wide variety of tasks
– Search and organize information
– Learn procedures and preferences
– Manage schedules
– Track people
• HCI is Key
– Multimodal interaction
– Avatar is not mandatory
– Assign tasks
– Summarize documents
– Mediate interactions
– Guide and remind the user
19