conversational services for multi-agency situational ...conversational things envisioning of isr...
TRANSCRIPT
Conversational Services for Multi-Agency Situational Understanding
Alun Preece (Cardiff)Dave Braines (IBM/Cardiff)
A bit of context• 2006-2016: US ARL & UK MoD funded
IBM-led £90M+ Network & Information Sciences (NIS) International Technology Alliance
• 2015: Partly based on the success of NIS ITA, Cardiff University established an interdisciplinary (soc sci + com sci) Crime and Security Research Institute (CSRI)
• 2016: ARL/MoD awarded the IBM-led $80M+ Distributed Analytics & Information Sciences (DAIS) ITA for up to 10 years
– Preece is UK Academic Technical Area Lead (TAL); Braines is UK Industry TAL
• 2020: CSRI will move into a new £60M building housing the “World’s First” Social Science Research Park (SPARK)
NATO Summit “OSCAR” Cardiff / IBM collaboration (NIS
ITA, 2014)
SPARK Building (2020)
DAIS ITA Goals
§ Derivation of situational understanding of complex situations by human users synergistically supported by machines
§ Distributed integration & exploitation of coalition data & information across heterogeneous information infrastructures
§ Dynamic adaptation of secure, resilient context-aware information systems
Advance Distributed Analytics & Information Science for Coalitions
Zero-Overhead Principle
“No feature may add training costs to the user”
“We observed that regardless of how powerful the new technology was that we deployed, our largest challenge was getting the analysts to adopt it.”DJ Patil, Building For The Enterprise — The Zero Overhead Principle, TechCrunch, 2012
Conversational Things
Envisioning of ISR data-to-decision chains in terms of ‘conversations’ among humans, devices, and services
Conversational Sensing (2014)Conversational Sensemaking (2015)Conversational Homes (2017)
Analytics services Decision makerData sources
Human-Machine Conversation (NIS ITA)Thinking and processing should be as close as possible
…so we need a language that is both thinkable and processable
Natural Language Controlled
NaturalLanguage
JavaXML
LogicProlog
Proc
essi
ng
Articulation as Language
Photographer: Sebastian Kaulitzki | Agency: Dreamstime.com
http://github.com/ce-store
Controlled English (CE)• Talking to machines in natural language is ideal but hard• CE as a compromise: “easy to read, harder to write”• Let’s bring the two together:
– Human users write NL sentences [easy to write]– Machine users convert to CE [easy to process]– Machine users respond in CE by default [easy to read]
there is a person named ‘John Smith’ that lives in Cardiff
and is a doctor.
low complexityno ambiguityITA ControlledEnglish (CE)
http://github.com/ce-store
Knowledge modelling in CEConcepts are introduced via “conceptualise” sentences:
conceptualise a ~ group ~ G.conceptualise an ~ ideology ~ I.
Relationships and properties can be added:conceptualise the group G ~ promotes ~ the ideology I and has the value T as ~ twitter account ~.
promotes is a relationship; twitter account is a property
Open Source CE Implementations
• Feature light (e.g., no CE rules)• Runs offline on edge devices or
server via NodeJS• Policies specify inter-agent comms• Open source: cenode.io
ce-storeCENode
• Feature rich• Extensible agent architecture• Java and HTTP APIs• Distributed design, in-memory DB• Open source: github.com/ce-store
ConversationalinterfaceWe defined a “speech act” protocol to enable interactions that flow between NL and CE
ask/tell
confirm
why
gist/expand
NL to CE
CE to CE
CE to NL
SHERLOCK is a crowdsourcing game, simulating tactical intelligence tasksSimpleHumanExperimentRegardingLocallyObservedCollectiveKnowledge
video: http://bit.ly/1O2jtsb
Participant taskingParticipants are given a set of ‘mugshots’ of the POIs so they can recognise them on 18 postersThey are tasked with answering 36 questions, e.g.:
What character eats oranges?Where is Giraffe?What character is wearing a red hat?What sport does Hippopotamus play?
IEEE HMS paper, May 2017
Reports results from the first SHERLOCK game (human actors):• Usability (operationalized as performance of adding facts to the
KB): despite close to zero training, 74% of the users inputted NL that was machine interpretable and addressed the assigned tasks
• Agent interaction capability (confirm-only vs confirm+ask-tell): no difference in performance
http://orca.cf.ac.uk/100131/
ICCRTS paper, Sept 2016(International Command & Control Research Symposium)
Reports results from 2nd set of runs (cartoon posters) comparing:• Online Condition: shared KB dynamically updated• Offline Condition: simulating unreliable connectivity at the edgeOffline participants outperformed online participants in information quantity, with no difference in quality
http://orca.cf.ac.uk/93425/Best paper award
Experimentation and outreachOct 2016
May
201
6
Watson@Wimbledon (IBM UK)Bringing the fans closer to the data via conversations
video: http://bit.ly/1knCLyI
• IBM Watsonfor unstructuredtext data
• Our conversational research for structured (database) data
• Realtime alerts based on stream processing
Conversational sensemaking
http://upsi.org.uk/oscar
OSCAR: Conversational Sensemaking
Primary site
Field teams
Secondary site
Social mediadata
NL
CE
Apps
KB
See: sl.dais-ita.org & nis-ita.org
Knowledge management in NIS & DAIS
Conversational places
9th International Conference on Advanced Cognitive Technologies and Applications, April 2017
(journal version in progress)
Recent experiment:• 12 participants• Simulated apartment
environment• 5 tasks (e.g, “Turn on the hall
light”)• Users are productive with zero
traininghttp://orca.cf.ac.uk/99165/Best paper award
“Alexa, ask Sherlock…”
https://flyingsparx.net/blog
Conversational City Hub
Tellability – fast data
Zero training & tooling – just chat
Transparency – ‘why?’ / logging
SPARK (early 2019)Reflection: what are the use cases?
“the world’s first Social Science Research Park”
• Co-location of academia, industry, and public sector stakeholders• Facilities include visualisation lab, secure data facility, human-information
interaction experimentation suite• Huge potential for socio/cognitive research and development
SPARK (early 2019)
Thanks to…Jon BakdashDave BrainesAndy DawsonDiyana DobrevaDan Harborne
Elliot HowellsMartin InnesTrudy LoweNick O’LearyGavin Pearson
Diego PizzocaroColin RobertsAnna ThomasWill WebberleyErin Zaroukian