SiridusSpecification, Interaction and Reconfiguration in

Dialogue Understanding Systems

an information state approach to flexible spoken dialogue systems

Overview

• Siridus project

• Information State Approach

• Main research areas

• Demonstrators

Siridus Project

• Partners– University of Gothenburg (technical coordinator)

– University of the Saarland (administrative coordinator)

– University of Seville

– Telefonica I+D

– Linguamatics

– SRI (original coordinator)

• Duration– Jan 2000 - Dec 2002

Motivation

• Provide flexible, user-centred dialogue systems• Improve reconfigurability of dialogue systems for

– new tasks

– new domains

• Provide architectures allowing the dialogue state to be accessible for speech recognition, generation of prosody etc.

IS transitions

IS for System

IS for System

• dialogue as transitions between IS states•what information do we need in the IS?•how should it be structured?•can this information be reused e.g. synthesis/recognition?

•examples–underspecified commands (DELFOS) –questions under discussion (GODIS)

Interpret U Generate U’ IS for System

TrindiKit

GoDiS

GoDiS-I GoDiS-A

TravelAgency

VCRmanager

IBDM/KOS ISU

approachDelfos

basic system

Delfosframework

DELFOS-NCL

Homemanager

ISUapproach

TeleDELFOS Home

manager

Research Areas

• Dialogue types• Dialogue phenomena• Enhancing Contextual Appropriateness of

System Output• Enhancing speech recognition through IS • Robust interpretation

Dialogue Types

• Natural command dialogues– user initiated commands e.g.

• transfer my calls to Mr. Jones• turn off all the lights upstairs and lock the front door

• Negotiative dialogues– Propose, negotiate & agree

• parameters for a task e.g. arrival-time, destination-time• alternative solutions to a problem e.g. different flight options

• Tutorial dialogues– guide student through task– non collaborative– hinting (to encourage active learning)

Dialogue phenomena• Conditional responses

– Enhancing collaboration by:• Indicating source of failure:

– U: Can I fly on the second?– S: Not if you want to fly economy class.

• Indicating contingency of success:– S: Yes, if you fly with SAS.

• Clarifications– Lack of specificity

• S: Which account do you want to transfer to?• U: My bank account• S: Do you mean your deposit account or current account?

– Unspecified task e.g.• U: Channel five• S: Do you want to add a program or change channel?

Enhancing Contextual Appropriateness of System Output

• In order to make dialogue system utterances sound natural and contextually appropriate, it is important– to control intonation of spoken output

– to control other aspects of realization, such as generation of short utterances

• We handle these aspects using Information Structure– what speaker means to address vs. what she wants to say about it

– what is the same vs. what discriminates between similar bits of information in context

• Information Structure is determined from the IS

Enhancing speech recognition using IS

• In flexible dialogue, may have dynamically generated prompts• Can’t rely on pre-defined grammars to recognise the response

for each prompt• Potential sources of information:

– previous move/dialogue history (choice from n-best)– syntactic/semantic coherence of fragments (direct parsing of word lattices)– reference resolution– state of the world

• Combine/contrast sources of evidence to decide– most likely utterance in word lattice / n-best– when to clarify

Robust Interpretation• Phrase spotters go to a particular depth of analysis even if a

full parse is possible

• When syntactic information is available should use it

• When not, should do at least as well as keyword/phrase spotting

• Two novel techniques:– distribute the semantic representation using a semantic chart

– ontological knowledge to propose combinations via semantics in competition with syntax: semantic-based composition

turn on the light and the kitchen

• Reconciling traditional syntax and compositional semantics with robust approaches

Main Demonstrators

• TrindiKit 3• Automated Telephone Operator

Trindikit 3

• Toolkit for dialogue researchers

• Developed from TrindiKit 1 built in EU Trindi Project

• IS update rules allow you to write your own dialogue manager– IS +move(s) IS

• Update rules suited for domain and language independent dialogue management

• Global information state – all modules can access all information

– composed from library of datatypes

Automated telephone operator

• Natural language telephone-based access to company telephone directory/PABX

• Functionality– Dial by name – Multi-party conference– Call transfers– Database search (e-mail address, office, phone number)

• KQML based message passing• Delfos dialogue management• User trials at Telefónica I+D

Telephone Operator GUI Corporate Directory / PABX

Further Demonstrations

• Specific modules– Intonation

– Conditional responses

– Robust interpretation

• Applications to illustrate reconfigurability– Travel agency

– VCR

– Command and control of multiple devices (D’Homme project)

Demonstrators: VCRS> “What can I do for you? "U> five thirtyS> "five thirty. I dont quite understand. Do you want to add a program or set the clock?“ U> add a program todayS> "Okay. add a program…. Lets see. today… Do you want from five thirty or until five thirty?“U> from five thirtyS> "Okay. from five thirty… What channel do you want? "U> set the clock S> "Okay. Do you want to set the clock?“U> yesS> "Lets see. What time? "U> four fifteenS> "Okay. four fifteen… The clock was set…Returning to add a program. What channel do you want? “U> channel fiveS> "Okay. channel five."U> pardon S> "Okay. channel five."U> yesS> "What time do you want to stop recording? "U> turn on the t v S> "turn on the TV… Sorry, I cannot control the TV set. What time do you want to stop recording? “U> seven thirtyS> "Okay. until seven thirty…The program has been added. "

+ FeedbackTask

clarification

Load plan

Issue clarification

Requesting other action

? Feedback Reraising

- Feedback (by User)

Rejection

Demonstrators: Home device control

Siridus Summary

• IS approach has provided a stimulus for theoretical work– use of IS to improve

• intonation of spoken output• recognition

– exploration of different dialogue genres and phenomena

• systems which are– modular – reconfigurable– between FS/form filling and BDI/planning

• an automated telephone operator demonstrator• research tool for dialogue developers (TrindiKit 3)