Dialogsystem del 2Informationstillstånd, TrindiKit, GoDiS

Staffan Larsson

Pragmatik VT04

Dialogspel för agenter:Conversational Game Theory

(Lewin 2000)

Informationstillstånd och dialogtillstånd

• Dialogtillstånd – ett tillstånd i en finit automat; ingen information lagrad

i tillståndet

• Informationstillstånd– ett ``dialogprotokoll'' som håller reda på

gemensamma antaganden, aktuella frågor, skyldigheter, referenter mm.

– kan även inkludera privata och sociala attityder– både privat och delad information– kan t o m inkludera dialogtillstånd (t ex ett heltal som

refererar till ett tillstånd i en automat)

CGT & dialogspelsbaserade agenter

• Teori som tillämpar dialoggrammatik i dialogsystem

• Använder också informationstillstånd• Spel representerade som RTNs (Recursive

Transition Networks)– d v s bågar i ett spel kan vara associerade med ett

annat spel• Kombineras med enkelt

informationstillstånd/kontext <Pd, Cm>:– Pd: Propositions under discussion

• < P, d(P) >, där d(P) är ett fokuserat element i P– Cm: Commitment slate

• Moves & games är funktioner som uppdaterar kontexten

• Moves uppdaterar Pd• Games uppdaterar Cm

– committments: ej mentala attityder utan ”publika objekt” som man kan bindas till

– social attityd

• Ej som i t ex Cohen & Perrault!– förvillkor och effekter i termer av mentala tillstånd,

privata attityder

Move types (urval)

• qw(p): wh-fråga– Pd := < p, 0 >

• rw(p): svar på wh-fråga– Pd := < p, 0 >

• ack: acknowledgement; Pd oförändrad• cnf(c): confirmation

– Pd före = < P, _ >– Pd := < P, c >

• Ryes: ja-svar ; Pd oförändrad• Rno: nej-svar ; Pd oförändrad

Games

• Att spela dialog involverar ”parsning” av spel m h a dialoggrammatiken– parallell, inkrementell parser– rankar möjliga parsningar m h a en preferensmekanism– detta sköts av en ”monitor”

• Men agenten måste också producera egna yttranden– sköts av dialogbidragsgenerator– genererar output om monitorn indikerar att det är systemets tur– vilken output som väljs beror dels av speltillstånd, dels av

informationstillstånd

0

4

1 2 3

qw

qw-r

rw

cnf

Ryes|Rno|Rmod

ack

QW(p)

QW(p) -> {qw | qw-r} rw (cnf {Ryes | Rno | Rmod}) ack...

Exempeldialog

game state

infostate

A: Where do you want to travel? 1 Pd = <{dest(X)}, 0>

Cm = {}

B: To Exeter 2 Pd = <{dest(exeter)}, 0>

A: Is your destination Exeter? 4 Pd = <{dest(exeter)}, dest(exeter)>

B: Yes 2

A: Okay 3 Pd =

Cm = {dest(exeter)}

Uppgifter (2-4 sidor)

• 10.1 Vilka egenskaper har en agent? Vilka attityder kan en agent ha, och hur hänger dessa samman med egenskaperna? (Wooldridge & Jennings)

• 10.2 Beskriv de olika approacherna till att bygga artificiella dialogagenter (Traum, Lewin)– planbaserad– logikbaserad– tillståndsbaserad

• 10.3 Vilken typ av information behöver en dialogagent hålla reda på i de olika approacherna? (Traum, Lewin)

Dialogsystem

Dialogue modelling

• Theoretical motivations– find structure of dialogue– explain structure– relate dialogue structure to informational and

intentional structure

• Practical motivations– build dialogue systems to enable natural human-

computer interaction– speech-to-speech translation– ...

Why build dialogue systems?

• theoretical: test theories– e.g. what kind of information does the system need to

keep track of?– problem: complex system with many components

• practical: natural language interfaces– databases (train timetables etc)– electronic devices (mobile phones,...)– instructional/helpdesk systems– booking flights etc– tutorial systems

What does a system need to be able to do?

• speech recognition• parsing, syntactic and semantic interpretation

– resolve ambiguities– anaphora and ellipsis resolution, etc...

• dialogue management– how does an utterance change the state of the dialogue?– given the current state of the dialogue, what should the

system do?

• natural language generation• speech synthesis

Why spoken dialogue?

• Spoken dialogue is the natural way for people to communicate– computers should adapt to humans rather than the

other way around

• important to enable system and user to communicate in a natural (human-like) way– mixed initiative– turntaking, feedback, barge-in – handle embedded subdialogues– ...

What’s happening with dialogue systems

• Beginning to be used commercially• Limited domains

– need to encode domain-specific knowledge; a general system would require general world knowledge

– speech recognition is harder with large lexicon

• Simple dialogue types– mostly information-seeking

• Need to bridge gap between dialogue theory and working systems

Dialogsystemtyper

• finite state automata (CLSU toolkit)– Dock ofta med möjlighet att ha tilltåndsvariabler

(=informationstillstånd)– Reaktiva agenter???

• frame-based (VoiceXML)

• plan-based (TRAINS, Allen, Cohen, Grosz, Sidner, ...)

• general reasoning (Sadek, ...)

• information states (TRINDI: Traum, Bos, ...)

Problemområden & teorierformell pragmatik för dialogsystem

• hantering av dialogstruktur– dialogspel– talakter

• implicit information– presupposition– implikatur– planigenkänning

• relatera explicit & implicit information till kontext; uppdatera kontext– pronomenlösning (DRT, Centering Theory, abduktion)– planigenkänning– accommodation

• välj/planera yttrande– planering– implicit information?

• kommunikationshantering– ICM, OCM– grounding– konversationsanalys

TrindiKit

What is TrindiKit?

• a toolkit for – building and experimenting with dialogue move engines and systems, – based on the information state approach

• not a dialogue system in itself

Architecture & concepts

module1 module…

Total Information State (TIS)•Information state proper (IS)•Module Interface Variables•Resource Interface Variables

resource1

control

modulei modulej module… modulen

resource… resourcem

DME

• an abstract data structure (record, DRS, set, stack etc.)

• accessed by modules using conditions and operations

• the Total Information State (TIS) includes– Information State proper (IS)– Module Interface variables– Resource Interface variables

Information State (IS)

• module or group of modules responsible for – updating the IS based on observed moves– selecting moves to be performed

• dialogue moves are associated with IS updates using IS update rules– there are also update rules no directly associated with any move

(e.g. for reasoning and planning)

• update rules: rules for updating the TIS– rule name and class– preconditon list: conditions on TIS– effect list: operations on TIS

• update rules are coordinated by update algorithms

Dialogue Move Engine (DME)

• Modules (dialogue move engine, input, interpretation, generation, output etc.) – access the information state– no direct communication between modules

• only via module interface variables in TIS• modules don’t have to know anything about other modules• increases modularity, reusability, reconfigurability

– may interact with user or external processes

• Resources (device interface, lexicons, domain knowledge etc.)– hooked up to the information state (TIS) – accessed by modules– defined as object of some type (e.g. ”lexicon”)

Modules and resources

What’s in TrindiKit?

What does TrindiKit provide?

• High-level formalism and interpreter for implementing dialogue systems– promotes transparency, reusability, plug-and-play, etc.– allows implementation and comparison of dialogue

theories – hides low-level software engineering issues

• GUI, WWW-demo • Ready-made modules and resources

– speech– interfaces to databases, devices, etc.– reasoning, planning

• a library of datatype definitions (records, DRSs, sets, stacks etc.)– user extendible

• a language for writing information state update rules • GUI: methods and tools for visualising the information

state• debugging facilities

– typechecking– logs of communication modules-TIS– etc.

TrindiKit contents (1)

• A language for defining update algorithms used by TrindiKit modules to coordinate update rule application

• A language for defining basic control structure, to coordinate modules

• A library of basic ready-made modules for input/output, interpretation, generation etc.;

• A library of ready-made resources and resource interfaces, e.g. to hook up databases, domain knowledge, devices etc.

TrindiKit contents (2)

Special modules and resources included with TrindiKit

• OAA interface resource– enables interaction with existing software and

languages other than Prolog

• Speech recognition and synthesis modules– TrindiKit shells for off-the-shelf products, e.g. Nuance

• Possible future modules:– planning and reasoning modules– multimodal input and output

Asynchronous TrindiKit

• Internal communication uses either– OAA (Open Agent Architecture) from SRI, or– AE (Agent Environment), a stripped-down version of

OAA, implemented for TrindiKit

• enables asynchronous dialogue management– e.g.: system can listen and interpret, plan the

dialogue, and talk at the same time

How to build a system

TrindiKitinformation state approach

How to use TrindiKit

• We start from TrindiKit– Implements the information state approach– Takes care of low-level programming:

dataflow, datastructures etc.

TrindiKit

basicdialogue theory

basic system

information state approach

How to build a basic system

• Formulate a basic dialogue theory – Information state– Dialogue moves– Update rules

• Add appropriate modules (speech recognition etc)

TrindiKit

basicdialogue theory

basic system

information state approach

genre-specific theoryadditions

genre-specific system

How to build a genre-specific system

• Add genre-dependent IS components, moves and rules

TrindiKit

basicdialogue theory

domain & languageresources

basic system

application

information state approach

genre-specific theoryadditions

genre-specific system

How to build an application

• Add application-specific resources

• Come up with a nice theory of dialogue• Formalise the theory, i.e. decide on

– Type of information state (DRS, record, set of propositions, frame, ...)

– A set of dialogue moves– Information state update rules, including rules for

integrating and selecting moves– DME Module algorithm(s) and basic control algorithm – any extra datatypes (e.g. for semantics: proposition,

question, etc.)

Building a domain-independent Dialogue Move Engine

Specifying Infostate type

• the Total Information State contains a number of Information State Variables– IS, the Information State ”proper”– Interface Variables

• used for communication between modules

– Resource Variables• used for hooking up resources to the TIS, thus making them

accessible from to modules

• use prespecified or new datatypes

example: BeadieEye IS

BEL: Set(Prop)DES: Set(Prop)INT: Set(Action)MBEL: Set(Prop)

information statetype

LM: Set(Move)

IS :

Specifying a set of moves

• amounts to specifying objects of type move (a reserved type)– there may be type constraints on the arguments of

moves• Example: GoDiS dialogue moves

– Ask(Q), Q is a question– Answer(A), A is an answer (proposition or fragment)– Request(), is an action– Confirm()– Greet– Quit

Writing rules

• rule = conditions + updates– if the rule is applied to the IS and its

conditions are true, the operations will be applied to the IS

– conditions may bind variables with scope over the rule (prolog variables, with unification and backtracking)

Example: BeadieEye moves and a rule

moves: assert(P), askif(P)

rule( integrate_assert,

[ in( $lm, assert(P) ) ],

add( is/mbel, P ),

add( is/bel, P ),

del( lm, assert(P) ) ] ).

Example BeadieEye rule application

BEL = { happy(sys) }DES = { knowif( happy(usr) ) }INT = { }MBEL = { }

Rule application: integrate_assert, > add( IS/MBEL, happy(usr) ),> add( IS/BEL, happy(usr) ),> del( LM, assert(happy(usr) )

LM = { assert( happy(usr) }

IS =

BEL = { happy(sys), happy(usr) }DES = { knowif( happy(usr) ) }INT = { }MBEL = { happy(usr) }

LM = { }

IS =

Example: a rule from GoDiS

rule( integrateUsrAnswer, [ $/shared/lu/speaker = usr,

assoc( $/shared/lu/moves, answer(R), false ),

fst( $/shared/qud, Q ),

$domain : relevant_answer( Q, R ),

$domain : reduce(Q, R, P)

], [

set_assoc( /shared/lu/moves, answer(R),true),

shared/qud := $$pop( $/shared/qud ),

add( /shared/com, P ) ] ).

Building modules

• Algorithm– For DME modules: coordinate update rules– For control modules: coordinate other modules

• TrindiKit includes a language for writing algorithms– For DME modules: basic imperative programming

constructs– For control module: basic imperative constructs plus

asynchronous triggers

Sample update algorithm

grounding, if $latest_speaker == sys then

try integrate, try database, repeat downdate_agenda, store

else repeat

integrate orelse accommodate orelse find_plan orelse

if (empty ( $/private/agenda ) then manage_plan else downdate_agenda

repeat downdate_agendaif empty($/private/agenda))then repeat manage_plan

repeat refill_agendarepeat store_nim try downdate_qud

Sample control algorithm (2)input: {

init => input:display_prompt, new_data(user_input) => input }

| interpretation: { import interpret, condition(is_set(input)) => [ interpret, print_state ] }

| dme: { import update, import select, init => [ select ], condition(not empty(latest_moves)) => [

update, if $latest_speaker == usr then select

] }

| generation: { condition(is_set(next_moves)) => generate }

| output: { condition(is_set(output)) => output } )).

From DME to dialogue system

Build or select from existing components: • Modules, e.g.

– input– interpretation– generation– output

• Still domain independent• the choice of modules determines e.g. the

format of the grammar and lexicon

Domain-specific systemBuild or select from existing components:• Resources, e.g.

– domain (device/database) interface– dialog-related domain knowledge, e.g. plan libraries etc.– grammars, lexicons

• Example resources: GoDiS VCR control – VCR interface– Domain knowledge– Lexicon

Extending TrindiKit

You can add

• Datatypes– Whatever you need

• Modules– e.g. General interfaces to speech recognizers and

synthesizers

• Resources– E.g. General interfaces to (passive) devices

• Important that all things added are reasonably general, so they can be reused in other systsems

Datatype definitions

• relations– relations between objects; true or false

• functions– functions from arguments to result

• selectors – selects an object embedded in another object

• Operations– Changes the information state

Building modules

• DME modules– Specific to a certain theory of dialogue management– Best implemented using rules and algorithms

• Other modules– Should be more general, less specific to certain theory

of dialogue management– May be easier to implement directly in prolog or other

language• TrindiKit algorithm language currently only covers checking

and updating the infostate• These modules may also need to interact with other programs

or devices

Building resources

• Resource– the resource itself; exports a set of predicates

• Resource interface– defines the resource as a datatype T, i.e. in terms of relations,

functions and operations

• Resource interface variable– a TIS variable whose value is an object of the type T

• By changing the value of the variable, resources can be switched dynamically– change laguage– change domain

Features and advantages of TrindiKit

• explicit information state datastructure – makes systems more transparent – enable e.g. context sensitive interpretation,

distributed decision making, asynchronous interaction

• update rules – provide an intuitive way of formalising theories in a

way which can be used by a system– represent domain-independent dialogue

management strategies

TrindiKit features

TrindiKit features cont’d

• resources– represent domain-specific knowledge– can be switched dynamically

• e.g. switching language on-line in GoDiS

• modular architecture promotes reuse– basic system -> genre-specific systems– genre-specific system -> applications

Theoretical advantages of TrindiKit

• theory-independent– allows implementation and comparison of competing

theories– promotes exploration of middle ground between

simplistic and very complex theories of dialogue

• intuitive formalisation and implementation of dialogue theories– the implementation is close to the theory

Practical advantages of TrindiKit

• promotes reuse and reconfigurability on multiple levels

• general solutions to general phenomena enables rapid prototyping of applications

• allows dealing with more complex dialogue phenomena not handled by current commercial systems

availability• TrindiKit website

– www.ling.gu.se/projects/trindi/trindikit

• SourceForge project– development versions available– developer community?

• licensed under GPL• more info in

– Larsson & Traum: NLE Special Issue on Best Practice in Dialogue Systems Design, 2000

– TrindiKit manual (available from website)

BeadieEye

BeadieEye

• Ett mycket enkelt dialogsystem

• Implementerat i TrindiKit

• Experimentera med – Talakter/dialogdrag– Attityder – Interaktion mellan dessa

• Enbart dialoghantering– ingen tolkning eller generering av NL

BeadieEye

• Informationstillstånd / mentala tillstånd– En uppsättning attityder– Och relaterade strukturer

• Dialogdrag / talakter– Typ + innehåll

• Uppdateringsregler för dialogagenter– Generellt: funktion IS -> IS’– Integreringsregler: IS + Move -> IS’– Selektionsregler: IS -> Move + IS’

• Talakter och uppdateringsregler– förvillkor kodas som selektionsregler– effekter kodas som integreringsregler

• Dialogspel– ej explicit kodade i t ex grammatik; emergent

egenskap

BeadieEye1

• Belief: bel

• Intention: int

• Desire: des

• Mutual belief: mbel

• Propositioner: godtycklig prologterm

example IS and rule

BEL: Set(Prop)DES: Set(Prop)INT: Set(Action)MBEL: Set(Prop)

information statetype

moves: assert(P), askif(P)

rule( integrate_assert,[ in( $lm, assert(P) ) ], add( is/mbel, P ), add( is/bel, P ), del( lm, assert(P) ) ] ).

LM: Set(Move)

IS :

Example rule applicationBEL = { happy(sys) }DES = { knowif( happy(usr) ) }INT = { }MBEL = { }

Rule application: integrate_assert, > add( IS/MBEL, happy(usr) ),> add( IS/BEL, happy(usr) ),> del( LM, assert(happy(usr) )

LM = { assert( happy(usr) }

IS =

BEL = { happy(sys), happy(usr) }DES = { knowif( happy(usr) ) }INT = { }MBEL = { happy(usr) }

LM = { }

IS =

Issue-based Dialogue Management in GoDiS

AI course, spring 2003

Staffan Larsson

overview of contents

1. Introduction

2. Basic issue-based dialogue management

3. Grounding and feedback

4. Adressing Unraised Issues

5. Action-oriented Dialogue

6. Multilinguality

7. Conclusions

1. Introduction, goals

• explore and implement issue-based dialogue management– starting from Ginzburg’s theory of dialogue semantics based on

notion of QUD (Questions Under Discussion)– adapt to dialogue system (GoDiS) and implement– extend theory coverage, taking in relevant theories

• general theory of dialogue– minimize effort for adapting dialogue system to new domains

• incrementally extending system to handle increasingly complex types of dialogue– clarifies relation between dialogue genres– promotes reuse of update rules

• Larsson (2002): Issue-based Dialogue Management (PhD Thesis)

GoDiS: an issue-based dialogue system

• Built using TrindiKit– Toolkit for implementing and experimenting with dialogue

systems based on the information state approach

• Explores and implements issue-based dialogue management

• Extends theory to more flexible dialogue– Multiple tasks, information sharing between tasks – Feedback and grounding– Accommodation, re-raising, clarification– Menu based action oriented dialogue– Multi-linguality & mutiple domains

inputinter-pret

TIS

DATABASE LEXICON DOMAIN

data-base

control

update selectgene-rate

output

lexicon domainknowledge

DME

TrindiKit

GoDiS

GoDiS-I GoDiS-A

TravelAgency

Auto-route

Xeroxmanual

VCRmanager

IBDM

homedevice

manager

ISapproach

genre-specific

application-specific

Issue-based dialogue management

• enquiry-oriented dialogue (database search)• basis:

– Ginzburg’s Dialogue Gameboard (DGB) and – related DGB update protocols

• dialogue moves: ask, answer, greet, quit• raising and addressing issues

– incl. short answers. e.g.”yes”, ”no”, ”paris”, ”in april”

• dialogue plans• sample domain: travel agency• extension:

– reraising issues– handling multiple issues

Semantics• simple First Order Logic without quantifiers, but

with questions• questions

– Y/N-questions: ?P, P is a proposition– wh-questions: ?x.p(x) (p is a predicate)

• ? works much like like – alt-questions: {?P1, …, ?Pn}

• Content of short answers– individual markers: paris, april, …– yes, no

Semantics, cont’d

• Q-A relations (adapted from Ginzburg)– resolves(A,Q): A resolves Q

• dest-city(paris) resolves ?x.dest-city(x)

– relevant(A,Q): A is relevant to Q (about Q)• not(dest-city(paris)) is relevant to ?x.dest-

city(x), but does not resolve it

basic GoDiS information state record type

PRIVATE :PLAN : stack( Action ) AGENDA : OpenQueue( Action )

SHARED :

BEL : set( Prop )

COM : set( Prop ) QUD : stack( Question )

LU: SPEAKER: Speaker MOVES: OQueue( Move )

sample dialogue plan

< findout(?x.transport(x))

findout(?x.dest-city(x))

findout(?x.depart-city(x))

findout(?x.dept-month(x))

findout(?x.dept-day(x))

raise({?class(economy), ?class(business)}

consultDB(?x.price(x)) >

Answer integration

• integrateAnswer update rule

• Before an answer can be integrated by the system, it must be matched to a question on QUD

pre:

eff:

in($/SHARED/LU/MOVES, answer(A)) fst($/SHARED/QUD, Q)

$DOMAIN:relevant(A, Q)

! DOMAIN: combine(Q, A, P) add(/SHARED/COM, P)

basic dialogue with updates

U: ”price information please”; raises price issue– if user asks Q, push respond(Q) on AGENDA– if respond(Q) on AGENDA and PLAN empty, find plan for Q

and load to PLAN– if findout(Q) first on PLAN, ask Q

S: ”where do you want to go?”U: ”Paris”

– if LM=answer(A) and A relevant to Q, add P=Q[A] to SHARED.COM

– if P in SHARED.COM and Q topmost on QUD and P resolves Q, pop QUD

– if P in SHARED.COM and P fulfils goal of findout(Q) and findout(Q) on PLAN, pop PLAN

basics cont’d

• …S: ”Do you want economy class or business

class?”U: ”economy class”

– if consultDB(Q) on PLAN, consult database for answer to Q; store result in PRIVATE.BEL

– if Q on QUD and P in PRIVATE.BEL s.t. P resolves Q, answer(P)

S: ”The price is £123”

Information sharing across plans

• GoDiS does not keep track of when propositions were added, or which plan was being executed

• so information sharing is determined by question sharing across plans

• plan for VISA question:findout(?x.dest-city(x))findout(?x.citizenship(x))– shares a questions with plan for ?x.price(x)

• so if visa-issue raised after price-issue, no need to ask for destination again

dealing with multiple open issues

• if user asks Q, push Q on QUD and load plan for dealing with Q

• if users asks Q’ while system is dealing with Q, throw out plan for Q but Q remains on QUD; load plan for Q’

• when Q’ resolved, Q topmost on QUD will trigger reloading plan for dealing with Q– general rule: if SHARED.COM contains info resolving Q, don’t

ask Q– so any resolved questions in plan will be thrown out

Sample dialogue: multiple tasks & info sharing

S> Welcome to the travel agency! U> price information S> (…) Lets see. How do you want to travel? U> by flightS> (…) What city do you want to go to? U> parisS> (…) What city do you want to go from? U> do I need a visa ?S> (…) Lets see. What country are you from? U> swedenS> Okay. Yes, you need a Visa. S> Returning to the issue of price. Lets see. What city do you want to

go from?

Addressing Unraised Issues

• QUD and ISSUES

• General strategy: question accommodation

• issue accommodation

• QUD accommodation

• dependent issue accommodation

• issue reacommodation

A problem with QUD

• If QUD=<q1,q2> and q1 is resolved, q2 is available for resolution of short answers– takes no account of how many turns since q2 was

raised– but short answers a long distance away from the

question are not as easily processed as an adjacent answer

ISSUES and QUD

• We extend Ginzburg’s DGB by adding ISSUES of type Stack(Question)

• ISSUES contains all raised but unresolved questions – ISSUES determines relevance of user answers

• QUD used for resolving short answers– questions drop off QUD after N turns– a short answer to a question that’s on ISSUES but not

QUD requires adjusting QUD by copying a question on ISSUES

Typical human-human dialogue

S(alesman), C(ustomer)

S: hi

C: flights to paris

S: when do you want to travel?

C: april, as cheap as possible

...

Accommodation

• Lewis (1979): If someone says something at t which requires X to be in the conversational scoreboard, and X is not in the scoreboard at t, then (under certain conditions) X will become part of the scoreboard at t

• Has been applied to referents and propositions, as parts of the conversational scoreboard / information state

Question accommodation

• If questions are part of the information state, they too can be accommodated

• If the latest move was an answer, and there is an action in the plan to ask a matching question, then– put that question on ISSUES – (and QUD if it is a short answer)

• Requires that the number of possible matching questions is not too large – (or can be narrowed down by asking clarification

question)

issue accommodation PLANISSUES

• If – LM=answer(A) – no Q in ISSUES s.t. about(A,Q)

• then– find findout(Q) in PLAN s.t. about(A,Q)– push Q on ISSUES

• used when prevously unraised question (available in plan) is answered using a short or full answer

QUD accommodation ISSUESQUD

• If – LM=answer(A)– no Q in QUD s.t. about(A,Q)

• then– find Q in ISSUES s.t. about(A,Q)– push Q on QUD– raise Q in ISSUES (make Q topmost)

• used when– previously raised question has dropped off QUD, but is

answered using a short answer– previously unraised question is answered using short answer

[needs PLANISSUES accommodation]

dependent issue accommodationDOMAINISSUES (+PLAN)

• If – LM=answer(A)– no Q in ISSUES s.t. about(A,Q)– no findout(Q) in PLAN s.t. about(A,Q)

• then– find Plan for some Q’ in DOMAIN s.t. findout(Q) or

raise(Q) in Plan and about(A, Q)– push Q’ on ISSUES– set PLAN to Plan

• used when previously unraised question, unavailable in PLAN, is answered using full or short answer (AKA ”task accomodation”)

Sample dialogue: accommodation

S: Welcome to the travel agency.U: From London to Paris in April

– not relevant to any question that has been raised, or to any current task– look in domain knowledge for a plan (for dealing with some question Q)

with matching questions– load this plan, push Q on ISSUES– find in the plan the question(s) matching the user’s answer– integrate answer (requres matching question on ISSUES)

S: Alright, you want to know about price. (…)

– proceed to next plan item

S: How do you want to travel?– ISSUES=<?x.how(x), ?x.price(x)>

Task clarification (VCR application)

S> What can I do for you? U> channel fiveS> channel five. I dont quite understand. Do you want to

add a program or change channel?U> change channelS> Okay. change channel.S> Lets see. channel five?U> yesS> The channel was changed

Issue reaccommodation SHARED.COMISSUES

• If – LM=answer(A) – no Q in ISSUES s.t. about(A,Q)– P in SHARED.COM s.t. there is a Q s.t. about(A,Q)

and about(P,Q)

• then– push Q on ISSUES– remove P from SHARED.COM

• Used when previously resolved question is answered again

Revision by reaccommodation and reraising

S> What can I do for you?

U> add program channel five

S> Okay. Do you want to add a program?

U> yes

S> Lets see. channel five?

U> yes

S> What date?

U> channel six

S> Okay. channel six… What date?

Issues and goals in Action-Oriented Dialogue (AOD)

• each goal action associated with a dialogue plan• add field

– SHARED.ACTIONS : StackSet(Action)

• New moves: request(Action), report(Action, Status)

• ACTIONS has a similar role to ISSUES• adapt accommodation strategies to AOD

AOD information state

PRIVATE :PLAN : stackset( Action )

AGENDA : stack( Action )

SHARED :

BEL : set( Prop )

TMP : (same type as SHARED)

COM : set( Prop )

QUD : stackset( Question )

ACTIONS : stackset( Action )

LU:…

ISSUES: stackset(Question)

Requests vs. answers

• requests adresses a general question– ”what shall I do next?”, ”what can I do for you?” or similar

(”prompt”)– semantics in AOD: ?x.action(x)

• dialogue– ”What can I do for you?” raises?x.action(x) – ”Search the phonebook” interpreted as

request(search_phonebook)

• Rule: – If ?x.action(x) is topmost on ISSUES and L-M is request(a)– then pop ISSUES

Questions vs. answers

• a question can be regarded as a special type of request

• so in a specific sense, questions are also answers…

• questions adresses a general question– ”what issue shall I address next?”, ”what can I do for

you?” or similar (”prompt”)– semantics: ?x.issue(x)– this question is not presupposed

• semantics of ”How can I help you?” in IOD is ask(?x.issue(x))

Action Oriented Dialogue subtypes

• ”Command dialogue”– user instructs system to perform actions

• device control; menu-based• travel agency booking

• ”Instructional dialogue”– system instructs user to perform actions

• ”interactive manual”

– system can explaing how to perform some action

• joint planning– user and system jointly agree to a plan– also involves negotiation

inputnuance

inter-pret

Information StateLEXICON DOMAIN DEVICES

lexiconvcrphone

control

update select gene-rate

outputnuance

devicephone

domainvcrphone

devicevcr

lexiconhome

domainhome

rel1

rel2

dim10

VCR

devicealert

Converting menus to plans

Menu Planmulti-choice list

P1, P2, ..., Pn

findout({?P1,?P2,...,?Pn})

tick-box +/-P findout(?P)

dialogue window

p=___

findout(?x.p(x))

pop-up message M inform(M)

Menu-based AODS: Welcome to the telephone and VCR manager!U: Channel one todayS: Right, I assume you want to add a program. Let’s see. Okay. Channel

one. Today. What time do you want to start recording?S: What time is it?U: It’s one forty fiveS: Set the clock to two forty fiveU: Okay, set the clock. Two forty five, is that correct?S: Yes.U: Okay. The clock was set. Returning to timer recording. So, what time

do you want to start recording?U: eight thirty to nine thirtyS: Okay. Starting at eight thirty. Ending at nine thirty. The program has

been added.

Task and question accommodation in AOD

S> Lets see. What can I do for you?

U> channel five today

S> Alright. add a program… Lets see. Okay. channel five… today… What time do you want to start recording?

U> from six thirty to seven thirty

S> Okay. from six thirty… to seven thirty… The program has been added. ”

Signals accommodation(sequencing ICM)

7. Multilinguality and multiple domains

• Language and domain resources can be switched on the fly

• Does not affect the rest of the system

• Thanks to modularity structure of system

Multilinguality

S> Lets see. What can I do for you? U> add programS> Okay. Do you want to add a program?U> yesS> Lets see. What channel do you want? U> svenskaS> Okej. S> Gå tillbaks till lägg till ett program. Vilken kanal vill du spela in? U> kanal ettS> kanal ett , är det korrekt? S> Så, Vilken kanal vill du spela in? U> englishS> Okay...

Multiple domainsS> Welcome to the VCR manager! S> Lets see. What can I do for you? U> change domainS> Okay. change domain… Lets see. What domain do you want? U> telephoneS> Okay. This is your automatic telephone operator… Lets see. Please

specify a function. U> callS> Okay. make a phone call.S> Lets see. Please specify the destination of the call. U> luisS> Okay. Luis… Calling

Conclusions: GoDiS & Issue-Based Dialogue Management

• general solutions to– dealing with multiple tasks– sharing information between tasks– grounding and feedback– user initiative (accommodation)– menu-based dialogue

• rapid prototyping of applications– dialogue plans

• switching language and domain online

Current and future work• Extend to more complex dialogue types

– Negotiation (theory exists, not yet implemented)– Tutorial dialogue– …

• Explore use of QUD and ISSUES to assign proper focus intonation – Stina Ericsson, forthcoming PhD thesis

• Build VCR control application, available on the phone– Users can phone up and get TV programs recorded on a computer– Collect corpus to use as basis for further developments

• Integrate with type-theoretical situation semantics (cf. Robin’s talk)

Relation to plan-based approaches

• IBDM is compatible with, and can (should) be extended to include plan-based approaches

– Task accommodation requires plan recognition; currently very simple (but task accommodation is not included as such in plan-based approaches)

• IBDM covers some dialogue-specific phenomena not usually covered in plan-based approaches

– Short answers– Grounding and feedback– Accommotion

• However, some plan-based accounts cover some of the phenomena also covered by IBDM

– We believe the issue-based account is simpler and thus more appropriate for simple kinds of dialogue

– Plan-based approaches often unnecessarily complex for simple kinds of dialogue– We want to explore the middle ground beteen simple and complex accounts of

dialogue

Uppgift: GoDiS

• Kör GoDiS i VCR-domänen och skriv ”channel five today”. Förklara vad som sedan händer i termer av uppdateringsregler och accommodation.