GoDiS

AI-course, ChalmersApril 22, 2002

Staffan Larsson

this lecture

• introduction to GoDiS– information state– moves, plans, actions– accommodation– optimistic grounding and acceptance

• information-oriented vs. action oriented dialogue

• command dialogue & menus• instructional dialogue & text

• originally: information-seeking dialogue– extended to action-oriented dialogue

• information state based Ginzburg’s notion of Questions Under Discussion (QUD)

• dialogue plans to drive dialogue• simpler than general reasoning and

planning• more versatile than frame-filling and

finite automata

GoDiS features

GoDiS & TrindiKit

TrindiKit

QUD-based dialoguetheory (IS, rules, ...)

domain & languageresources

generic GoDiS system

domain-specific GoDiS system

information state approach

inputinter-pret

TIS•IS proper•5 module interface variables•3 resource interface variablesDATABASE LEXICON DOMAIN

data-base

control

update selectgene-rate

output

lexicon domainknowledge

DME

basic GoDIS infostate type

PRIVATE :PLAN : stackset( Action ) AGENDA : stack( Action )

SHARED :

BEL : set( Prop ) TMP : (same type as SHARED)

COM : set( Prop ) QUD : stack( Question )

LU: SPEAKER: Speaker MOVES: assocSet( Move )

+ module interface variablesINPUT : StringLATEST-MOVES: Set(Move)LATEST-SPEAKER: Speaker

NEXT-MOVES: Set(Move)OUTPUT: String

+ resource interface variablesLEXICON : LexiconDOMAIN : DomainDATABASE : Database

PRIVATE = PLAN =

AGENDA = { findout(?return) }

SHARED =

findout(?x.month(x))findout(?x.class(x))

respond(?x.price(x))

COM = dest-city(paris)

transport(plane) task(get_price_info)

QUD = < x.depart-city(x) >LU = SPEAKER= sys

MOVES = {?x.depart-city(x)) - false }

BEL = { }

TMP = (same structure as SHARED)

Sample GoDiS information state

Activity-based moves in GoDiS

• Moves are determined by the relation of the content to the domain– utterance U is an answer if the content A of U is

a relevant answer to a question Q in the domain– moves are not necessarily speech acts!

• GoDiS basic dialogue moves– ask(Q), Q:question– answer(A), A:answer (proposition or fragment)

– inform(P), P:proposition– request-repetition– greet, quit

• integrateAnswer

• Before an answer can be integrated by the system, it must be matched to a question on QUD

pre:

eff:

in($SHARED.LM, answer(usr, A))fst($SHARED.QUD, Q)

$DOMAIN:relevant_answer(Q, A)

pop(SHARED.QUD) DOMAIN: reduce(Q, A, P)

add(SHARED.COM, P)

Sample update rule

Dialogue plans

• domain-specific• partial specifications of how the system

should act in a dialogue– specifies default behaviour– may be overridden by user initiative

• operational semantics of plan constructs – determined by information state update

rules

Dialogue plan constructs

• action A• exec(S)

– S is a task/subplan

• sequence <C1,C2,..., Cn>

• if P then C– P is a proposition and C a construct– If P is (believed to be) true, execute C

• case(<P1, C1>, ...,<Pn-1, Cn-1>, Cn)

Actions

• findout(Q): find the answer to Q• raise(Q): raise Q• respond(Q): provide answer to Q

if there is one in the private beliefs• inform(P): inform of P• (Q is a question, P a proposition)

Questions

• y/n-question: ?P– ”Do you want a return ticket?”– ”Do you want to call NN?”

• wh-question: ?x.P(x)– “What city do you want to go to?”– ”Who do you want to search for?”

• alt-question: {?P1, ?P2, ..., ?Pn}– ”Do you want to search, add to, erase from

or change the phonebook?”

Typical human-computer dialog

S: Hello, how can I help you?U: I want price information pleaseS: Where do you want to go?U: ParisS: How do you want to travel?U: A flight pleaseS: When do you want to travelU: AprilS: what class did you have in mind?…S: The price is $123

Dialogue plans for information-seeking dialogue

• Find out how user wants to travel• Find out where user wants to go to• Find out where user wants to travel

from• Find out when user wants to travel• …• Lookup database• Tell user the price

Dialogue plan, formal representation

• findout(?x.transport(x))• findout(?x.dest-city(x))• findout(?x.depart-city(x))• findout(?return)• …• consultDB(?x.price(x))• respond(?x.price(x))

Typical human-human dialogue

S(alesman), C(ustomer)S: hiC: flights to parisS: when do you want to travel?C: april, as cheap as possible...

Accommodation

• Lewis (1979): – If someone says something at t which requires X to

be in the conversational scoreboard, and X is not in the scoreboard at t, then (under certain conditions) X will become part of the scoreboard at t

• A: My cat is hungry– B accommodates “A has a cat”

• Has been applied to referents and propositions, as parts of the conversational scoreboard / information state

Question accommodation

• If questions are part of the information state, they too can be accommodated

• If – the latest move was an answer, and – there is an action in the plan to ask a matching

question, – then, push that question on QUD

• Requires that the number of possible matching questions is not too large – or can be narrowed down by asking clarification

question

Update rule for question accommodation

QuAcc

pre:

eff:

in($SHARED.LM, answer(usr, A)) in($PRIVATE.PLAN, findout(Q))

$DOMAIN:relevant_answer(Q, A)

delete(PRIVATE.PLAN, findout(Q))push(SHARED.QUD, Q)

question accommodation…

PRIVATE :PLAN : stackset( Action ) AGENDA : stack( Action )

SHARED :

BEL : set( Prop ) TMP : (same type as SHARED)

COM : set( Prop ) QUD : stack( Question )

LU: SPEAKER: Speaker MOVES: assocSet( Move )

Task accommodation

• In some cases, the system may not even know what task the user wants the system to perform

• If latest move was an answer, and there is currently no plan, – find (in the domain resource) a task and

corresponding plan containing a matching question; – accommodate the task and load the appropriate plan

• Similar to plan recognition, but no dynamic plan reconstruction or assumption that the full plan is shared

Update rule for task accommodation

taskAcc

- An answer move matches a task if the plan contains a question matching the answer

- More complex version generates clarification question if the number of plans > 1

pre:

eff:

in($SHARED.LM, Move) $DOMAIN:: match_task(Move, Task, Plan)

add(SHARED.BEL, task(Task)) set(PRIVATE.PLAN, Plan)

task accommodation…

DATABASE= LEXICON = DOMAIN= travel travel_english travel

databasetravel

lexicontravel

english

domaintravel

PRIVATE :PLAN : stackset( Action ) AGENDA : stack( Action )

SHARED :

BEL : set( Prop ) TMP : (same type as SHARED)

COM : set( Prop ) QUD : stack( Question )

LU: SPEAKER: Speaker MOVES: assocSet( Move )

Question and task accommodation in information-seeking dialogue

S: hiU: flights to paris• system finds plan containing appropriate

questions, and loads it into the plan field in the information state

• system accommodates questions: how does user want to travel + where does user want to go, and integrates the answers “flight” and “to paris”

• system proceeds to next question on planS: when do you want to travel?

An optimistic approach to grounding

• DPs assume their utterances are understood– If A asks a question with content Q, A will update

SHARED.LM with ask(A,Q)

• If addressee indicates that the previous utterance was not understood (reqRep), backtrack– uses the PRIVATE.TMP field, which contains a copy of

the SHARED field from before the latest utterance was made

• The alternative is a pessimistic approach– If A asks Q, A will not update SHARED.LM until B

indicates understanding (implicitly or explicitly)

Update rule for optimistic grounding

• assumeSysMovesGrounded

pre:

eff:

$LATEST-SPEAKER == sys

SHARED.LU.SPEAKER := sys

clear( SHARED.LU.MOVES )

forall_do( in( LATEST-MOVES, Move),

add( /SHARED/LU/MOVES, Move, false) )

input

inter-pret

update selectgene-rate

output

PRIVATE :PLAN : stackset( Action ) AGENDA : stack( Action )

SHARED :

BEL : set( Prop ) TMP : (same type as SHARED)

COM : set( Prop ) QUD : stack( Question )

LU: SPEAKER: Speaker MOVES: assocSet( Move )

INPUT : StringLATEST-MOVES: Set(Move)

LATEST-SPEAKER: SpeakerNEXT-MOVES: Set(Move)OUTPUT: String

optimistic grounding

Optimistic approach to acceptance

• DPs assume their utterances are accepted (and integrated into SHARED)– If ask(A,Q) is in SHARED.LM, A will make Q topmost

on SHARED.QUD

• If addresse indicates rejection, backtrack– using the PRIVATE.TMP field

• No need to indicate acceptance explicitly; it is assumed

• The alternative is a pessimistic approach– If A asks a question with content Q, A will wait for an

acceptance (implicit or explicit) before putting Q on top of QUD

optimistic acceptance

PRIVATE :PLAN : stackset( Action ) AGENDA : stack( Action )

SHARED :

BEL : set( Prop ) TMP : (same type as SHARED)

COM : set( Prop ) QUD : stack( Question )

LU: SPEAKER: Speaker MOVES: assocSet( Move )

• Adapted for travel agency and autoroute domains, as well as acting as interface to handheld computer or mobile phone

• Question and task accommodation to enable mixed initiative

• Simple “optimistic” grounding strategy• Focus intonation based on information state

contents• Has been extended to handle instructional

dialogue (IMDiS)• Also being extended to handle negotiative

dialogue (SIRIDUS)

GoDiS features

Action- and information-oriented dialogue

Information- vs. action-oriented dialogue

• Information oriented dialogue (IOD) has the primary goal of exchanging information, regardless of how this information will be used in future actions

• Action oriented dialogue (AOD) has the primary goal of one or several participants performing or being obliged to perform an action (or plan, i.e. a complex action)

Information-oriented dialogue

• Information-seeking dialogue (ISD): one DP asks the questions, the other answers them

• Information-exchange (information oriented) dialogue: both DPs ask questions and provide answers

• IOD can be seen as a sequence of infoseeking dialogues, possibly with embedded subdialogues

• shared information is stored in shared.com

Action-oriented dialogue

• In simple AOD, only one participant becomes obliged/comitted to some action or plan

• Actions can either be performed ”online” while the dialogue is happening, or they may be stored as a plan to be performed after the dialogue (”offline”)

• Requires a store of obliged actions in the IS (SHARED.ACTIONS)

• Two subtypes;– instructional dialogue– command dialogue

GoDIS AOD-extended infostate type

PRIVATE :PLAN : stackset( Action )

AGENDA : stack( Action )

SHARED :

BEL : set( Prop )

TMP : (same type as SHARED)

COM : set( Prop )

QUD : stack( Question )

ACTIONS : stack( Action )

LU:…

Menu-based natural command dialogue (D’Homme)

• system carries out actions• conversion menu 2 dialogue

– menu-based interfaces can be automatically or semi-automatically converted into dialogue plans

• flexible dialogue– dialogue plans are used in a way that

enables more flexible interaction than the original menu-based interface

• Larsson, Cooper, Ericsson (2001): menu2dialog. IJCAI workshop paper.

Menus vs. dialogue:Drawbacks of menus

• User must follow predesigned menu structure

• this structure may be complicated and difficult to learn

• menus allow only one activity at a time

Menu vs. dialogue:Advantages of flexible dialogue

• provide information in any order• provide information without pre-

specifying the task • not necessary to learn menu

structure• doing several things at once

Menu components

• multiple-choice list– the user selects one of several prespecified

choices

• dialogue window– user types requested information

• tickbox– user ticks box (or not)

• system message– information from system; user clicks ”OK”

Sample menu from a Nokia 3210 mobile phone

• search phonebook– dialogue window to type in name

• Name: _______

– message showing the number• Kalle 031 12 34 56

– tickbox: whether to call this person• Call?

– if a call is made, a message:• Calling 031 12 34 56

Converting menus to plans

Menu Planmulti-choice list L1, L2, ..., Ln

findout({?L1,?L2,...,?Ln});

case(<L1,T1>,...,<Ln,Tn>)where T1is the task corresponding to L1

tick-box +/-P findout(?P)

dialogue windowp=___

findout(?x.p(x))

pop-up message M

inform(M)

Converting menus to plans, cont’d

• in addition, we need– if-then statements to check conditions and

pick out propositions– device actions, e.g.

• “call up number N”• “add name N with number M to phonebook”• …

• these are not currently assumed to be part of the menu specification

the menu again

• search phonebook– dialogue window to type in name

• Name: _______

– message showing the number• Kalle 031 12 34 56

– tickbox: whether to call this person• Call?

– if a call is made, a message:• Calling 031 12 34 56

Corresponding plan

findout(?x.name(x));lookup_database(?x.number(x));if number(N) then

respond(?x.number(x) ),findout( ?call ),

if call and name(M) then inform(calling_name(M))))

Another plan: change ringtone

findout(?x.name(x));findout(?x.ringtone(x));if name(N) and ringtone(R) then

save-ringtone( N, R ),inform(ringtone-changed);

Example 1: standard menu navigation

U:Go to the phonebookS: Phonebook. Do you want to search, add, or

remove an entry, or assign a ringing tone? U: Assign ringing toneS: What name?U: JohnS: Which ringtone?U: BossanovaS: The ringtone for John is now Bossanova.

Example 2: question accommodation

U: Assign ringtoneS: What name?U: Bossanova for John

accommodate ”which ringtone?”

S: The ringing tone for John is now Bossanova.

Example 3: task accommodation

U: Bossanova for Johnaccommodate task: assign ringtoneaccommodate ”which ringtone?”accommodate ”what name?”

S: The ringing tone for John is now Bossanova.

Example 4: task clarification

U: 5 o’clock– matches two tasks; ask for

clarification

S: Do you want to set the clock or the alarm clock?

U: alarm clock– accommodate ”what time?”

S: the alarm clock is now set to 5

Example 5 (video): task and question accommodation

U: record from 19:30 to 20:00 today– accommodate task: add program– accommodate ”what starting time?”– accommodate ”what end time?”– accommodate ”what day?”

S: What channel?U: SevenS: OK, I will record channel 7 from 19:30

to 20:00 today.

Example 6: doing several things at once

U: Search the phonebook for JimS: The number is 031 997321U: Assign ringtone “Attraction”

• name(jim) is already a shared proposition

S: The ringtone for Jim is now Attraction. Do you want to make a call?

U: No

• Note: proposition name(jim) is used by both plans

Instructional dialogue

• user carries out actions• goals of this research:

– explore the relation between monologue and dialogue in instructional discourse

– investigate possibilities of using domain task plans to generate dialogue plans providing various levels of interactivity

– adapt GoDiS to handle instructional dialogue (and monologue)

• NB. We are not dealing with educational (a.k.a. instructional) dialogue

• TrindiBook chapter 9; Larsson & Zaenen (2000) SigDial paper

Example from a Xerox manual

• Reinstalling the print head• Caution: make sure that the green carriage lock lever is

STILL moved all the way forward before you reinstall the print head

• 1. Line up the hole in the print head with the green post on the printer carriage

• Lower the print head down gently into position• 2. Gently push the green carriage lock lever up until it

snaps into place• This secures the print head• 3. Close the tops cover and re-attach the scanner• 4. Press and release the yellow LED button• The printer will prepare the cartridge for printing• Note: if the carriage does not move from the center

position after you press the carriage change button, remove and reinstall the print head

dialogue mode• 8 moves (6 standard GoDiS moves, + Instruct

and Confirm)• Confirmations are integrated by assuming

that the current top-most action of the shared.actions has been performed

• rule integrateUsrConfirm (slightly simplified)

pre:

eff:

val(SHARED.ACTIONS, A)

pop(SHARED.ACTIONS)

add(SHARED.COM, done(A))

in(SHARED.LU.MOVES, confirm)

Sample information state

PRIVATE = PLAN =

AGENDA = { findout(?moved(carriage)) }

SHARED =

if_then( not moved( carriage ), [ remove(print_head),

reinstall(print_head) ] )

COM = moved_forward(carriage_lock)

done(secure(print_head)) done(close(top_cover)) done(reattach(scanner))

QUD = < > LU.MOVES { instruct(sys, press_and_release(yellow_button) }

BEL = { }

TMP = (same structure as SHARED)

ACTIONS= < press_and_release(yellow_button) >

Example from a Xerox manual (repeated)

• Reinstalling the print head• Caution: make sure that the green carriage lock lever is

STILL moved all the way forward before you reinstall the print head

• 1. Line up the hole in the print head with the green post on the printer carriage

• Lower the print head down gently into position• 2. Gently push the green carriage lock lever up until it

snaps into place• This secures the print head• 3. Close the tops cover and re-attach the scanner• 4. Press and release the yellow LED button• The printer will prepare the cartridge for printing• Note: if the carriage does not move from the center

position after you press the carriage change button, remove and reinstall the print head

dialogue behavior with minimal interactivity

• yes/no questions• confirm (”ok”)• requestRepeat (”what?”)• grounding and avoidance of irrelevant

information• example:

• S: Press and release the yellow button• U: ok• S: Has the carriage moved from the center position?• U: yes• S: the print head is now installed• U: what?• S: the print head is now installed

increased interactivity

• skipping subtask instructions– S: Put the print head in place– U: Ok, done, what now?– S: Close the top cover

• requesting subtask instructions; popping out of subdialogue– S: Put the print head in place– U: how?– S: Line up the hole in the print head with the

green post on the printer carriage– U: done, I now remember the rest, the print

head is secured

conclusions• GoDiS is based on QUD and dialogue plans

– QUD has independent motivation in dialogue semantics & ellipsis resolution (Ginzburg)

– no planning needed in many domains; use ready-made plans but allow user initiative

• QUD accommodation– theoretically motivated– enables mixed initiative

• Demonstrates reusability– Originally implemented for information-oriented dialogue– but straightforward to extend to handle simple action-

oriented dialogue

• Demonstrate exploratory use of TrindiKit – explore relation between instructional text and dialogue