Generating Feedback and Sequencing Moves in a

Dialogue SystemAAAI Spring Symposium 2003

Staffan LarssonGöteborg University, Sweden

Overview

• Interactive Communication Management (ICM)

• ”Verification” in dialogue systems• Classifying and formalising ICM• ICM for a dialogue system• Examples• Conclusions & Future work

ICM (Allwood)

• Interactive Communication Management – As opposed to Own Communication Management

(OCM): self-corrections, hesitations, etc.

• Feedback moves– (short) utterances which signal grounding status of

previous utterance (”mm”, ”right”, ”ok”, ”pardon?”, ”huh?” etc.)

• Sequencing moves– utterances which signal dialogue structure (”so”,

”now”, ”right”, ”anyway” etc.)– Dialogue structure part of / modeled by common

ground

• Turntaking moves

Grounding and ICM in current commercial systems

• Limited to ”verification”• Examples (San Segundo et. al. 2001)

– I understood you want to depart from Madrid. Is that correct? [”explicit v.”]

– You leave from Madrid. Where are you arriving at? [”implicit v.”]

• Involves repetition or reformulation • Appears in H-H dialogue, but not very

common

From verification to ICM in dialogue systems

• ”Verification” is just one type of ICM behaviour– Perhaps the one most cruicial in dialogue systems

given poor speech recognition

• Could a wider range of the ICM behaviour occurring in H-H dialogue be useful in dialogue systems?

• We want a typology of ICM moves for H-H dialogue– Feedback and sequencing moves

• We want to formalise it and use it in a system– Still we will implement only a subset, but more than

verification

Classifying feedback

• Level of action• Polarity• Eliciting / noneliciting• Form (syntactic realisation)• Content type: object- or metalevel

Feedback levels

• Action levels in dialogue (Allwood, Clark, Ginzburg)– Contact: whether a channel of communication is

established– Perception: whether DPs are perciveving each other’s

utterances– Understanding: Whether DPs are understanding each

other’s utterances• Non-contextual (”semantic”) meaning• Contextual (”pragmatic”) meaning

– Acceptance: Whether DPs are accepting each other’s utterances

• The function of feedback is to signal the status of utterance processing on all levels

Feedback polarity

• Polarity (Allwood et.al. 1992)– Positive: indicates contact, perception, understanding,

acceptance– Negative: indicates lack of contact, perception,

understanding, acceptance– We add a ”neutral” or ”checking” polarity – there is one

or more hypotheses, but the DP lacks confidence in them

• Examples – ”I don’t understand”: negative– ”Do you mean that the destination is Paris?”: checking– ”To Paris.”: positive– ”Pardon”: negative

Formalising ICM dialogue moves

• Action levels– con: contact– per: perception– sem: semantic understanding (no context)– und: pragmatic understanding (relevance in context)– acc: acceptance

• Polarity– pos– neg– chk (”int” in paper)

Feedback move notation

• icm:Level*Polarity{:Args}– icm:per*pos:String – ”I heard you say

’londres’”– icm:und*neg – ”Sorry, I don’t understand”– icm:und*chk:AltQ – ”Do you mean x or y?”– icm:und*pos:P – ”To Paris.”– icm:acc*neg:Q – ”Sorry, I can’t answer Q”– icm:acc*pos – ”Okay”

System feedback for user utterances in GoDIS

• contact– negative (”I didn’t hear anything from you.”, ”Hello?”)

[icm:con*neg]• perception

– negative: fb-phrase (”Pardon?”, ”I didn’t hear what you said”) [icm:per*neg]

– positive: repetition (”I heard ’to paris’”) [icm:per*pos:String]

• semantic understanding:– negative: fb-phrase (”I don’t understand”) [icm:sem*neg]– positive: reformulation (”Paris.”) [icm:sem*pos:Content]– checking: reformulation (”Paris, is that correct?”, ”Paris?”)

[icm:sem*chk:Content]

System feedback, cont’d

• pragmatic understanding– negative: fb-phrase (”I don’t quite understand”)

[icm:und*neg]– positive: reformulation (”To Paris.”)

[icm:und*pos:Content]– checking: reformulation (”To Paris, is that correct?”,

”To Paris?”) [icm:und*chk:Content]

• acceptance/integration– negative:fb-phrase with reformulation (”Sorry, I

cannot answer Q”, ”Sorry, Paris is not a valid destination city.”) [icm:acc*neg:Content]

– positive: fb-word (”okay.”) [icm:acc*pos]

Feedback selection

• Feedback polarity selected depending on (for each level respectively):– Contact: Whether any speech is observed – Perception: Speech recognition score– Semantics: Whether system can find a non-

contextual interpretation– Pragmatic understanding: Whether system

can find a relevant contextual interpretation– Acceptance: Whether system can accept

what’s been said

Feedback on multiple levels

S> Lets see. What can I do for you? U> (mumble mumble)S> Sorry, I didnt hear what you said. What can I do for you?

U> siridusS> I heard you say siridus . Sorry, I dont understand. What can Ido for you?

U> channel oneS> channel one. I dont quite understand. Do you want to add aprogram or change channel?

Negative perception

feeback

Positive perception

Negative understanding

Positive understanding

(word meaning)

Negative understanding

(relevance) (Task clarification)

U> turn on the t vS> turn on the TV… Sorry, I cannot control the TV set. Do you want to

add a program or change channel?”

U> pardonS> Sorry, I cannot control the TV set. Do you want to add a program or

change channel?U> change channelS> Okay.

Rejection(negative

acceptance feedback)

Positiveunderstanding

User negative perception

System repetition

Positiveacceptance

Sequencing ICM

• Discourse particles– ”so” may be used to reraise a previously

discussed question– ”right”, ”anyway”, ”let’s see…”

• Verbose sequencing– ”Let’s talk about …”, ”Let’s return to the

issue of …”

• How do we classify these?– We will correlate them with updates to the

common ground

Sequencing moves generated by GoDiS

• Formal notation schema:– icm:Type{:Content}

• Loading a plan– icm:loadplan – ”let’s see.”

• Question accommodation– icm:accommodate – ”alright.”

• Reraising an issue– icm:reraise – ”so, ”– icm:reraise:Q – ”returning to the issue of …”

Sequencing ICM: reraising issue, loading a plan

S> Welcome to the travel agency! U> price information S> (…) Lets see. How do you want to travel? U> by flightS> (…) What city do you want to go to? U> parisS> (…) What city do you want to go from? U> do I need a visa ?S> (…) Lets see. What country are you from? U> swedenS> Okay. Yes, you need a Visa. S> Returning to the issue of price. Lets see. What city do

you want to go from?

Signals loading a plan

Reraising an issue explicitly

Sequencing ICM: Accommodation

S> Lets see. What can I do for you?U> channel five today

S> Alright. add a program… Lets see. Okay. channel five… today… What time do you want to start recording?

U> from six thirty to seven thirtyS> Okay. from six thirty… to seven thirty… The

program has been added. ”

Signals accommodation

Conclusions

• Current dialogue systems use only a fraction of the range of ICM used by humans

• By extending the range of ICM used by systems, their communication becomes more natural and comprehensive

• We have provided an initial classification of feedback and sequencing ICM useful in a dialogue system, and implemented it

• (ICM can be correlated with updates to common ground, and used to signal these updates to the user)

Future work

• Replace current ad-hoc forms based on corpus studies and existing work– ”discourse markers” (Schiffrin)– ”cue phrases” (Grosz & Sidner and others)

• ”Real” generation rather than templates– Integrate ICM with other utterances, rather than

just appending them

• Explore other ways of generating ICM– Intonation, facial expression, etc.

• Extend the range of ICM generated (and interpreted) by the system

Questions?

Object- or metalevel content

• Utterances with metalevel content explicitly refer to contact, perception, understanding or acceptance

• Object-level utterances instead refer to the task at hand• Example

– S: What city are you going to?– U: Paris– S(1a): Did you say you’re going to Paris? [meta]– S(1b): Are you going to Paris? [object]– S(2a): Do you mean Paris, France or Paris, Texas?– S(2b): Do you want to go to Paris, France or Paris, Texas?

• This dimension does not apply to all feedback, e.g. ”Paris.”, ”Pardon?”

• (Is 2b feedback or simply an alternative question?)

Realisation of feedback moves

• Syntactic form:– declarative: ”I didn’t hear what you said.”; ”The

destination city is Paris.”– interrogative: ”What did you say?”; ”Do you want to

go to Paris?”– imperative: ”Please repeat your latest utterance!”– elliptical

• interrogative: ”Paris?”, ”To Paris or from Paris?”• declarative: ”To Paris.”

• In general, the exact formulation of ICM phrases may depend on various contextual factors– including activity, noise level, time constraints etc.

Correcting the system

S> Lets see. What can I do for you? U> add programS> Okay. Add a program…S> Lets see. What channel do you want? U> channel sixS> OK, channel five.U> noS> not channel five. So, What channel do you want? U> channel sixS> Okay. channel six.”

(dialog4_edit)

Interrogative feedback

Negative answer to fb.

Signals reraising an issue (Sequencing

ICM)

Reactions to positive and checking feedback

• After positive feedback, system leaves a pause for the user to agree or disagree– if user gives no response, system

assumes it was right

• After check feedback, positive response is needed– or system will assume it was wrong

Implicit feedback?

• Clark: ”relevant followup” to U– What is relevant?

• simple cases for followups to questions:– answer to question– ”subquestion”– feedback concering question

• Complex cases: all other utterances– In general, complex inference and knowledge may be

needed (implicatures)– Currently, irrelevant followup counts as negative feedback

• What about no followup at all?– in reaction to ask-move or interrogative feedback, counts as

negative– in reaction to answer or positive feedback, counts as positive

Rejection?

S: ”Where do you want to go?”U1: ”Nowhere”U2: ”I don’t know”

• Should these count as rejections?– U1: negative answer? presupposition

failiure? rejection?– U2: rejection?

• but not as definite as ”No comment!”

Relation to Traum’s computational theory of

grounding• Focus on understanding-level

– ”grounding” here refers only to the understanding level

– Acceptance and rejection seen as ”core speech acts”

• Focus on positive feedback and corrections (self and other)– Based on the TRAINS corpus of H-H dialogue– Deals with the question, when does a contribution

end?– Corrections not included here; involves turntaking

and OCM

• Does not include sequencing ICM

GoDiS: an issue-based dialogue system

• Explores and implements Issue-based dialogue management (Larsson 2002)– Based on Ginzburg’s notion of a dialogue

gameboard involving Questions Under Discussion (QUD)

– Uses (mostly pre-scripted) dialogue plans

• Extends theory to more flexible dialogue– Multiple tasks, information sharing between tasks – Feedback and grounding– Question accommodation, re-raising, clarification

– …

Eliciting / nonelciting feedback

• (Allwood et. al. 1992)• Eliciting feedback is intended to

evoke a response from the user• Noneliciting feedback is not so

intended– But may nevertheless recieve a

response

Simplifying assumptions regarding feedback

• We only represent action level and polarity• In polarity, we replace ”neutral” by ”checking”

– We exclude feedback which is neutral but not check-questions

• Eliciting/noneliciting dimension implicit– Negative feedback is eliciting; since something went

wrong, it must be fixed– Checking feedback is also eliciting, since it poses a

question that must be adressed– Positive feedback is not eliciting (we assume)

• Syntactic form not included

Grounding

• ”To ground a thing … is to establish it as part of common ground well enough for current purposes.” (Clark)

• making sure that the participants are percieving, understanding, and accepting each other’s utterances