© löckelt, becker, pfleger, alexandersson; dfki edilog 2002 workshop jan alexandersson (tilman...

© Löckelt, Becker, Pfleger, Alexandersson; DFKI Edilog 2002 Workshop

Jan Alexandersson

(Tilman Becker, Markus Löckelt, Norbert Pfleger)

German Research Center for Artificial Intelligence – DFKI GmbHStuhlsatzenhausweg 3, Geb. 43.8 - 66123 Saarbrücken

phone: (+44) 681 302 5347Email: [email protected]

http://www.dfki.de/~janal

Overlay – a non-monotonic operation for discourse processing

© Alexandersson; DFKI Nancy 18.10.2002

What is this Talk about• Dialogue systems must deal with utterances relating

elliptically to previous dialogue

User: „half past five“

System: „at what time would you like to start recording?“

User: „I would like to start recording at half past five“

• ... or utterances only partially related to the previous discourse

System: [Showing a list of films] „Here is a list of films“

User: „what films are on TV tonight?“

User: ``Thats a boring program. I‘d rather go to the movies´´

User: ``Thats a boring program. I‘d rather go to the movies tonight´´


What is this Talk about?

``...an expensive large portrait of the queen by Wainright hanging in the library´´

``and one of the princess over the mantel´´

• There are similar elliptical phenomena like one-anaphora

``and an expensive large portrait of the princess hanging over the mantel´´


Overview

• SmartKom

• Domain Model

• Overlay

• Discourse Modelling

• Action Planning

• Processing Partial Utterances

• Conclusion


What is the challenge?

• General tasks for a dialogue manager– Enrichment:

• Given incomplete information, enrich it from other sources in order to proceed.

– Validation: • Validate hypotheses against the current discourse state


Our Approach

• One general operation – Overlay– Domain representation:

• (Typed) Feature Structures

– Non-monotonic:• Always succeeds

– Computes a score:• Reflecting the structural consistency of the two

arguments of overlay


SmartKom: Task-Oriented Dialogue System

MMDialogue

Back-Bone

Home:Consumer Electronics

EPG

Public:Cinema,

Phone, Fax,

Mail, Biometrics

Mobile:Car andPedestrianNavigation

Application

Layer

SmartKom-Mobile

SmartKom-Public

SmartKom-Home/Office


Module Overview on the SmartKom Control GUI


The True Story


The DFKI Dialogue Back-Bone

Communication pools

Main data flow

Context information

Analysers

ExternalServices

ModalityFusion

DiscourseModelling

DiM

ActionPlanning

AP

PresentationManager

Generators• Speech

• Gesture

• Speech

• Graphics

• Gesture


theater: MovieTheater

movie: Movie

reservationNumber: PositiveInteger

Domain Model• Used for communication in the back-bone

• Frame-based ontology; representation as Typed Feature Structures in M3L (XML)

name: Stringdirector: Person

cast: PersonListyearOfProduction: PositiveInteger…

address: Addressseats: SeatStructure…

CinemaReservation

• Application objects composed of subobjects

• Slots: Feature paths meaningful for the dialogue (entities that can be talked about / referenced to); e.g. movie:director:lastName in a CinemaReservation object

• Slots can recursively contain other slots

firstName: StringlastName: String…


Discourse Modeling• Lattice of intention hypothesis sequences

from Modality Fusion

• Enrichment and Validation

– compare and enrich with a selected number of discourse states:

• fill in consistent information

• compute a score

– for each hypothesis - background pair:• Overlay (covering, background)

• Overlay gives a score representing how well the covering fits the background.

• Scores of overlay, speech interpretation etc. give overall score

• One augmented hypothesis sequence with best score is passed on to Action Planning

Covering:Background:

Intentionlattice

Selectedaugmentedhypothesis

sequence


Example for Overlay with TFS

User: What films are on TV tonight?

System: [presents list of films]

User: That‘s a boring program, I‘d rather go to the movies.

How do we inherit “tonight” ?


The Domain Model

String:...

Time:entEntertainm

title

beginTime

...Cinema:

ePerformanccinema

...Channel:

Broadcastchannel

A named entertainment at

some time

A named TV program at some

time on some channel

A named Movie at some time at some cinema


Unification Simulation

Films on TV tonight

...:

"":

Broadcast

anychanneltonightTimebeginTime

Go to the movies

...:

:ePerformanc

anycinemaTimebeginTimeFail – type clash

String:...

Time:entEntertainm

title

beginTime

...Cinema:

ePerformanccinema

...Channel:

Broadcastchannel


...:

""Time:

ePerformanc

anycinematonightbeginTime

Overlay Simulation

String:...

Time:entEntertainm

title

beginTime

...Cinema:

ePerformanccinema

...Channel:

Broadcastchannel

Go to the moviesFilms on TV tonight

...:

"":

Broadcast

anychanneltonightTimebeginTime

...:

Time:ePerformanc

anycinemabeginTime

..."":

ePerformanc

tonightTimebeginTime

Assimilation

Background

Covering


``Formal´´ Definition Overlay

• Let– co be covering

– bg be background

• Step 1:– Assimilate(co,bg)

T

bg

co

• Step 2:– Overlay(co,assimilate(co,bg))

• If co and bg are frames: recursion

• If co is empty: use bg

• If bg is empty: use co

• If conflict: use co


Domain Models with Multiple Inheritance

• Assimilate(co,bg) – Compute the set of minimal upper

bounds (MUB)

– Specialize the MUBs

– Unify the specialized MUBs

T

co bg

• Overlay remains untouched

MUB MUB


Processing Partial User Contributions

• Our Dialogues contain a lot of partial contributions– ``Three´´

– ``Three o‘clock´´

– `Àt three o‘clock´´

– `Àt three [] o‘clock´´

– ``Later´´

– `Ìnformation []´´– ...

• Requirement– Advanced discourse modelling and action planning

– Bidirection communication DiM - AP


Discourse Modelling - Three-Tiered Context Representation

DO1 DO2

LO1

DO10

DO3 DO9

Modality layer

Discourse layer

System: Here [pointing gesture] I show you a list of films running in Heidelberg.

show heidelberglist

LO2 LO3. . .

Domain layer DomainObject1

ticket first

DO11 DO12

reserve

LO4 LO5 LO6

DomainObject2

GO1

. . .

. . .

User: Reserve a ticket for the first!


Action Planner – Regression Planning

• AP processes plan operators– Uniform description of mixed initiative with user or applications based on

Games and Moves. Example:• User: request – System: response

• System: request – Application: response

• Processing steps

1. Constructs a plan from plan operators– A Plan specifies a partial order of games

2. Interprets the plan– Execute the games

• Expectations are published during execution of the plan– Guide the interpretation of input


Action Planning - Regression planning

provides(Y)needs(D)

condition(D=x)

Goalprovides()

needs(X,Y)provides(X)needs(A,B)

provides(C)needs()

provides(B)needs(D)

provides(A)needs(C)

provides(F,G)needs(H)

provides(D)needs(E)

For Example: Slot D=y

Current Stateprovides(?)needs(?)

successorstates


AP - Example Plan: VCR

Goal: VCR_record

VCR_checkMedium

VCR_getChannel

VCR_getStartTime VCR_getEndTime

VCR_rewindMedium


Action Planning

• Initiation of sub-dialogues to gather necessary information

• Order proposed by the system, but the user may not follow the trail

• Several cases of user responses to requests:– Providing a matching answer,

– Supplying information not (yet) asked for,

– Changing already established information

– Acting uncooperatively or changing goals


Expectations

• When AP initiates a sub-dialogue, it expects a certain answer

• An Expectation structure is published to help the analysis modules– Expected Slots: corresponding to the request

– Possible Slots: plausible in context

– Filled Slots: already provided

– The currently active Application

• Allows for a fine-grained distinction between types of possible utterances

• Discourse Modelling can employ different rankings for each category


Integration of Partial

• Partial Utterances are analyzed as subobjects– validation and enrichment include the integration into the

preceding context (to achieve a coherent discourse)

– we consider three types of partial utterances: 1. The system has the initiative and the user responds elliptically to a

system request

2. An elliptical user utterance does not correspond to a request but can be interpreted in context

3. Misinterpretation or non-cooperative user behaviour


Integration of Partial - Possible Situations

User InitiativeSystem Initiative

Expected slot N A

expected,

unify,

plan continues

Possible slot

plausible,

unify,

replanning

plausible,

unify,

replanning

Filled slot

plausible,

overlay,

replanning

plausible,

overlay,

replanning

Other implausible – recover strategies

User contribution

DiM processing

AP processing


DiM - Processing User Responses

• User provides expected slots– Bridging:

• a new application object is created (of same type as the current application object)

• for each subobject in the hypotheses its type is compared to the expected slots

– matching subobjects are integrated into the new application object

– remaining subobjects are interpreted as plausible partial utterances (next slide)

– Overlay:• The extended application object is overlayed over the application object

currently in focus


Processing Plausible Partial Utterances

• User provides possible slots– processed like expected slots (but get a score penalty)

• User changes filled slots– search for a discourse object DO of the same type in the discourse

context

– create application object AO1 of the same type as the application object AO2 within which DO was mentioned

– overlay AO1 over AO2

– gets a score penalty based on position in discourse history

• If integration fails, all subobjects are passed on unchanged (integration fails)


Example: Expected and Possible Slots• (1) User: I want to record a film.

(2) System: When should I start recording?

(3) User: 1:30pm on channel two.

• Analysis of (1) contains the setting of a goal VCR_record (empty application object of type VCR introduced)

• AP triggers presentation (2) and publishes an expectation containing: – VCR as the type of the current application object

– VCR_startTime as the (only) expected slot

– some possible slots (e.g., VCR_endTime and VCR_channel)

– and maybe some filled slots (e.g. VCR_mediumPresent)

• Analysis of (3) contains two subobjects– the expected slot is filled by “1:30pm”

– but also one possible slot is filled by “channel two”


Example: Filled Slots

(1) User: What is currently running on TV?

(2) System: The following [Smartakus points at a list of programmes] programmes are currently running.

(3) User: And tonight?


Example: Filled Slots

show program currently

...

...

""

avMediumavMedium

now

timebeginTime

broadcast

ctdomainObje

hypothesis

Modality layer

Discourse layer

Domain layer

DO1 DO2

LO1 LO2

DO3

LO3

""tonight

time

...

...

""

avMediumavMedium

tonight

timebeginTime

broadcast

ctdomainObje

hypothesis

tonight

DO4

LO4

DO3


Conclusion

• Uniform, flexible and robust mechanism for interpreting possible partial utterances

• Pragmatical classification of partial utterances from the point of view of the action planner

• Modality independent

• Approach integrates seamlessly with other aspects of discourse processing, e.g. anaphora resolution

• Implemented and running in, e.g., a large multimodal dialogue system


Papers

• Jan Alexandersson and Tilman Becker. Overlay as the Basic Operation for Discourse Processing in a Multimodal Dialogue System. In: Proceedings of the IJCAI Workshop ``Knowledge and Reasoning in Practical Dialogue Systems,´´ Seattle, 2001.

• Norbert Pfleger, Jan Alexandersson, and Tilman Becker. Scoring Functions for Overlay and their Application in Discourse Processing. In Proceedings of ``KONVENS 2002,´´ Saarbrücken, Germany, 2002.

• Markus Loeckelt, Tilman Becker, Norbert Pfleger and Jan Alexandersson Making Sense of Partial. In: Bos, Foster & Matheson (eds): ``Proceedings of the sixth workshop on the semantics and pragmatics of dialogue (EDILOG 2002),´´ Edinburgh, UK, Pages 101-107.

• Jan Alexandersson and Tilman Becker. The Formal Foundations Underlying Overlay. Submitted to IWCS5


Thank you very much for your attention!Merci infiniment de votre attention!Babelfish.altavista.com

© löckelt, becker, pfleger, alexandersson; dfki edilog 2002 workshop jan alexandersson (tilman...

Documents