Dialogue Managers in two projects: Comic and Amitie

Roberta Catizone

University of Sheffield

COMIC system features

Multimodal (speech and pen input)

Mixed initiative, but aimed to keep control

Bathroom application in two phases Inputting room measurements and features Browsing and Choosing tilesets for your bathroom 3-D view of you bathroom

COMIC system

Phase 1 Very goal driven

necessary for the user to input: fours walls + dimensions door placement window placement

Phase 2 Browsing so only weakly goal driven

User should select preferred tileset

COMIC architecture

Input from Fusion(XML)

Output to Fission (XML)

DAM decides what the system will do next

COMIC Dialogue Manager

General purpose control structure

Domain specific structures - DAFS

Modality independent

External resourcesDialogue HistoryOntology

Dialogue manager

The core mechanism for the DAM is a simple pop-push stack, onto which structures are loaded and run.

The structures are Augmented Transition Networks (ATNs) which have Turing power.

A control structure analyses the input and chooses an appropriate ATN to load – or continues to run the current ATN. This control structure has to deal with problems like topic change.

COMIC DM

Input from the Natural Language Understanding Module (DA, Event Type and Object type specified)

Output to Fission module with DA + Event type + Object Type) *There may be multiple output strings.

COMIC DM

DM domain knowledge built by hand using a GUI DAF editor

DAFs built manually using common sense knowledge based on how people would use the system (no data collection for pretuning)

Not built to discuss topics outside of the COMIC system.

Dialogue Action Forms (DAFs)

Augmented transition networks (nodes + arcs)Arcs are made up of a test and an action

(java programs)System executes the DAFs as specified

waiting for input where appropriate.

DAF example

Figure 2: A simple DAF

2 No test

1 3 “Please put in the wall length” “Thank you!”

Valid length

Invalid length

“Sorry, invalid length”

DAF example for showing and describing a tileset

DAF Gui editor

COMIC DAFs

DAFs prestacked, but there is a mechanism for overriding if necessary.

DAF identification and confirmation based on indexing terms (crude triple) Dialogue act (request, inform etc) Event type (show, input-dimension) Object type (wall, door, tileset, color etc)

Read/write access to a set of context registers

COMIC Control structure

Runs DAFs on the stack in order

Has the capability to work around a stack entry that needs rescheduling.

Phase 4 DAFPhase3 DAFPhase2 DAFwindow DAF

Phase1 DAF

bye DAFclarify ATN

window DAF

bye DAF

door DAF

COMIC system example

System Output: Greeting and phase1 introduction. Please input the first wall. Inform greeting and phase 1 introduction (sent to Fission) Request to draw a wall (sent to Fission) Expectation of a wall (sent to ASR, pen and Fusion)

User input: user draws a line on the touchscreen DAM input:

DialogueAct Response EventType Add_Bathroom_Part ObjectType Wall startY 100 startX 100 endY 100 endX 400 inkid 0 streamid1 1

COMIC system example

System Output: Please input the length of this wall Beautification of the wall (sent to visoft application) Request length of wall (sent to Fission) Expectation of a wall length (sent to ASR, pen and Fusion)

User input: something unintelligible DAM input: reason ASR_TOO_UNCERTAIN DialogueAct AnalysisError

COMIC system example

System output: Please repeat Request repeat (sent to Fission) Expectation of a wall length (sent to ASR, pen and Fusion)

User input: inputs the length of the wall DAM input:

DialogueAct Response ObjectType Wall EventType Modify_Bathroom_Part length Size sizeMeasure m sizeValue 3 ...

Amitie system

Telephone banking call center

Galaxy Communicator architecture

Amitie system features

Frame-filling approach Uses data-driven strategy Tasks

Verifying the customer’s identity Identifying the customer’s desired transaction Executing the transaction

Balance enquiry Report of lost or stolen card Debit card payments Change of customer’s address

Amitie Dialogue Manager

Frame filling process - data driven.

Order independent

As frames are filled, need for dialogue decreases

Maintains task history and attribute history

Amitie DM architecture

Task Info

Dialogue ActClassifier

Frame Agent

Task IDFrame Agent

Verify-Caller Frame Agent

Input from NLUVia HUB

(token string, language id,Named entities

Task ExecutionFrame Agents

Response Decision

CustomerDatabase

Dialogue history

External filesDomain-specific

Via HUB DB server

Amitie Dialogue Manager

Task HistoryRecords the topology of tasks

Tasks that have been executed successfully or unsuccessfully

Task currently under control

Amitie Dialogue Manager

Attribute HistoryRecords the list of attributes needed by the

current taskRecords attributes requested by the systemRecords attributes provided by the user

Amitie system

If the system fails to recognize or gather all the necessary data from the user, re-prompts are used, but not more than once. The user is passed to the customer service dept in the case of failure.

Amitie system task identification

To choose the task the customer wants to perform given An utterance A list of possible tasks

Collected data from over 500 call center conversations. Annotations

dialogic Stylistic Semantic

Discovered 14 distinct tasks, 4 chosen to be implemented.

Amitie classification in Task ID frame Agent Adapted a vector-based similarity method Uses term-vector approach common in

information retrieval Domain-independent & automatically trained Terms are stemmed content words found in

task-specific utterances Documents are vectors of weighted term

frequencies derived from the corpus.

Amitie training process purpose

Creates document vectors to be used in task identification

Queries are compared with documents to determine the most likely task.

Classification accuracy - 84.7% (based on confidence scores)

Amitie Dialogue Act Classifier

Purpose is to identify a caller’s utterance as one or more domain-independent dialogue acts: Accept, Reject, Non-understanding, Opening, Closing, Backchannel, Expression)

Amitie Dialogue Act Classifier

Trained the classifier on corpus of transcribed, calls and used vector-based classification techniques. Differs from the task identifier training in 2 ways

1) an utterance may have multiple correct classifications 2) a different stoplist is necessary

Performs well if utterance is short and falls into one of the selected categories (86%)

COMIC and Amitie DM comparison

Both use a form-filling approach for gathering necessary domain specific data

Both use general DAs that are domain independent Amitie classifies the user utterance (dialogue act,task id)

based on similarity to dialogue segments in training set. COMIC identifies the semantic content of a user utterance

(dialogue act, event type and object type) using the system expectation plus the output of the NLU module which assigns semantic classes based on sets of 1) pre-defined basic dialogue acts (questions vs statements), 2) verbs (show, describe) and 3) nouns (wall, door, tileset)