Incremental generation of spatial referring expressions in situated dialogJohn D. Kelleher, Geert-Jan M. Kruijff

Dublin Institute of TechnologyDublin, Ireland

ACL2006

Introduction A computational approach to the generation of spatial locative expressions

Ex. the book [T] on the table [L] T=target, L=landmark

可以用在 Image annotation task 上嗎 ? 傳統的 image annotation 沒有位置資訊

Introduction Given

(1) domain description (2) target object Generate

description of the target object that distinguishes it from the other objects (distractor objects) in the domain

Introduction 之前的方法 - predefined scene model 作者的方法 - dynamically construct scene model Problem

Too many objects Too many relations

Solution 要能決定 object 的重要性和 relation 的重要性

Data English has more than eighty spatial prepositions

static prepositions Ex. the tree is behind [static] the house

dynamic prepositions the man walked across [dyn.] the road

Data static prepositions can be divided into two sets:

Topological: a region that is proximal to the landmark (ex. at, near) Projective: a region projected from the landmark in a particular direction (ex. to the right of, to the left of)

Data projective prepositions: the circle on the left of the square topological prepositions:the circle near the black square topological and projective prepositions: the circle to the right of the square + the circle near the square

Approach We base our GRE approach on an extension of the incremental algorithm

A property is added to the list of selected properties if it reduces the size of the distractor object set The algorithm succeeds

all the distractors have been ruled out fails

all the properties have been processed and there are still some distractor objects

Cognitive Ordering of Contexts geometric level, functional level, pragmatic level Cognitive load

we propose the following ordering: Topological contrastive < topological relative < projective constrastive < projective relative

Cognitive Ordering of Contexts the contrastive-relative distinction is dependent on the number of objects within the region of space defined by the preposition

Landmarks and Descriptions A landmark can be

the speaker :the ball on my right [speaker] the hearer :the ball to your left [hearer] the scene :the ball on the right [scene] an object in the scene :the ball to the left of the box [an object in the scene] a group of objects in the scene :the ball in the middle [group of objects]

Landmarks and Descriptions Intuitively, in most situations, either of the interlocutors are ideal landmarks generally, the landmark object is more permanently located, larger, and taken to have greater geometric complexity However, the salience of an object is intrinsically linked to the context it is embedded in

Landmarks and Descriptions basic constraint on landmark selection is that the landmark should be distinguishable from the target

Ex. the man to the left of the man Ex. the man to the right of the ball

we treat an object as a candidate landmark if the following conditions are met the object is not the target it is not in the distractor set either

Landmarks and Descriptions target landmark

a member of the candidate landmark set that stands in relation to the target distractor landmark

a member of the candidate landmark set that stands in the considered relation to a distractor object

Algorithm

Algorithm If we cannot create a distinguishing locative description we face two choices

(1)iterate on to the next relation in the hierarchy (2) create an embedded locative description distinguishing the landmark Ex. the dog to the right of the car Ex. the dog near the car to the right of the house

Algorithm An important point in this context is the issue of infinite regression

Ex. 花在桌子上面 , 桌子在花下面 , 花在桌子上面 This way the algorithm cannot distinguish a landmark using its target Complexity - polynomial complexity

Example

Algo1 Result: ballAlgo2 Result: the ball near the box. (B1=target)

Related work ?? Krahmer and Theune 也提出過 relational subsumption hierarchy subsumption hierarchy illustrated in (Krahmer and Theune, 2002) the relation next to subsumes the relations left of and right of

Conclusions We have presented a framework that addresses this issue by

contextually defining the set of objects in the context that may function as a landmark sequencing the order in which spatial relations are considered using a cognitively motivated hierarchy of relations