conceptual coherence in the generation of referring expressions
DESCRIPTION
conceptual coherence in the generation of referring expressions. Albert Gatt & Kees van Deemter University of Aberdeen {agatt, kvdeemte}@csd.abdn.ac.uk. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/1.jpg)
conceptual coherence in the generation of referring expressions
Albert Gatt & Kees van Deemter
University of Aberdeen
{agatt, kvdeemte}@csd.abdn.ac.uk
![Page 2: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/2.jpg)
Gatt and Van Deemter 2007: “Lexical Choice and conceptual perspective in the generation of plural referring expressions”. Journal of Logic Language and Information (JoLLI) 16 (4), p.423-444.
![Page 3: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/3.jpg)
some received wisdom…
Choice is ultimately dependent on the perspective you decide to take on the referent (...).
Will it be more effective for me to refer to my sister as my sister or as that lady or as the physicist ? (Levelt `99, p. 226)
![Page 4: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/4.jpg)
the rest of this talk…
1. Generation of Referring Expressions2. Perspective and Conceptual Coherence
reference to sets experimental work
3. An algorithm evaluation
4. Extensions: local (Conceptual) Coherence in discourse
![Page 5: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/5.jpg)
Generation of Referring Expressions (GRE)
Part of micro-planning (Reiter/Dale `00)
At this stage, the content of a message is being determined, including descriptions of domain objects (Noun Phrases)
The task of GRE:– given a set of intended referents, look up properties of these
referents that will distinguish them from their distractors in a Knowledge Base
![Page 6: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/6.jpg)
Content determination strategies
Most algorithms inspired by the Gricean maxims (Grice `75)– especially Brevity (Dale `89, Gardent `02)
But compare:?? λx: professor(x) V plump(x)
?? λx: professor(x) V [plump(x) & man(x)]
λx: biologist(x) V physicist(x) Not all of these have an equally good ring to them.
entity base type occupation specialisation girth
e1 woman professor physicist plump
e2 woman lecturer geologist thin
e3 man lecturer biologist thin
e4 man postgraduate thin
![Page 7: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/7.jpg)
the Conceptual Coherence constraint
Sets (and disjunction): λx: p(x) V q(x) ‘the p and the q’– reference to a plurality suggests to the listener that there is a relationship
holding between elements of the pluralities– p and q should be related or “similar”– semantic relatedness allows the listener to conceptualise the plurality more
easily (Sanford and Moxey, `95)
Gatt and van Deemter (`02):– People’s preference for descriptions of this form were highly correlated to
the semantic similarity of disjuncts– Best results achieved with a distributional definition of similarity (Lin `98)– sim(w,w’) is a function of how often w and w’ occur in the same
grammatical relations in a corpus
![Page 8: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/8.jpg)
Lin’s definition of distributional similarity
Let w1, w2 be two words of the same grammatical category.
E.g. dog, cat GR contains information about a syntactic relation w occurs in:
– GR = <w, R, x, p>– w the target word, R the relation, x the co-argument of w– p is the probability of w and x occurring in this construction (as
mutual information).– Example: <dog, modified-by, stray, 0.002>
sim(w1, w2) is calculated using the GR triples that w1 and w2 share.
We use SketchEngine, a large-scale implementation of this theory, based on the BNC (Kilgarriff, `03)
![Page 9: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/9.jpg)
experiment 1: multimodal sentence completion
General idea:– To refer to a set, people will prefer to use a plural that
respects the conceptual coherence constraint– If this is impossible, then they will break down the set in
manageable parts. Experimental domains:
– 3 targets (a,b,c) + 1 distractor (d)– sim(a,b) could be high or low– sim(a,c) ≈ sim(b,c) = low
Expectation:– if 2 of the targets have semantically high-sim types, they will
be referred to in a plural description
![Page 10: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/10.jpg)
experiment 1: example domain
£5
£5 £5
£20
Complete the following by clicking on the pictures:
The _____________ and the _____________ cost £5.
The _____________ also costs £5.
Experimental domain:
1. Participants completed the sentences by clicking on the pictures.
2. Manipulation of similarity of two of the objects (a,b).
3. Hypothesis:
If {a,b} are similar, they are more likely to be referred to in the plural.
a
bc
d
![Page 11: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/11.jpg)
experiment 1: results
Proportion of plural references to designated targets {a,b} when:
{a,b} are semantically similar {a,b} are semantically dissimilar
![Page 12: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/12.jpg)
experiment 2: sentence continuation
Does similarity play a role in content determination?
Distinguishing properties: nouns (12) or adjectives (12 ). Expectation:
– Participants will select similar properties in the plural description
A university building was robbed last night. The police have detained three suspects for questioning, all of whom work or study at the university. 1. One of them is a postgraduate. He is a physicist. 2. Another is a Greek, an undergraduate. 3. Also among the suspects is a cleaner. He is an Italian. Both ______________________ were held in custody, but the physicist was released last night.
![Page 13: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/13.jpg)
experiment 2: results
Friedman 45.89, p < .001trend as expected
Friedman 36.3, p < .001trend in the opposite direction
Proportion of references using pairwise similar properties:
Nouns: Adjectives:
![Page 14: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/14.jpg)
summary of findings so far
In referential situations, people prefer to produce plural descriptions if the entities can be conceptualised under the same perspective.
This holds for types, but not modifiers– Types correspond to “concepts”, and are the way we carve
up the world and categorise objects– Modifiers correspond to properties of those objects.
Results have been corroborated in other experiments
![Page 15: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/15.jpg)
Aloni (2002): answers to questions “wh x?” must conceptualise the different x using one and the same perspective (relevant given hearer’s information state and the context)
Our experiments confirm that this idea is on the right track …
![Page 16: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/16.jpg)
The challenge for an algorithm:
Complete coherence is often not possible “the Italian, the Greek and the Spaniard” –
But what if there are 5 Spaniards? “the Italian, the Greek and ?” – What if you
don’t know the person’s nationality? “the table, the chair and the plant” – What if
you need to refer to an object that’s of different kind of the other two?
![Page 17: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/17.jpg)
a GRE algorithm
The algorithm should try to find the most coherent description possible. Assumption: this should be done even at the cost of brevity!
Main knowledge source:– The relation sim (Kilgarriff `03)
Input:– Knowledge Base– Target referents (R )
![Page 18: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/18.jpg)
step 1
1. Lexicalise properties in the KB2. Identify types (nominal properties) and modifiers The set of types and the similarity relation define a
semantic space S = <T, sim>
Definition 1: PerspectiveA perspective P is a convex subset of S, i.e.:
∀ t, t’, t’’ T: ∈t, t’ ∈ P & sim(t, t’’) ≥ sim(t, t’) t’’ P∈
Computed using a clustering algorithm (Gatt `06), which recursively groups together semantic nearest neighbours.
![Page 19: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/19.jpg)
perspective graph
T: {lecturer, professor, postgraduate}
T: {woman, man}M: {plump, thin}
T: {geologist, physicist,biologist, chemist}
32
1
1 0.6
1
Aim: find a description for R that minimises the distance between perspectives from which properties are selected.
Weight of a description, w(D): the sum of distances between perspectives represented in D.
– w( ‘the professor and the plump man’ ) = 1– w( ‘the biologist and the physicist’ ) = 0
![Page 20: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/20.jpg)
descriptive coherence
Definition 2: Maximal coherence
D is maximally coherent if there is no D’ coextensive with D such that w(D’) < w(D)
– implies finding a shortest connection network in the perspective graph (intractable!)
Definition 3: Local coherenceD is locally coherent if there is no D’ coextensive with D s.t.:1. D’ is obtained by replacing a perspective in D 2. w(D’) < w(D)
![Page 21: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/21.jpg)
N ∅ //the perspectives represented in D root perspective with most referents in its extension starting from root do:
– Check types and modifiers. – If a property excludes distractors:
add it to D add the perspective to N
– If R is not distinguished, go to the next perspective, which is
search procedure
NuVPPuw ),(min
(V is the set of perspectives).
![Page 22: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/22.jpg)
evaluation
Do people prefer coherence over brevity?– (Two Gricean maxims: “Be brief” vs. “Be orderly”)
Method: subjects (N = 39) shown 6 discourses. – Each discourse introduces 3 entities– Followed by 2 possible continuations– Subjects had to indicate their preferred continuation
Each of the 6 discourses represented a condition: – Brevity: descriptions equally (in-)coherent, but one is brief– Coherence: descriptions equally (non-)brief; only one is
coherent– Trade-off: coherent description is non-brief
![Page 23: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/23.jpg)
Example: the domain
Three old manuscripts were auctioned at Sotheby’s:
e1: One of them is a book, a biography of a composer
e2: The second, a sailor’s journal, was published in the form of a pamphlet. It is a record of a voyage.
e3: The third, another pamphlet, is an essay by Hume
![Page 24: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/24.jpg)
Intuitively, this is about texts– of different genres (e.g., essay)– published in different forms (e.g., pamphlet)
Of course our corpus-based model doesn’t use these concepts …
![Page 25: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/25.jpg)
Example: continuations:
(+c,-b) The biography, the journal and the essay were sold to a collector
(+c,+b) The book and the pamphlets were sold to a collector
(-c,+b) The biography and the pamphlets were sold to a collector
(-c,-b) The book, the record and the essay were sold to a collector
![Page 26: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/26.jpg)
results: no preference for brevity
both descriptions coherentx2 = .023, p = .8
both descriptions non-coherentx2 = .64, p = .4
![Page 27: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/27.jpg)
results: preference for coherence
both descriptions minimalx2 = 16.03, p < .001
both descriptions non-minimalx2 = 13.56, p < .001
![Page 28: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/28.jpg)
results: trade-off
x2 = 39.0, p < .001
Finally, (+c,-b) preferred over (-c,+b)
In other words Coherence was more important than brevity In fact, brevity made no difference at all!
– we did not confirm that +b is preferred over –b
![Page 29: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/29.jpg)
Conclusion
When it’s impossible to use the same perspective, use perspectives that are similar
A version of Grice’s maxim “be orderly”?
![Page 30: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/30.jpg)
Methodology
Many experiments were done– to find a suitable notion of similarity/coherence– to discover how coherence and brevity relate
Different algorithmic interpretations would be possible
Algorithms are almost always under-determined by the empirical evidence
![Page 31: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/31.jpg)
A limitation
Ambiguity/polysemy is not taken into account For example, we might generate
– “the river and the/its bank”
These issues investigated in Imtiaz Khan’s PhD project
One remark: “river” might disambiguate “bank”
![Page 32: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/32.jpg)
An open question
Why doesn’t coherence play the same role for modifiers as for types?
![Page 33: conceptual coherence in the generation of referring expressions](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813c5f550346895da5e375/html5/thumbnails/33.jpg)