referential choice as a probabilistic multi-factorial process
DESCRIPTION
REFERENTIAL CHOICE AS A PROBABILISTIC MULTI-FACTORIAL PROCESS. Andrej A. Kibrik , Grigorij B. Dobrov , Natalia V. Loukachevitch , Dmitrij A. Zalmanov [email protected]. Referential choice in discourse. - PowerPoint PPT PresentationTRANSCRIPT
1
REFERENTIAL CHOICE
AS A PROBABILISTIC MULTI-FACTORIAL PROCESS
Andrej A. Kibrik, Grigorij B. Dobrov, Natalia V. Loukachevitch,
Dmitrij A. Zalmanov [email protected]
222
Referential choice in discourse
When a speaker needs to mention (or refer to) a specific, definite referent, s/he chooses between several options, including: Full noun phrase (NP)
• Proper name (e.g. Pushkin)• Common noun (with modifiers) = definite
description (e.g. the poet)
Reduced NP, particularly a third person pronoun (e.g. he)
3
Example
Tandy said consumer electronics sales at its Radio Shack stores have been slow, partly because a lack of hot, new products. Radio Shack continues to be lackluster, said Dennis Telzrow, analyst with Eppler, Guerin Turner in Dallas. He said Tandy has done <...>
How is this choice made?
Full NPPronounantecedent coreference
anaphors
4
Why is this important?
Reference is among the most basic cognitive operations performed by language users
It is the linguistic representation of what is known as attention and working memory in psychology
Reference constitutes a lion’s share of all information in natural communication
Consider text manipulation according to the method of Biber et al. 1999: 230-232
5
Referential expressions marked in green
Tandy said consumer electronics sales at its Radio Shack stores have been slow, partly because a lack of hot, new products. Radio Shack continues to be lackluster, said Dennis Telzrow, analyst with Eppler, Guerin Turner in Dallas. He said Tandy has done <...>
6
Referential expressions removed
Tandy said consumer electronics sales at its Radio Shack stores have been slow, partly because a lack of hot, new products. Radio Shack continues to be lackluster, said Dennis Telzrow, analyst with Eppler, Guerin Turner in Dallas. He said Tandy has done <...>
7
Referential expressions kept
Tandy said consumer electronics sales at its Radio Shack stores have been slow, partly because a lack of hot, new products. Radio Shack continues to be lackluster, said Dennis Telzrow, analyst with Eppler, Guerin Turner in Dallas. He said Tandy has done <...>
888
Plan of talk
I. Referential choice as a multi-factorial process
II. The RefRhet corpus and the machine learning-based approach
III. The probabilistic character of referential choice
999
Multi-factorial character of referential choice
Many factors of referential choice Distance to antecedent
• Along the linear discourse structure• Along the hierarchical discourse structure
Antecedent role Referent animacy Protagonisthood.........................................
None of these factors alone can explain referential choice
101010
Factors integration
At every poing in discourse factors are somehow summed and give rise to an integral characterization – the referent’s activation score
Activation score is the referent’s status with respect to the speaker’s working memory
Activation score predetermines referential choice Low full NP Medium full or reduced NP High reduced NP
111111
Multi-factorial model of referential choice
(Kibrik 1999)
Variousproperties
of the referentor
discourse context
Referent’s activation
score
Referential choice
Activation factors
121212
Modeling multi-factorial processes: machine learning-based methods
Neural networks approach (Gruening and Kibrik 2005) Machine learning algorithm
• Automatic selection of factors’ weights• Automatic reduction of the number of factors («pruning»)
However:• Small data set• Single method of machine learning• Low interpretability of results
Hence a new study Large corpus Implementation of several machine learning
methods Statistical model of referential choice
131313
The RefRhet corpus
English Business prose Initial material – the RST Discourse
Treebank Annotated for hierarchical discourse structure 385 articles from Wall Street Journal
The added component – referential annotation
The RefRhet corpus Over 30 000 referential expressions
141414
Example of a hierarchical graph
151515
Scheme of referential annotation
The ММАХ2 program Krasavina and Chiarcos 2007All markables are annotated,
including: Referential expressions Their antecedents
Coreference relations are annotated Features of referents and context are
annotated that can potentially be factors of referential choice
161616
171717
Work on referential annotation
O. KrasavinaA. AntonovaD. ZalmanovA. LinnikM. Khudyakova Students of the Department of
Theoretical and Applied Linguistics, MSU
181818
Current state of the RefRhet referential annotation
2/3 completed Further results are based on the following
data: 247 texts 110 thousand words 26 024 markables
• 7097 proper names• 8560 definite descriptions• 1797 third person pronouns
3756 reliable pairs «anaphor – antecedent»• Proper names — 1623 (43%)• Definite descriptions — 971 (26%)• Pronouns — 1162 (31%)
191919
Factors of referential choice
Properties of the referent: Animacy Protagonisthood
Properties of the antecedent: Type of syntactic phrase (phrase_type) Grammatical role (gramm_role) Form of referential expression (np_form,
def_np_form) Whether it belongs to direct speech or not
(dir_speech)
202020
Factors of referential choice
Properties of the anaphor: First vs. nonfirst mention in discourse (referentiality) Type of syntactic phrase (phrase_type) Grammatical role (gramm_role) Whether it belongs to direct speech or not (dir_speech)
Distance between the anaphor and the antecedent: Distance in words Distance in markables Linear distance in clauses Hierarchical distance in elementary discourse units
2121
Goals for the machine learning-base study
Dependent variable: Form of referential expression (np_form)
Binary prediction: Full NP vs. pronouns
Three-way prediction: Definite description vs. proper name vs. pronoun
Accuracy maximization: Ratio of correct predictions to the overall
number of instances
21
222222
Machine learning methods (Weka, a data mining system)
Easily interpretable methods: Logical algorithms
• Decision trees (C4.5)• Decision rules (JRip)
Higher quality:Logistic regression
Quality control – the cross-validation method
2323
Examples of decision rules generated by the JRip algorithm
(Antecedent’s grammatical role = subject) & (Hierarchical distance ≤ 1.5) & (Distance in words ≤ 7) => pronoun
(Animate) & (Distance in markables ≥ 2) & (Distance in words ≤ 11) => pronoun
23
2424
Main results
Accuracy Binary prediction:
logistic regression – 86.1% logical algorithms – 85%
Three-way prediction: logistic regression – 74% logical algorithms – 72%
24
2525
Comparison of single- and multi-factor accuracy
Feature Three-way prediction
Binary prediction
The largest class 43% 69%
Distance in words
55% 76%
Hierarchical distance
53.5% 74.8%
Anaphor’s grammatical role
45.2% 70%
Anaphor in direct speech
43.8% 70%
Animate 47.3% 71.5%
Combination of factors
74% 86.1%25
2626
Referential choice is a probabilistic process
According to Kibrik 1999Potential
referential expressions Actual
referential expressions
Full NP only (19%) Full NP (49%) Full NP, ?pronoun (21 %)
Pronoun or full NP (28%)Pronoun
(51%) Pronoun, ?full NP (23%)
Pronoun only (9%)
2727
Probabilistic character of referential choice in the RefRhet study
Prediction of referential choice cannot be fully deterministic
There is a class of instances in which referential choice is random
It is important to tune up the model so that it could process such instances in a special manner We are working on this problem
Logistic regression generates estimates of probability for each referential option
This estimate of probability can be interpreted as the activation score from the cognitive model
282828
Probabilistic multi-factorial model of referential choice
Activation score = probability of using
a certainreferential expression
Referentialchoice
Activation factors
Variousproperties
of the referentor
discourse context
2929
Conclusions about the RefRhet study
Quantity: Large corpus of referential expressions
Quality: A high level of accurate prediction is already attained And this is not the limit
Theoretical significance: the following fundamental properties of referential choice are addressed: Multi-factorial character of referential choice Probabilistic character of referential choice
This approach can be applied to a wide range of linguistic and other behavioral choices