statistical modelling of metaphor -...

Statistical Modelling of Metaphor

Ekaterina Shutova

International Computer Science Institute,Institute for Cognitive and Brain Sciences

University of California, Berkeley

EACL 2014 Workshop on Multiword Expressions

1 / 49Statistical Modelling of Metaphor

N

Modelling metaphor: Why?

I think that metaphor really is a key to explainingthought and language. [..] Our powers of analogyallow us to apply ancient neural structures tonewfound subject matter, to discover hidden laws andsystems in nature, and not least, to amplify theexpressive power of language itself. (Pinker, 2007)

Metaphor structures our conceptual system

It helps us derive and comprehend new information

It is frequent in language

It is important in building computer applications that carry outhuman-like interaction


N

What is metaphor?

“A political machine”


N

What is metaphor?

“A political machine”

“The wheels of Stalin’sregime were well oiledand already turning”

“Time to mend ourforeign policy ”

“20 Steps towards aModern, WorkingDemocracy ”


N

How does it work?

Conceptual Metaphor Theory(Lakoff and Johnson, 1980)

Metaphorical associations between concepts

POLITICALSYSTEM︸︷︷︸target

is a MECHANISM︸︷︷︸source

Cross-domain knowledge projection and inference

Reasoning about the target domain in terms of the propertiesof the source


N

Metaphor influences our decision-making

Thibodeau and Boroditsky (2011)

investigated how metaphor influencesdecision-making

subjects read a text containing metaphorsof either

1 CRIME IS A VIRUS2 CRIME IS A BEAST

then they were asked a set of questions onhow to tackle crime in the city

1 preventive measures2 punishment, restraint


N

Real-world text processing applications

that we no longer believe in hell , and that mutes , carrying black ostrich plumes , are out of favour . The changes have been more fundamental and some of them may have affected us in ways that we do not immediately recognise . All centuries , of course , have been centuries of change ; but few would deny that in the nineteenth century <m>change was greatly accelerated ; that much was apparent to the more perceptive of those living at that time . Some felt that they were <m>hurrying into an epoch of unprecedented enlightenment , in which better education and beneficent technology would ensure wealth and leisure for all . This was Herbert Spencer 's view , namely that an <m>upward evolutionary process was inherent in the human condition . To others , including Tennyson and Arnold , it seemed as if ‘ <m>the ringing grooves of change ’ were carrying them at <m>break-neck speed into a <m>future full of uncertainty and alarm . Browning was more hopeful , but he , too , was impressed by the transiency of the <m>world , flashing past the carriage windows : ‘ Must the rose sigh ‘ Pluck — I perish ! ’ Must the eve weep ‘ Gaze — I fade ! ’ ’ The Impressionist painters <m>caught the contagion , and <m>the new race of photographers tried to <m>seize the fleeting moment and <m>make it stay . Cultures and historical periods differ greatly in their concepts of time and the continuity of life . We live in a <m>century imprinted on the present , which regards the <m> past as little more than the springboard from which we <m>were launched on our way . Ours , for better or for worse , is the century of youth . Earlier centuries , in contrast , had an appreciation of the past that embodied more than nostalgia or antiquarian interest . Age and experience were valued in the belief that <m>length of days had provided some guidance on how to live and what to expect in the life to come . For the <m> life on earth of each individual was not a finite entity , complete in itself , but <m> a transition to another <m>mode of existence ; <m> the gateway to that unknown land was death — Mors Janna Vitae , as the memorial tablets had it . This belief was fostered by the churches , the floors and walls of which were incised with the records of those who had gone before , but it was expressed positively in the family . Families cherished their forbears , whether these had lived in humble cottages or in manor houses . Sons aspired <m>to follow in their fathers ' trades or professions . The landed gentry <m> planted for their grandchildren avenues of hardwood that they themselves would never see . In the nineteenth century this leisurely <m> view of the <m>pageant of time began to speed up . <m>People became more mobile , both physically and socially ; <m> men wished to rise in the world . Young <m> men with feet on the ladder , provided by the Industrial Revolution , <m> looked askance at the old-fashioned ways of their fathers . Growing more acquisitive in the present , they prepared to <m>disown the past ; the future was to be different , both for themselves and their children , and they had to <m>run to catch up with it . Improving life expectancy gave them every hope of doing so , especially if they belonged to the <m>rising middle-class . For the middle-class was both the agent and product of these changes . <m>The rise of the middle-class was not , on the whole , predicated on an aspiration to join the aristocracy , whose way of life , especially during the Regency , <m>met with a good deal of disapprobation ; but it was determined by the resolute intention of the new men to <m>distance themselves in every possible way from the working-class , out of which so <m>many of them had raised themselves . Their rise during the Industrial Revolution was expressed in capital accumulation ; the status of the aristocracy still derived from birth and ownership of land . The new men were not aping the landed gentry ; they were <m>basing their careers upon the infrastructure provided by urban Britain . There was no coherent <m>ideology embracing the entire middle-class , but there were two ideologies that subsumed its more active sectors . The older one was that of the Evangelicals and Dissenters , of whom more will be written in chapter three . The newer ideology was that of the followers of Jeremy Bentham ( 1745–1832 ) , the so-called Utilitarians ; it was far from <m> sharing a common world view with the Evangelicals , but there were certain social issues , such as abolition of slavery , on which concerted action was possible . Macaulay was one of those who <m>had a foot in

Metaphor occurs on averagein every third sentence!(according to corpus studies)

Information Retrieval

Machine Translation

Sentiment Analysis

Question Answering

Information Extraction

Text Mining


N

Metaphor processing tasks

1 Learn metaphorical associations from corpora

“POLITICAL SYSTEM is a MECHANISM”

2 Identify metaphorical language in text

“mend the policy ”

3 Interpret the metaphorical language

“mend the policy ” means “improve the policy;address the downsides of the policy"


N

Metaphor processing tasks

1 Learn metaphorical associations from corpora

“POLITICAL SYSTEM is a MECHANISM”

2 Identify metaphorical language in text

“mend the policy ”

3 Interpret the metaphorical language

“mend the policy ” means “improve the policy;address the downsides of the policy"

DATA-DRIVEN STATISTICAL APPROACH!


N

Data-driven statistical approach

extract contextual informationfrom a large text corpus

to do this...

parse the corpus

extract grammatical relationsbetween words

and co-occurrence statistics

V: drink V: mend248 water 20 fence197 tea 18 way193 coffee 12 car110 wine 9 roof84 beer 8 shoe... ...


N

Example feature vectors (verb–object relations)

N: game N: politics1170 play 31 dominate202 win 30 play99 miss 28 enter76 watch 16 discuss66 lose 13 leave63 start 12 understand42 enjoy 8 study22 finish 6 explain... 5 shape20 dominate 4 influence18 quit 4 change17 host 4 analyse17 follow ...17 control 2 transform... ...


N




N



NEED TO FIND A GOOD WAY TO PARTITION THE SPACE!


N

Semi-supervised learning for metaphor identification

Start from a set of seed metaphorical expressions

Implicitly model the underlying metaphorical associations

Identify new metaphorical associations and expressions

by means of spectral clustering and contextual features


N

Clusters

marriage affair democracy cooperation

relationship ...

mechanism computer bike

machine engine ...

work mend repair function oil

operate run break

ABSTRACT CONCRETE

VERBS


N

Clusters


relationship ...


machine engine ...


operate run break

ABSTRACT CONCRETE

VERBS


N

Output and results

Output sentences from the British National CorpusK2W 1771 The committee heard today that gangs regularly hurledabusive comments at local people.AD9 3205 He tried to disguise the anxiety he felt when he foundthe comms system down, [..]ADK 634 Catch their interest and spark their enthusiasm sothat they begin to see the product’s potential.

Evaluation:

Against human judgements of system output

compared to a baseline using WordNet

5 annotators (agreement κ = 0.63)

evaluation in terms of precision (P): system P = 0.79 vs.baseline P = 0.44


N

Unsupervised learning for metaphor identification


N

Creating the graph

Experimental setupALGORITHM: Hierarchical graph factorization clustering(HGFC) (Yu, Yu and Tresp, 2006)

DATASET: 2000 most frequent nouns in the BNC

FEATURES: subject, direct and indirect object relations; verblemmas indexed by relation type (extracted from Gigaword)

LEVELS: 10


N

Hierarchical clustering using graph factorization


N



N

Identifying metaphorical associations in the graph

start with the source concept, e.g. "fire"

output a ranking of potential target concepts

SOURCE: fireTARGET: sense hatred emotion passion enthusiasm sentiment hope inter-est feeling resentment optimism hostility excitement angerTARGET: coup violence fight resistance clash rebellion battle drive fightingriot revolt war confrontation volcano row revolution struggle

SOURCE: diseaseTARGET: fraud outbreak offence connection leak count crime violationabuse conspiracy corruption terrorism suicideTARGET: opponent critic rival


N

Identifying metaphorical expressions


N

Metaphorical expressions retrieved

FEELING IS FIRE

anger blazed (Subj), passion flared(Subj), interest lit (Subj), fuel resent-ment (Dobj), anger crackled (Subj), lightwith hope (Iobj) etc.

CRIME IS A DISEASE

cure crime (Dobj), abuse trans-mitted (Subj), suffer fromcorruption (Iobj), diagnose abuse(Dobj) etc.

Output sentences from the BNCEG0 275 In the 1930s the words "means test" was a curse, fuelling theresistance against it both among the unemployed and some of itsadministrators.HL3 1206 [..] he would strive to accelerate progress towards the eco-nomic integration of the Caribbean.HXJ 121 [..] it is likely that some industries will flourish in certain coun-tries as the market widens.


N

Evaluation

Source target domain mappings

Selected 10 source domains

Extracted top 5 target domains for each source

Evaluated precision against the judgements of 2 annotators:agreement κ = 0.60, system P = 0.69 (human P = 0.80)

Recall against a human-constructed gold standard: R = 0.61

Metaphorical expressions

For all of the above mappings, we extracted all metaphoricalexpressions the system found.

Random sample of 211 expressions

Evaluated precision against the judgements of 2 annotators:agreement κ = 0.56; system P = 0.65 (human P = 0.79)


N

Multilingual metaphor processing

Statistical methods are portable to other languages

Metaphor identification systems for Russian and Spanish:

work!reveal a number of interesting cross-cultural differences

Cross-cultural differences identified by the systemSpanish: stronger metaphors for poverty (fight poverty, eradicatepoverty -> POVERTY IS AN ENEMY, PAIN etc.)English: stronger metaphors for immigration (IMMIGRATION IS ADISEASE etc.)Russian: sporting events / competitions associated with WAR

37 / 49Statistical Modelling of MetaphorN

Metaphor interpretation as paraphrasing

Identifying the closest literal paraphrase

PhrasesThis news stirred an uncontrollable excitement in her.a carelessly leaked report

ParaphrasesThis news provoked an uncontrollable excitement in her.a carelessly disclosed report


N

Paraphrasing system overview

“carelessly leaked report” –> “carelessly ... report”

1 Paraphrase selection model: meaning retention

2 WordNet similarity filtering

3 Selectional preference model: quantifying literalness


N

Context-based paraphrase ranking model

Example

carelessly leaked report –> carelessly (w1) ... (i) report (w2)

P(i , (w1, r1), (w2, r2), ..., (wN , rN)) =

P(i) · P((w1, r1)|i) · ... · P((wN , rN)|i)where i is the interpretation, w1, ..., wN and r1, ..., rN represent the context

P(i) =f (i)∑k f (ik )

P(wn, rn|i) =f (wn, rn, i)

f (i)

where f (i) is the frequency of the interpretation on its own

f (wn, rn, i) - the frequency of the co-occurrence of the interpretation with the context word wn in the relation rn .

P(i , (w1, r1), (w2, r2), ..., (wN , rN)) =

∏Nn=1 f (wn, rn, i)

(f (i))N−1 ·∑

k f (ik )


N

Shared features in WordNet

The paraphrasing model overgenerates

Thus we need to filter out unrelated verbs

Metaphor is based on similarity

We define similarity as sharing a common hypernym in WordNet

Example

How can I kill a process?How can I terminate a process?Kill and terminate share a common hypernym.


N

Example output

Candidate paraphraseshold back truth:-13.09 contain-14.15 conceal-14.62 suppress-15.13 hold-16.23 keep-16.24 defendstir excitement:-14.28 create-14.84 provoke-15.53 make-15.53 elicit-15.53 arouse-16.23 stimulate-16.23 raise-16.23 excite-16.23 conjure


N

Selectional preference model

Selectional preference strength (Resnik, 1993)

SR(v) = D(P(c|v)||P(c)) =∑

c

P(c|v) logP(c|v)P(c)

Selectional association (Resnik, 1993)

AR(v , c) =1

SR(v)P(c|v) log

P(c|v)P(c)

P(c) is the prior probability of the noun class, P(c|v) its posterior probability given the verb; R is the GR


N

Paraphrasing system output

Initial rankingstir excitement:-14.28 create-14.84 provoke-15.53 make-15.53 elicit-15.53 arouse-16.23 stimulate-16.23 raise-16.23 excite-16.23 conjurehold back truth:-13.09 contain-14.15 conceal-14.62 suppress-15.13 hold-16.23 keep-16.24 defend

SP rerankingstir excitement:0.0696 provoke0.0245 elicit0.0194 arouse0.0061 conjure0.0028 create0.0001 stimulate≈ 0 raise≈ 0 make≈ 0 excitehold back truth:0.1161 conceal0.0214 keep0.0070 suppress0.0022 contain0.0018 defend0.0006 hold


N

Evaluation

Setting 1

the annotators were given the interpretations produced bythe system and asked to evaluate them

7 annotators (agreement κ = 0.62)

evaluation in terms of accuracy

system accuracy = 0.81 vs. baseline accuracy = 0.55

Setting 2

the annotators were given the list of metaphors and asked toparaphrase them

5 annotators

evaluation in terms of mean reciprocal rank (MRR)

system MRR = 0.63 vs. baseline MRR = 0.55


N

What’s next?

1 Moving up in accuracy, while maintaining scalability

improving statistical techniques

2 Modelling meaning and inference

3 Real-world applications

NLP applicationsData mining


N

Things to remember going forward

1 Metaphor is a well-structured phenomenon suitable forcomputational modeling

2 reveals a lot about cognition and has scientific relevance

3 frequent in language, has numerous real-world applications

4 used in a range of creative tasks and plays a part in innovation

5 Statistical methods yield promising results

6 Interdisciplinary approach: incorporating insights fromlinguistics, cognitive science and computer science.

7 Far from being a solved problem!


N

Would you like to learn more?

The Second Workshop on Metaphor in NLP at ACL 2014

email: [email protected]


N

Questions?

1 Metaphor is a well-structured phenomenon suitable forcomputational modeling

2 reveals a lot about cognition and has scientific relevance

3 frequent in language, has numerous real-world applications

4 used in a range of creative tasks and plays a part in innovation

5 Statistical methods yield promising results

6 Interdisciplinary approach: incorporating insights fromlinguistics, cognitive science, and computer science.

7 Far from being a solved problem!

email: [email protected]


N

References I

G. Lakoff and M. Johnson.1980.Metaphors We Live By.University of Chicago Press, Chicago.

G. Lakoff and E. Wehling.2012.The Little Blue Book: The Essential Guide to Thinking and Talking Democratic.Free Press, New York.

J. H. Martin.1990.A Computational Model of Metaphor Interpretation.Academic Press Professional, Inc., San Diego, CA, USA.

S. Pinker.2007.The Stuff of Thought: Language as a Window into Human Nature.Viking Adult, USA.

J.A. Barnden and M.G. Lee.2002.An artificial intelligence approach to metaphor understanding.Theoria et Historia Scientiarum, 6(1):399–412.


N

References II

D. Fass.1991.met*: A method for discriminating metonymy and metaphor by computer.Computational Linguistics, 17(1):49–90.

S. Krishnakumaran and X. Zhu.2007.Hunting elusive metaphors using lexical resources.In Proceedings of the Workshop on Computational Approaches to Figurative Language, pages 13–20,Rochester, NY.

J. H. Martin.2006.A corpus-based analysis of context effects on metaphor comprehension.In A. Stefanowitsch and S. T. Gries, editors, Corpus-Based Approaches to Metaphor and Metonymy, Berlin.Mouton de Gruyter.

Z. J. Mason.2004.Cormet: a computational, corpus-based conventional metaphor extraction system.Computational Linguistics, 30(1):23–44.

S. Narayanan.1999.Moving right along: A computational model of metaphoric reasoning about events.In Proceedings of AAAI ’99), pages 121–128, Orlando, Florida.


N

References III

P. Resnik.1993.Selection and Information: A Class-based Approach to Lexical Relationships.Ph.D. thesis, Philadelphia, PA, USA.

E. Shutova, L. Sun and A. Korhonen.2010.Metaphor Identification using Verb and Noun Clustering.In Proceedings of COLING 2010, Beijing, China.

E. Shutova.2010.Automatic metaphor interpretation as a paraphrasing task.In Proceedings of NAACL 2010, Los Angeles, USA.

P. Thibodeau and L. Boroditsky.2011.Metaphors We Think With: The Role of Metaphor in Reasoning.PLoS ONE, 6(2): e16782.

T. Veale and Y. Hao.2008.A fluid knowledge representation for understanding and generating creative metaphors.In Proceedings of COLING 2008, pages 945–952, Manchester, UK.


N

References IV

Y. Wilks.1978.Making preferences more active.Artificial Intelligence, 11(3):197–223.

Koon-Kiu Yan, Gang Fang, Nitin Bhardwaj, Roger P. Alexander, and MarkGerstein.2010.Comparing genomes to computer operating systems in terms of the topology and evolution of theirregulatory control networks.Proceedings of the National Academy of Sciences, 107(20):9186–9191.


N

Pictures come from: I

open.salon.com

www.deslive.com

pctechnotes.com


N

statistical modelling of metaphor -...

Documents