natural language processing for collective discourse...the opening sentence of madame bovary by...
TRANSCRIPT
![Page 1: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/1.jpg)
Natural Language Processing for the Health Sciences
Dragomir Radev MIDAS/SI/CSE
MIDAS Town Hall March 30, 2016
CLAIR
![Page 2: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/2.jpg)
• Today’s person is subjected to more information in a day than a person in the middle ages in a lifetime
• The search engine market is $94B a year – Feb 2016, New York Times
• Siri Gets 1 Billion requests a week – Jan 2016, USA Today, citing Apple
• PubMed adds 1.29 journal articles per minute! – 2010, Arif Jinha in Learned Publishing
• Users send out 168 Million emails every minute – 2015, go-globe.com
• Google indexes at least 48 Billion web pages – 2016, WorldWideWebSize.com
• The global healthcare NLP market is expected to grow from $1.10 billion in 2015 to $2.67 billion by 2020 – 2015, marketsandmarkets
Natural Language Text is Everywhere
2
![Page 3: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/3.jpg)
Examples of NLP Systems
3
![Page 4: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/4.jpg)
Natural Language Processing
Machine Learning
Database Systems
Linguistics Graph Theory
Artificial Intelligence
Human Computer Interaction
Stat/Data Science
Information Retrieval Humanities Social
Sciences Health
Sciences Education Finance Media
Connections Between NLP and Other Areas
4
![Page 5: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/5.jpg)
NLP for the Health Sciences • Goals
– improving clinical workflow – delivery of on-time information – reducing readmissions – patient satisfaction
• Information Extraction – medical papers – building structured databases from unstructured text
• Coding and documentation – clinical notes – electronic patient records
• Personal information management – information retrieval
• Telemedicine – dialogue systems
• Clinical decision support – summarization and survey generation
• Interoperability
5
![Page 6: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/6.jpg)
NLP methods • Sequence-based
– part of speech tagging – information extraction
• Tree-based – parsing – sentence simplification
• Graph-based – semi-supervised learning – clustering
• Semantic – reasoning – ontologies – similarity
6
![Page 7: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/7.jpg)
Dependency Parsing
girl omelet fork
the a the with
ate
ROOT
root
det det det case
nsubj dobj nmod
7
![Page 8: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/8.jpg)
Language Understanding • Semantic Analysis
Girl (g1) Omelet (o1) Fork (f1) Eating (e1) ^ Eater (e1,g1) ^ Eaten (e1,o1) ^ Instrument (e1,f1)
• World Knowledge Omelet (X) => Food (X)
• Inference Hungry (Z,t0) ^ Eater (e1,Z) ^ Eaten (e1,Y) ^ Time (e1,t1) ^ ^ Food (Y) ^ Precedes (t0,t1)=> ¬Hungry (Z,t1)
• Conclusion ¬Hungry (g1,t1)
8
![Page 9: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/9.jpg)
Vector Semantics
http://www.tensorflow.org/tutorials/word2vec/index.md
![Page 10: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/10.jpg)
Recursive NN for sentiment (Socher et al.)
![Page 11: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/11.jpg)
NLP to logical form interface (Dong and Lapata, 2016)
![Page 12: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/12.jpg)
12
![Page 13: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/13.jpg)
Query-based Summarization: Biased LexRank
∑∈
++=qw
wqwsw idftftfqsrel *)1log(*)1log()|( ,,
++
=w
w sfN5.0
1logidf
)|(),(
),()1()|(
)|()|( qvvvzsim
vssimdqzrel
qsreldqsvCv
CzCz
∑∑∑ ∈∈∈
−+=
vBAv Tdd ])1([ −+=
13
![Page 14: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/14.jpg)
CLAIR Summarization of Scientific Articles
• Goals: – Generating short summaries
of research articles – Generating short surveys of
research areas
• Approach: – Citing sentences
[Qazvinian et al., JAIR 2013]
Mohammed et al. (2009) suggested using citation information to generate surveys of scientific paradigms. Qazvinian and Radev (2008) proposed a method for summarizing scientific articles by building a similarity network of the citation sentences that cite the target paper, and then applying network analysis techniques to find a set of sentences that covers as much of the summarized paper facts as possible. We use this method as one of the baselines when we evaluate our approach. Qazvinian et al. (2010) proposed a citation-based summarization method that
14
![Page 15: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/15.jpg)
Graph-based Summary Generation
15
![Page 16: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/16.jpg)
Bipartite Graph Summarization
16
![Page 17: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/17.jpg)
Survey Generation (in progress) • Input: Word Sense Disambiguation • Output:
Word-sense disambiguation, a problem that once seemed out of reach for systems without a great deal of handcrafted linguistic and world knowledge, can now in some cases be done with high accuracy when all information is derived automatically from corpora (Brown, Lai, and Mercer 1991; Yarowsky 1992; Gale, Church, and Yarowsky 1992; Bruce and Wiebe 1994). WSD approaches can be classified as (a) knowledge-based approaches, which make use of linguistic knowledge, manually coded or extracted from lexical resources (Agirre and Rigau, 1996; Lesk 1986); (b) corpus-based approaches, which make use of shallow knowledge automatically acquired from corpus and statistical or machine learning algorithms to induce disambiguation models (Yarowsky, 1995; Schtze 1998); and (c) hybrid approaches, which mix characteristics from the two other approaches to automatically acquire disambiguation models from corpus supported by linguistic knowledge (Ng and Lee 1996; Stevenson and Wilks, 2001). Many corpus based methods have been proposed to deal with the sense disambiguation problem when given definition for each possible sense of a target word or a tagged corpus with the instances of each possible sense, e.g., supervised sense disambiguation (Leacock et al. , 1998), and semi-supervised sense disambiguation (Yarowsky, 1995).
[Jha, King, Coke, Radev, AAAI 2015] 17
![Page 18: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/18.jpg)
CLAIR Protein Interaction Extraction
• Goal: – Extracting protein interaction networks from the
literature – Measuring the associations between proteins and
diseases
18
![Page 19: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/19.jpg)
19
![Page 20: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/20.jpg)
Prostate Cancer Seed Genes
20
![Page 21: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/21.jpg)
Output
21
![Page 22: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/22.jpg)
![Page 23: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/23.jpg)
More about NLP
• talk at MIDAS – April 8 at 4 pm • midas.umich.edu
23
![Page 24: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/24.jpg)
Extra slides
24
![Page 25: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/25.jpg)
The NLP Pipeline
The girl ate the omelet with a fork.
DET N VBD DET N PRP DET N
25
![Page 26: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/26.jpg)
Constituent Parsing
S
NP VBD
ate
NP VP
PP
NP PRP
with
The girl
DET N
omelet the
DET N
a fork
DET N
26
![Page 27: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/27.jpg)
• Part of speech ambiguity – Ali won the first round – The cantaloupe is a round fruit
• Polysemy and word sense disambiguation – Melissa celebrated in a bar – Melissa passed the bar exam
• Noun phrase parsing – (Ann Arbor) News – Ann (Arbor News)
• Idiomatic expressions – This car cost an arm and a leg – I heard the news straight from the horse’s mouth
• World Knowledge – Every American has a mother – Every American has a president
• Similarity and semantic relatedness – A large airplane is parked on the runway – A jet is sitting on the tarmac
What Makes NLP Hard
27
![Page 28: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/28.jpg)
• Morphological similarity – respect-respectful
• Synonymy – talkative-chatty
• Ontological similarity – cat-tabby
• Distributional similarity – doctor-nurse
• Relational similarity – Aspirin-cold
• Sentence similarity – (paraphrases)
28
Similarity and Relatedness
![Page 29: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/29.jpg)
Applications of Sentence Similarity • Paraphrasing
– We were in class when the head-master came in, followed by a "new fellow," not wearing the school uniform, and a school servant carrying a large desk.
– We were in the study-hall when the headmaster entered, followed by a new boy not yet in school uniform and by the handy man carrying a large desk.
• Sentence simplification and retargeting – The best known feature of Jupiter is the Great Red Spot, a persistent anticyclonic storm that is larger than
Earth, located 22° south of the equator – One of the biggest features in Jupiter's atmosphere is the Great Red Spot
• Machine translation – No word yet on when the airline may start the service – Aucun mot encore sur le moment où la compagnie aérienne pourrait démarrer le service
• Entailment – Neiman Marcus Group, Inc. files for its long awaited IPO – Neiman Marcus goes public
• Presupposition, inference, and background knowledge – Stockbridge mayor Tim Thompson resigned. – There is a city called Stockbridge. Cities have mayors. Stockbridge has (had) a mayor. Tim Thompson is no
longer mayor of Stockbridge
29
![Page 30: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/30.jpg)
Human Judgments of Similarity
[Finkelstein et al. 2002]
http://wordvectors.org/suite.php
tiger cat 7.35 tiger tiger 10.00 book paper 7.46 computer keyboard 7.62 computer internet 7.58 plane car 5.77 train car 6.31 telephone communication 7.50 television radio 6.77 media radio 7.42 drug abuse 6.85 bread butter 6.19 cucumber potato 5.92
bar - Sense 1 barroom, bar, saloon, ginmill, taproom => room => area => structure, construction => artifact, artefact => object, physical object => entity, something bar - Sense 9 legal profession, bar, legal community => profession, community => occupation, vocation, occupational group => body => gathering, assemblage => social group => group, grouping
https://wordnet.princeton.edu/
[Miller and Fellbaum]
30
![Page 31: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/31.jpg)
Vector Semantics
• vector(‘king’) - vector(‘man’) + vector(‘woman’) ≈ vector(‘queen’) – [Word2Vec, Mikolov 2013]
31 Image courtesy of Daniel Jurafsky
![Page 32: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/32.jpg)
Acquiring Word Similarity from Collective Discourse
• A person rides a bicycle on concrete.
• A woman riding a bicycle on a large concrete area.
• A woman riding a mountain bike on a large concrete area.
• On a large, flat concrete area a woman in shorts rides a two-wheeled bike.
• The woman is riding a bicycle.
[Everingham & al. 2008] 32
![Page 33: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/33.jpg)
Graph-based Learning for NLP • Representation
– Lexical networks – vertices=text – edges=similarities
• Properties – Centrality – Diversity
• Learning tasks – Supervised (Classification) – Unsupervised (Clustering) – Semi-supervised
• Methods – Random Walks – Harmonic Functions
• Solid Mathematical Foundation – Scaleability
• Efficient Algorithms – Spectral Methods – Diffusion – Diversity – Evolution over time
33
![Page 34: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/34.jpg)
Similarity Functions
• Cosine similarity
• KL divergence
• Language model generation probability • Syntactic kernels
∑∑
∑
==
=
⋅
=n
ii
n
ii
n
iii
qd
qdqd
1
2
1
2
1),cos(
34
![Page 35: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/35.jpg)
Syntactic Kernels
[Özateş, Özgür, and Radev, LREC 2016] 35
![Page 36: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/36.jpg)
Modularity-based Clustering
36
![Page 37: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/37.jpg)
CLAIR Text Summarization
• Goal: – Reduce a long document
… or a set of related documents
– for shorter/easier reading … or for a mobile device (while preserving the most important information) (can be generic or query-based)
• Approach: – Lexical networks – Baseline: Centroid method, Radev et al. 2001
37
![Page 38: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/38.jpg)
Iraqi Vice President Taha Yassin Ramadan announced today, Sunday, that Iraq refuses to back down from its decision to stop cooperating with disarmament inspectors before its demands are met. Iraqi Vice president Taha Yassin Ramadan announced today, Thursday, that Iraq rejects cooperating with the United Nations except on the issue of lifting the blockade imposed upon it since the year 1990. Ramadan told reporters in Baghdad that "Iraq cannot deal positively with whoever represents the Security Council unless there was a clear stance on the issue of lifting the blockade off of it. Baghdad had decided late last October to completely cease cooperating with the inspectors of the United Nations Special Commission (UNSCOM), in charge of disarming Iraq's weapons, and whose work became very limited since the fifth of August, and announced it will not resume its cooperation with the Commission even if it were subjected to a military operation. The Russian Foreign Minister, Igor Ivanov, warned today, Wednesday against using force against Iraq, which will destroy, according to him, seven years of difficult diplomatic work and will complicate the regional situation in the area. Ivanov contended that carrying out air strikes against Iraq, who refuses to cooperate with the United Nations inspectors, ``will end the tremendous work achieved by the international group during the past seven years and will complicate the situation in the region.'' Nevertheless, Ivanov stressed that Baghdad must resume working with the Special Commission in charge of disarming the Iraqi weapons of mass destruction (UNSCOM). The Special Representative of the United Nations Secretary-General in Baghdad, Prakash Shah, announced today, Wednesday, after meeting with the Iraqi Deputy Prime Minister Tariq Aziz, that Iraq refuses to back down from its decision to cut off cooperation with the disarmament inspectors. British Prime Minister Tony Blair said today, Sunday, that the crisis between the international community and Iraq ``did not end'' and that Britain is still ``ready, prepared, and able to strike Iraq.'' In a gathering with the press held at the Prime Minister's office, Blair contended that the crisis with Iraq ``will not end until Iraq has absolutely and unconditionally respected its commitments'' towards the United Nations. A spokesman for Tony Blair had indicated that the British Prime Minister gave permission to British Air Force Tornado planes stationed in Kuwait to join the aerial bombardment against Iraq.
38
![Page 39: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/39.jpg)
39
![Page 40: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/40.jpg)
d4s1
d1s1
d3s2
d3s1
d2s3
d2s1
d2s2
d5s2 d5s3
d5s1
d3s3
Lexical Centrality (t=0.3)
40
![Page 41: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/41.jpg)
d4s1
d1s1
d3s2
d3s1
d2s3
d2s1
d2s2
d5s2 d5s3
d5s1
d3s3
Lexical Centrality (t=0.2)
41
![Page 42: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/42.jpg)
d4s1
d1s1
d3s2
d3s1
d2s3 d3s3
d2s1
d2s2
d5s2 d5s3
d5s1
Lexical Centrality (t=0.1)
d4s1
d3s2
d2s1
42
![Page 43: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/43.jpg)
Recurrence Relation
Can guarantee solution by allowing “jump” probability
d/N.
s 0.5
0.3
0.8 0.2
0.1 0.3
0.9
0.2 0.4
LexRank – Centrality in Text Graphs
43 [Erkan and Radev, JAIR 2006]
![Page 44: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/44.jpg)
DivRank: Diversity Reranking
[Mei, Guo and Radev, SIGKDD 2010] 44
![Page 45: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/45.jpg)
Objective:
By taking the partial derivative of fv, we get:
DivRank
45
![Page 46: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/46.jpg)
NIST DUC Evaluation
46
![Page 47: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/47.jpg)
Abstracts vs. citation summaries
47
![Page 49: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/49.jpg)
49
![Page 50: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/50.jpg)
(Ming-Wei 2006)
In the context of DPs, this edge based factorization method was proposed by Eisner (1996).
(McDonald 2005)
Eisner (1996) gave a generative model with a cubic parsing algorithm based on an edge factorization of trees.
(Lee 1997)
Eisner (1996) proposed an O(n3) parsing algorithm for PDG.
(Buchholz 2006)
If the parse has to be projective, Eisner’s bottom-up-span algorithm (Eisner, 1996) can be used for the search.
Example
Eisner, J. (1996) “Three New Probabilistic Models for Dependency Parsing.”
50
![Page 51: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/51.jpg)
51
![Page 52: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/52.jpg)
Generated Summary
52
![Page 53: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/53.jpg)
![Page 54: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/54.jpg)
![Page 55: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/55.jpg)
![Page 56: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/56.jpg)
![Page 57: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/57.jpg)
![Page 58: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/58.jpg)
CLAIR 9. Protein-Vaccine Networks
• Goal: – Identify gene interaction networks associated with
vaccines
![Page 59: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/59.jpg)
![Page 60: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/60.jpg)
![Page 61: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/61.jpg)
![Page 62: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/62.jpg)
![Page 63: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/63.jpg)
![Page 64: Natural Language Processing for Collective Discourse...The opening sentence of Madame Bovary by Gustave Flaubert \⠀㠀㔀㜀尩\爀屲Original \爀一漀甀猀 琀椀漀渀猀 氠ᤀ준琀甀搀攀Ⰰ](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f103c5d7e708231d4481c4d/html5/thumbnails/64.jpg)