towards a universal grammar for natural language processing · towards a universal grammar for...
TRANSCRIPT
Towards a Universal Grammar for Natural Language Processing
Joakim Nivre
Uppsala UniversityDepartment of Linguistics and Philology
Based on collaborative work with Filip Ginter, Yoav Goldberg, Jan Hajič, Chris Manning, Ryan McDonald, Natalia Silveira, Marie de Marneffe, Slav Petrov,
Sampo Pyysalo, Reut Tsarfaty, Daniel Zeman and many others
“In its substance, grammar is one and the same in all languages, even if it accidentally varies.”
“In its substance, grammar is one and the same in all languages, even if it accidentally varies.”
“In its substance, grammar is one and the same in all languages, even if it accidentally varies.”
Universal Grammar
Universal Grammar
• All human languages are species of a common genus
Universal Grammar
• All human languages are species of a common genus
• Language structure is constrained by a universal cause
Universal Grammar
• All human languages are species of a common genus
• Language structure is constrained by a universal cause
• There is order in the chaos of linguistic variation
Natural Language Processing
Natural Language Processing
• Linguistic diversity makes our life harderWhy 90% parsing accuracy for English but only 80% for Finnish?
Can we even compare the numbers?
Natural Language Processing
• Linguistic diversity makes our life harderWhy 90% parsing accuracy for English but only 80% for Finnish?
Can we even compare the numbers?
• Current NLP relies heavily on linguistic annotation:“In its substance, grammar is the same in all languages, even if the annotation accidentally varies.”
Natural Language Processing
• Linguistic diversity makes our life harderWhy 90% parsing accuracy for English but only 80% for Finnish?
Can we even compare the numbers?
• Current NLP relies heavily on linguistic annotation:“In its substance, grammar is the same in all languages, even if the annotation accidentally varies.”
• We need to bring some order into the chaos
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rader og møs
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
1
Language X
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rader og møs
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
1
Language X
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rader og møs
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
1
Language Y
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rader og møs
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
1
Language X
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rader og møs
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
1
Language YEn katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rader og møs
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
1
Language Z
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rader og møs
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
1
Language X
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rader og møs
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
1
Language YEn katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rader og møs
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
1
Language Z
Which languages are most closely related?
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rader og møs
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
1
Language X
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rader og møs
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
1
Language YEn katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rader og møs
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
1
Language Z
Which languages are most closely related?
1/5
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rader og møs
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
1
Language X
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rader og møs
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
1
Language YEn katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rader og møs
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
1
Language Z
Which languages are most closely related?
1/5
2/5
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rader og møs
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
1
Language X
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rader og møs
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
1
Language YEn katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rader og møs
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
1
Language Z
Which languages are most closely related?
1/5
2/5
2/5
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rader og møs
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
1
Language X
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rader og møs
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
1
Language YEn katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rader og møs
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
1
Language Z
Which languages are most closely related?
1/5
2/5
2/5
Swedish
Danish
English
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rotter og mus
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
Toutefois , les filles adorent les desserts .
toutefois , les fille adorer les dessert .
ADV PUNCT DET NOUN VERB DET NOUN PUNCT
Definite=Def Gender=Fem Number=Plur Definite=Def Gender=MascNumber=Plur Number=Plur Person=3 Number=Plur Number=Plur
Tense=Pres
advmod
punct
det nsubj
root
det
dobj
punct
1
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rotter og mus
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
Toutefois , les filles adorent les desserts .
toutefois , les fille adorer les dessert .
ADV PUNCT DET NOUN VERB DET NOUN PUNCT
Definite=Def Gender=Fem Number=Plur Definite=Def Gender=MascNumber=Plur Number=Plur Person=3 Number=Plur Number=Plur
Tense=Pres
advmod
punct
det nsubj
root
det
dobj
punct
1
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rotter og mus
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
Toutefois , les filles adorent les desserts .
toutefois , les fille adorer les dessert .
ADV PUNCT DET NOUN VERB DET NOUN PUNCT
Definite=Def Gender=Fem Number=Plur Definite=Def Gender=MascNumber=Plur Number=Plur Person=3 Number=Plur Number=Plur
Tense=Pres
advmod
punct
det nsubj
root
det
dobj
punct
1
Why is this a problem?
Why is this a problem?
• Hard to compare empirical results across languages
Why is this a problem?
• Hard to compare empirical results across languages
• Hard to evaluate cross-lingual learning
Why is this a problem?
• Hard to compare empirical results across languages
• Hard to evaluate cross-lingual learning
• Hard to build and maintain multilingual systems
Why is this a problem?
• Hard to compare empirical results across languages
• Hard to evaluate cross-lingual learning
• Hard to build and maintain multilingual systems
• Hard to make progress towards a universal parser
Universal Dependencieshttp://universaldependencies.org
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rotter og mus
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
Toutefois , les filles adorent les desserts .
ADV PUNCT DET NOUN VERB DET NOUN PUNCT
Definite=Def Gender=Fem Number=Plur Definite=Def Gender=MascNumber=Plur Number=Plur Person=3 Number=Plur Number=Plur
Tense=Pres
advmod
punct
det nsubj
root
det
dobj
punct
1
Universal Dependencieshttp://universaldependencies.org
Part-of-speech tags
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rotter og mus
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
Toutefois , les filles adorent les desserts .
ADV PUNCT DET NOUN VERB DET NOUN PUNCT
Definite=Def Gender=Fem Number=Plur Definite=Def Gender=MascNumber=Plur Number=Plur Person=3 Number=Plur Number=Plur
Tense=Pres
advmod
punct
det nsubj
root
det
dobj
punct
1
Universal Dependencieshttp://universaldependencies.org
Part-of-speech tags
Morphological features
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rotter og mus
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
Toutefois , les filles adorent les desserts .
ADV PUNCT DET NOUN VERB DET NOUN PUNCT
Definite=Def Gender=Fem Number=Plur Definite=Def Gender=MascNumber=Plur Number=Plur Person=3 Number=Plur Number=Plur
Tense=Pres
advmod
punct
det nsubj
root
det
dobj
punct
1
Universal Dependencieshttp://universaldependencies.org
Part-of-speech tags
Morphological features
Dependency relations
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rotter og mus
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
Toutefois , les filles adorent les desserts .
ADV PUNCT DET NOUN VERB DET NOUN PUNCT
Definite=Def Gender=Fem Number=Plur Definite=Def Gender=MascNumber=Plur Number=Plur Person=3 Number=Plur Number=Plur
Tense=Pres
advmod
punct
det nsubj
root
det
dobj
punct
1
Universal Dependencieshttp://universaldependencies.org
Stanford Dependencies
Universal Dependencieshttp://universaldependencies.org
Stanford Dependencies
Google UD
Universal Dependencieshttp://universaldependencies.org
Stanford Dependencies
Google UD
Stanford UD
Universal Dependencieshttp://universaldependencies.org
Stanford Dependencies
Google UD
Stanford UD
HamleDT
Universal Dependencieshttp://universaldependencies.org
Stanford Dependencies
Google UD
Stanford UD
HamleDT
Interset
Universal Dependencieshttp://universaldependencies.org
Stanford Dependencies
Google UD
Stanford UD
HamleDT
Interset
Google universal tags
Universal Dependencies
Universal Dependencieshttp://universaldependencies.org
Universal Dependencies
• Milestones:• Kick-off meeting at EACL in Gothenburg, April 2014• Release of annotation guidelines, Version 1, October 2014• Release of treebanks for 10 languages, January 2015• Release of treebanks for 18 languages, May 2015• Release of treebanks for 33 languages, November 2015
• Open community effort – anyone can contribute!
Universal Dependencieshttp://universaldependencies.org
Goals and Requirements
Goals and Requirements
• Cross-linguistically consistent grammatical annotation
Goals and Requirements
• Cross-linguistically consistent grammatical annotation
• Support multilingual research and development in NLP
Goals and Requirements
• Cross-linguistically consistent grammatical annotation
• Support multilingual research and development in NLP
• Based on common usage and existing de facto standards
Design Principles
Design Principles
• Dependency• Widely used in practical NLP systems
• Available in treebanks for many languages
Design Principles
• Dependency• Widely used in practical NLP systems
• Available in treebanks for many languages
• Lexicalism• Basic annotation units are words – syntactic words
• Words have morphological properties
• Words enter into syntactic relations
Golden Rules
Golden Rules
Maximize parallelism• Don’t annotate the same thing in different ways
• Don’t make different things look the same
Golden Rules
Maximize parallelism• Don’t annotate the same thing in different ways
• Don’t make different things look the same
But don’t overdo it• Don’t annotate things that are not there
• Languages select from a universal pool of categories
• Allow language-specific extensions
Morphology
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rotter og mus
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
Toutefois , les filles adorent les desserts .
toutefois , les fille adorer les dessert .
ADV PUNCT DET NOUN VERB DET NOUN PUNCT
Definite=Def Gender=Fem Number=Plur Definite=Def Gender=MascNumber=Plur Number=Plur Person=3 Number=Plur Number=Plur
Tense=Pres
advmod
punct
det nsubj
root
det
dobj
punct
1
• Lemma representing the semantic content of the word
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rotter og mus
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
Toutefois , les filles adorent les desserts .
toutefois , le fille adorer le dessert .
ADV PUNCT DET NOUN VERB DET NOUN PUNCT
Definite=Def Gender=Fem Number=Plur Definite=Def Gender=MascNumber=Plur Number=Plur Person=3 Number=Plur Number=Plur
Tense=Pres
advmod
punct
det nsubj
root
det
dobj
punct
Toutefois , les filles adorent les desserts .
toutefois , le fille adorer les dessert .
ADV PUNCT DET NOUN VERB DET NOUN PUNCT
Definite=Def Gender=Fem Number=Plur Definite=Def Gender=MascNumber=Plur Number=Plur Person=3 Number=Plur Number=Plur
Tense=Pres
advmod
punct
det nsubj
root
det
dobj
punct
1
Morphology
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rotter og mus
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
Toutefois , les filles adorent les desserts .
toutefois , les fille adorer les dessert .
ADV PUNCT DET NOUN VERB DET NOUN PUNCT
Definite=Def Gender=Fem Number=Plur Definite=Def Gender=MascNumber=Plur Number=Plur Person=3 Number=Plur Number=Plur
Tense=Pres
advmod
punct
det nsubj
root
det
dobj
punct
1
• Lemma representing the semantic content of the word
• Part-of-speech tag representing the abstract lexical category associated with the word
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rotter og mus
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
Toutefois , les filles adorent les desserts .
toutefois , le fille adorer le dessert .
ADV PUNCT DET NOUN VERB DET NOUN PUNCT
Definite=Def Gender=Fem Number=Plur Definite=Def Gender=MascNumber=Plur Number=Plur Person=3 Number=Plur Number=Plur
Tense=Pres
advmod
punct
det nsubj
root
det
dobj
punct
Toutefois , les filles adorent les desserts .
toutefois , le fille adorer les dessert .
ADV PUNCT DET NOUN VERB DET NOUN PUNCT
Definite=Def Gender=Fem Number=Plur Definite=Def Gender=MascNumber=Plur Number=Plur Person=3 Number=Plur Number=Plur
Tense=Pres
advmod
punct
det nsubj
root
det
dobj
punct
1
Morphology
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rotter og mus
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
Toutefois , les filles adorent les desserts .
toutefois , les fille adorer les dessert .
ADV PUNCT DET NOUN VERB DET NOUN PUNCT
Definite=Def Gender=Fem Number=Plur Definite=Def Gender=MascNumber=Plur Number=Plur Person=3 Number=Plur Number=Plur
Tense=Pres
advmod
punct
det nsubj
root
det
dobj
punct
1
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rotter og mus
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
Toutefois , les filles adorent les desserts .
toutefois , les fille adorer les dessert .
ADV PUNCT DET NOUN VERB DET NOUN PUNCT
Definite=Def Gender=Fem Number=Plur Definite=Def Gender=MascNumber=Plur Number=Plur Person=3 Number=Plur Number=Plur
Tense=Pres
advmod
punct
det nsubj
root
det
dobj
punct
1
• Lemma representing the semantic content of the word
• Part-of-speech tag representing the abstract lexical category associated with the word
• Features representing lexical and grammatical properties associated with the lemma or the particular word form
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rotter og mus
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
Toutefois , les filles adorent les desserts .
toutefois , le fille adorer le dessert .
ADV PUNCT DET NOUN VERB DET NOUN PUNCT
Definite=Def Gender=Fem Number=Plur Definite=Def Gender=MascNumber=Plur Number=Plur Person=3 Number=Plur Number=Plur
Tense=Pres
advmod
punct
det nsubj
root
det
dobj
punct
Toutefois , les filles adorent les desserts .
toutefois , le fille adorer les dessert .
ADV PUNCT DET NOUN VERB DET NOUN PUNCT
Definite=Def Gender=Fem Number=Plur Definite=Def Gender=MascNumber=Plur Number=Plur Person=3 Number=Plur Number=Plur
Tense=Pres
advmod
punct
det nsubj
root
det
dobj
punct
1
Morphology
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rotter og mus
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
Toutefois , les filles adorent les desserts .
toutefois , les fille adorer les dessert .
ADV PUNCT DET NOUN VERB DET NOUN PUNCT
Definite=Def Gender=Fem Number=Plur Definite=Def Gender=MascNumber=Plur Number=Plur Person=3 Number=Plur Number=Plur
Tense=Pres
advmod
punct
det nsubj
root
det
dobj
punct
1
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rotter og mus
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
Toutefois , les filles adorent les desserts .
toutefois , les fille adorer les dessert .
ADV PUNCT DET NOUN VERB DET NOUN PUNCT
Definite=Def Gender=Fem Number=Plur Definite=Def Gender=MascNumber=Plur Number=Plur Person=3 Number=Plur Number=Plur
Tense=Pres
advmod
punct
det nsubj
root
det
dobj
punct
1
En katt jagar rattor och moss
det nsubj conj
dobj
conj
En kat jager rotter og mus
nsubj
? dobj cc conj
A cat chases rats and mice
det nsubj dobj cc
conj
Toutefois , les filles adorent les desserts .
toutefois , les fille adorer les dessert .
ADV PUNCT DET NOUN VERB DET NOUN PUNCT
Definite=Def Gender=Fem Number=Plur Definite=Def Gender=MascNumber=Plur Number=Plur Person=3 Number=Plur Number=Plur
Tense=Pres
advmod
punct
det nsubj
root
det
dobj
punct
1
• Taxonomy of 17 universal part-of-speech tags, based on the Google Universal Tagset (Petrov et al., 2012)
• All languages use the same inventory, but not all tags have to be used by all languages
Part-of-Speech TagsOpen Closed OtherADJ ADP PUNCT
ADV AUX SYM
INTJ CONJ X
NOUN DET
PROPN NUM
VERB PART
PRON
SCONJ
Features
• Standardized inventory of morphological features, based on the Interset system (Zeman, 2008)
• Languages select relevant features and can add language-specific features or values with documentation
LexicalInflectional Nominal
Inflectional Verbal
PronType Gender VerbForm
NumType Animacy Mood
Poss Number Tense
Reflex Case Aspect
Definite Voice
Degree Person
Negative
Syntax
The cat could have chased all the dogs down the street .
DET NOUN AUX AUX VERB DET DET NOUN ADP DET NOUN PUNCT
det
nsubj
aux
aux
root
det
det
dobj
case
det
nmod
punct
The cat could have chased all the dogs down the street .
DET NOUN AUX AUX VERB DET DET NOUN ADP DET NOUN PUNCT
det
nsubj
aux
aux
root
det
det
dobj
case
det
nmod
The cat could have chased all the dogs down the street .
DET NOUN AUX AUX VERB DET DET NOUN ADP DET NOUN PUNCT
nsubj
root
dobj
nmod
2
Syntax
• Content words are related by dependency relations
The cat could have chased all the dogs down the street .
DET NOUN AUX AUX VERB DET DET NOUN ADP DET NOUN PUNCT
det
nsubj
aux
aux
root
det
det
dobj
case
det
nmod
punct
The cat could have chased all the dogs down the street .
DET NOUN AUX AUX VERB DET DET NOUN ADP DET NOUN PUNCT
det
nsubj
aux
aux
root
det
det
dobj
case
det
nmod
The cat could have chased all the dogs down the street .
DET NOUN AUX AUX VERB DET DET NOUN ADP DET NOUN PUNCT
nsubj
root
dobj
nmod
2
Syntax
• Content words are related by dependency relations
• Function words attach to the content word they modify
The cat could have chased all the dogs down the street .
DET NOUN AUX AUX VERB DET DET NOUN ADP DET NOUN PUNCT
det
nsubj
aux
aux
root
det
det
dobj
case
det
nmod
punct
The cat could have chased all the dogs down the street .
DET NOUN AUX AUX VERB DET DET NOUN ADP DET NOUN PUNCT
det
nsubj
aux
aux
root
det
det
dobj
case
det
nmod
The cat could have chased all the dogs down the street .
DET NOUN AUX AUX VERB DET DET NOUN ADP DET NOUN PUNCT
nsubj
root
dobj
nmod
2
Syntax
• Content words are related by dependency relations
• Function words attach to the content word they modify
• Punctuation attach to head of phrase or clause
The cat could have chased all the dogs down the street .
DET NOUN AUX AUX VERB DET DET NOUN ADP DET NOUN PUNCT
det
nsubj
aux
aux
root
det
det
dobj
case
det
nmod
punct
The cat could have chased all the dogs down the street .
DET NOUN AUX AUX VERB DET DET NOUN ADP DET NOUN PUNCT
det
nsubj
aux
aux
root
det
det
dobj
case
det
nmod
The cat could have chased all the dogs down the street .
DET NOUN AUX AUX VERB DET DET NOUN ADP DET NOUN PUNCT
nsubj
root
dobj
nmod
2
The dog was chased by the cat .DET NOUN AUX VERB ADP DET NOUN PUNCT
det
nsubjpass
auxpass
root
case
det
nmod
punct
Hunden jagades av katten .NOUN VERB ADP NOUN PUNCT
Definite=Def Voice=Pass Definite=Def
nsubjpass
root
case
nmod
punct
The dog was chased by the cat .DET NOUN AUX VERB ADP DET NOUN PUNCT
nsubjpass
root
nmod
punct
Hunden jagades av katten .NOUN VERB ADP NOUN PUNCT
Definite=Def Voice=Pass Definite=Def
nsubjpass
root
nmod
punct
The dog was chased by the cat .DET NOUN AUX VERB ADP DET NOUN PUNCT
det
nsubjpass
root
det
nmod
punct
Hunden jagades av katten .NOUN VERB ADP NOUN PUNCT
Definite=Def Voice=Pass Definite=Def
nsubjpass
root
nmod
punct
3
The dog was chased by the cat .DET NOUN AUX VERB ADP DET NOUN PUNCT
det
nsubjpass
auxpass
root
case
det
nmod
punct
Hunden jagades av katten .NOUN VERB ADP NOUN PUNCT
Definite=Def Voice=Pass Definite=Def
nsubjpass
root
case
nmod
punct
The dog was chased by the cat .DET NOUN AUX VERB ADP DET NOUN PUNCT
nsubjpass
root
nmod
punct
Hunden jagades av katten .NOUN VERB ADP NOUN PUNCT
Definite=Def Voice=Pass Definite=Def
nsubjpass
root
nmod
punct
The dog was chased by the cat .DET NOUN AUX VERB ADP DET NOUN PUNCT
det
nsubjpass
root
det
nmod
punct
Hunden jagades av katten .NOUN VERB ADP NOUN PUNCT
Definite=Def Voice=Pass Definite=Def
nsubjpass
root
nmod
punct
3
The dog was chased by the cat .DET NOUN AUX VERB ADP DET NOUN PUNCT
det
nsubjpass
auxpass
root
det
nmod
punct
Hunden jagades av katten .NOUN VERB ADP NOUN PUNCT
Definite=Def Voice=Pass Definite=Def
nsubjpass
root
nmod
punct
4
The dog was chased by the cat .DET NOUN AUX VERB ADP DET NOUN PUNCT
det
nsubjpass
auxpass
root
case
det
nmod
punct
Hunden jagades av katten .NOUN VERB ADP NOUN PUNCT
Definite=Def Voice=Pass Definite=Def
nsubjpass
root
case
nmod
punct
The dog was chased by the cat .DET NOUN AUX VERB ADP DET NOUN PUNCT
nsubjpass
root
nmod
punct
Hunden jagades av katten .NOUN VERB ADP NOUN PUNCT
Definite=Def Voice=Pass Definite=Def
nsubjpass
root
nmod
punct
The dog was chased by the cat .DET NOUN AUX VERB ADP DET NOUN PUNCT
det
nsubjpass
root
det
nmod
punct
Hunden jagades av katten .NOUN VERB ADP NOUN PUNCT
Definite=Def Voice=Pass Definite=Def
nsubjpass
root
nmod
punct
3
Dependency Relations
Dependency Relations
• Taxonomy of 40 universal grammatical relations, broadly attested in language typology (de Marneffe et al., 2014)• Language-specific subtypes may be added
Dependency Relations
• Taxonomy of 40 universal grammatical relations, broadly attested in language typology (de Marneffe et al., 2014)• Language-specific subtypes may be added
• Organizing principles• Three types of structures: nominals, clauses, modifiers
• Core arguments vs. other dependents (not complements vs. adjuncts)
Dependents of Clausal Predicates
Nominal Clausal Other
Core
nsubjnsubjpass
dobjiobj
csubjcsubjpassccompxcomp
Non-Core
nmodvocative
discourseexpl
advcl
advmodnegaux
auxpasscop
markpunct
Mary was quietly reading a book in the garden .PROPN AUX ADV VERB DET NOUN ADP DET NOUN PUNCT
nsubj
aux
advmod
root
det
dobj case
det
nmod
punct
If you are sick , you should not exercise .SCONJ PRON AUX ADJ PUNCT PRON AUX ADV VERB PUNCT
mark
nsubj
cop
advcl
punct
nsubj
aux
neg
root
punct
Peter thought that he should stop smoking .PROPN VERB SCONJ PRON AUX VERB VERB PUNCT
nsubj
root
mark
nsubj
aux
ccomp
xcomp
punct
Cairo , the lovely capital of EgyptPROPN PUNCT DET ADJ NOUN ADP PROPN
mark
det
amod
appos
case
nmod
Hewey , Dewey and LouiePROPN PUNCT PROPN CONJ PROPN
punct
conj
cc
conj
5
Mary was quietly reading a book in the garden .PROPN AUX ADV VERB DET NOUN ADP DET NOUN PUNCT
nsubj
aux
advmod
root
det
dobj case
det
nmod
punct
If you are sick , you should not exercise .SCONJ PRON AUX ADJ PUNCT PRON AUX ADV VERB PUNCT
mark
nsubj
cop
advcl
punct
nsubj
aux
neg
root
punct
Peter thought that he should stop smoking .PROPN VERB SCONJ PRON AUX VERB VERB PUNCT
nsubj
root
mark
nsubj
aux
ccomp
xcomp
punct
Cairo , the lovely capital of EgyptPROPN PUNCT DET ADJ NOUN ADP PROPN
mark
det
amod
appos
case
nmod
Hewey , Dewey and LouiePROPN PUNCT PROPN CONJ PROPN
punct
conj
cc
conj
5
Mary was quietly reading a book in the garden .PROPN AUX ADV VERB DET NOUN ADP DET NOUN PUNCT
nsubj
aux
advmod
root
det
dobj case
det
nmod
punct
If you are sick , you should not exercise .SCONJ PRON AUX ADJ PUNCT PRON AUX ADV VERB PUNCT
mark
nsubj
cop
advcl
punct
nsubj
aux
neg
root
punct
Peter thought that he should stop smoking .PROPN VERB SCONJ PRON AUX VERB VERB PUNCT
nsubj
root
mark
nsubj
aux
ccomp
xcomp
punct
Cairo , the lovely capital of EgyptPROPN PUNCT DET ADJ NOUN ADP PROPN
mark
det
amod
appos
case
nmod
Hewey , Dewey and LouiePROPN PUNCT PROPN CONJ PROPN
punct
conj
cc
conj
5
Mary was quietly reading a book in the garden .PROPN AUX ADV VERB DET NOUN ADP DET NOUN PUNCT
nsubj
aux
advmod
root
det
dobj case
det
nmod
punct
If you are sick , you should not exercise .SCONJ PRON AUX ADJ PUNCT PRON AUX ADV VERB PUNCT
mark
nsubj
cop
advcl
punct
nsubj
aux
neg
root
punct
Peter thought that he should stop smoking .PROPN VERB SCONJ PRON AUX VERB VERB PUNCT
nsubj
root
mark
nsubj
aux
ccomp
xcomp
punct
Cairo , the lovely capital of EgyptPROPN PUNCT DET ADJ NOUN ADP PROPN
mark
det
amod
appos
case
nmod
Hewey , Dewey and LouiePROPN PUNCT PROPN CONJ PROPN
punct
conj
cc
conj
5
Mary was quietly reading a book in the garden .PROPN AUX ADV VERB DET NOUN ADP DET NOUN PUNCT
nsubj
aux
advmod
root
det
dobj case
det
nmod
punct
If you are sick , you should not exercise .SCONJ PRON AUX ADJ PUNCT PRON AUX ADV VERB PUNCT
mark
nsubj
cop
advcl
punct
nsubj
aux
neg
root
punct
Peter thought that he should stop smoking .PROPN VERB SCONJ PRON AUX VERB VERB PUNCT
nsubj
root
mark
nsubj
aux
ccomp
xcomp
punct
Cairo , the lovely capital of EgyptPROPN PUNCT DET ADJ NOUN ADP PROPN
mark
det
amod
appos
case
nmod
Hewey , Dewey and LouiePROPN PUNCT PROPN CONJ PROPN
punct
conj
cc
conj
5
Mary was quietly reading a book in the garden .PROPN AUX ADV VERB DET NOUN ADP DET NOUN PUNCT
nsubj
aux
advmod
root
det
dobj case
det
nmod
punct
If you are sick , you should not exercise .SCONJ PRON AUX ADJ PUNCT PRON AUX ADV VERB PUNCT
mark
nsubj
cop
advcl
punct
nsubj
aux
neg
root
punct
Peter thought that he should stop smoking .PROPN VERB SCONJ PRON AUX VERB VERB PUNCT
nsubj
root
mark
nsubj
aux
ccomp
xcomp
punct
Cairo , the lovely capital of EgyptPROPN PUNCT DET ADJ NOUN ADP PROPN
mark
det
amod
appos
case
nmod
Hewey , Dewey and LouiePROPN PUNCT PROPN CONJ PROPN
punct
conj
cc
conj
5
Mary was quietly reading a book in the garden .PROPN AUX ADV VERB DET NOUN ADP DET NOUN PUNCT
nsubj
aux
advmod
root
det
dobj case
det
nmod
punct
If you are sick , you should not exercise .SCONJ PRON AUX ADJ PUNCT PRON AUX ADV VERB PUNCT
mark
nsubj
cop
advcl
punct
nsubj
aux
neg
root
punct
Peter thought that he should stop smoking .PROPN VERB SCONJ PRON AUX VERB VERB PUNCT
nsubj
root
mark
nsubj
aux
ccomp
xcomp
punct
Cairo , the lovely capital of EgyptPROPN PUNCT DET ADJ NOUN ADP PROPN
mark
det
amod
appos
case
nmod
Helsinki , the lovely capital of FinlandPROPN PUNCT DET ADJ NOUN ADP PROPN
mark
det
amod
appos
case
nmod
Huey , Dewey and LouiePROPN PUNCT PROPN CONJ PROPN
punct
conj
cc
conj
5
Dependents of Nominals
Nominal Clausal Other
nummodapposnmod
aclamoddetcase
• Coordinate structures are headed by the first conjunct• Subsequent conjuncts depend on it via the conj relation
• Conjunctions depend on it via the cc relation
• Punctuation marks depend on it via the punct relation
Mary was quietly reading a book in the garden .PROPN AUX ADV VERB DET NOUN ADP DET NOUN PUNCT
nsubj
aux
advmod
root
det
dobj case
det
nmod
punct
If you are sick , you should not exercise .SCONJ PRON AUX ADJ PUNCT PRON AUX ADV VERB PUNCT
mark
nsubj
cop
advcl
punct
nsubj
aux
neg
root
punct
Peter thought that he should stop smoking .PROPN VERB SCONJ PRON AUX VERB VERB PUNCT
nsubj
root
mark
nsubj
aux
ccomp
xcomp
punct
Cairo , the lovely capital of EgyptPROPN PUNCT DET ADJ NOUN ADP PROPN
mark
det
amod
appos
case
nmod
Huey , Dewey and LouiePROPN PUNCT PROPN CONJ PROPN
punct
conj
cc
conj
5
CoordinationCoordination
conj
cc
(punct)
Multiword Expressions
• UD annotation does not permit “words with spaces”• Multiword expressions are analysed using special relations
• The mwe, name and goeswith relations are always head-initial
• The compound relation reflects the internal structure
Relation Examples
mwe in spite of, as well as, ad hoc
name Roger Bacon, Carl XVI Gustaf, New York
compound phone book, four thousand, dress up
goeswith notwith standing, with out
Other Relations
Relation Explanation
parataxis Loosely linked clauses of same rank
list Lists without syntactic structure
remnant Orphans in ellipsis linked to parallel elements
reparandum Disfluency linked to (speech) repair
foreign Elements within opaque stretches of code switching
dep Unspecified dependency
root Syntactically independent element of clause/phrase
Language-Specific Relations• Language-specific relations are subtypes of universal
relations added to capture important phenomena
• Subtyping permits us to “back off” to universal relations
Relation Explanation
acl:relcl Relative clause
compound:prt Verb particle (dress up)
nmod:poss Genitive nominal (Mary’s book)
nmod:agent Agent in passive (saved by the bell)
cc:preconj Preconjunction (both … and)
det:predet Predeterminer (all those …)
Where are we now?
Where are we now?• Universal Dependencies, Version 1
• Guidelines released October 2014
• Latest treebank release November 2015 (v1.2):Ancient Greek, Arabic, Basque, Bulgarian, Croatian, Czech, Danish, Dutch, English, Estonian, Finnish, French, German, Gothic, Greek, Hebrew, Hindi, Hungarian, Indonesian, Irish, Italian, Japanese, Latin, Norwegian, Old Church Slavonic, Persian, Polish, Portuguese, Romanian, Slovenian, Spanish, Swedish, Tamil
Where are we now?• Universal Dependencies, Version 1
• Guidelines released October 2014
• Latest treebank release November 2015 (v1.2):Ancient Greek, Arabic, Basque, Bulgarian, Croatian, Czech, Danish, Dutch, English, Estonian, Finnish, French, German, Gothic, Greek, Hebrew, Hindi, Hungarian, Indonesian, Irish, Italian, Japanese, Latin, Norwegian, Old Church Slavonic, Persian, Polish, Portuguese, Romanian, Slovenian, Spanish, Swedish, Tamil
• Future plans:• New releases every six months (May, November)
• Revision of guidelines as needed
Where are we now?• Universal Dependencies, Version 1
• Guidelines released October 2014
• Latest treebank release November 2015 (v1.2):Ancient Greek, Arabic, Basque, Bulgarian, Croatian, Czech, Danish, Dutch, English, Estonian, Finnish, French, German, Gothic, Greek, Hebrew, Hindi, Hungarian, Indonesian, Irish, Italian, Japanese, Latin, Norwegian, Old Church Slavonic, Persian, Polish, Portuguese, Romanian, Slovenian, Spanish, Swedish, Tamil
• Future plans:• New releases every six months (May, November)
• Revision of guidelines as needed
• Have a look at http://universaldependencies.org
So what exactly is UD?
So what exactly is UD?• A new linguistic theory?
Not at all, but we like to think it is informed by linguistic theory and potentially useful also for linguistic studies
So what exactly is UD?• A new linguistic theory?
Not at all, but we like to think it is informed by linguistic theory and potentially useful also for linguistic studies
• A better parsing framework?Probably not, since parsers seem to prefer function words as heads so we may have to tweak the representations for parsing
So what exactly is UD?• A new linguistic theory?
Not at all, but we like to think it is informed by linguistic theory and potentially useful also for linguistic studies
• A better parsing framework?Probably not, since parsers seem to prefer function words as heads so we may have to tweak the representations for parsing
• The ultimate annotation scheme?Not quite, more like a lingua franca for treebank developers and definitely useful for some annotation projects
So what exactly is UD?• A new linguistic theory?
Not at all, but we like to think it is informed by linguistic theory and potentially useful also for linguistic studies
• A better parsing framework?Probably not, since parsers seem to prefer function words as heads so we may have to tweak the representations for parsing
• The ultimate annotation scheme?Not quite, more like a lingua franca for treebank developers and definitely useful for some annotation projects
• A universal grammar?Not in the Chomskyan sense, but hopefully in the more practical sense of facilitating multilingual NLP by bringing a little order into the chaos
So what exactly is UD?• A new linguistic theory?
Not at all, but we like to think it is informed by linguistic theory and potentially useful also for linguistic studies
• A better parsing framework?Probably not, since parsers seem to prefer function words as heads so we may have to tweak the representations for parsing
• The ultimate annotation scheme?Not quite, more like a lingua franca for treebank developers and definitely useful for some annotation projects
• A universal grammar?Not in the Chomskyan sense, but hopefully in the more practical sense of facilitating multilingual NLP by bringing a little order into the chaos
Well, who knows?
Acknowledgments• Core UD group: Filip Ginter, Yoav Goldberg, Jan Hajič, Chris Manning, Ryan
McDonald, Natalia Silveira, Marie de Marneffe, Slav Petrov, Sampo Pyysalo, Reut Tsarfaty, Dan Zeman
• UD contributors: Željko Agić, Riyaz Ahmad, Maria Jesus Aranzabe, Masayuki Asahara, Aitziber Atutxa, Cristina Bosco, Giuseppe G. A. Celano, Jinho Choi, Çağrı Çöltekin, Kaja Dobrovoljc, Timothy Dozat, Binyam Ephrem, Tomaž Erjavec, Richárd Farkas, Jennifer Foster, Koldo Gojenola, Iakes Goenaga, Bruno Guillaume, Nizar Habash, Dag Haug, Anders Trærup Johannsen, Hiroshi Kanayama, Jenna Kanerva, Simon Krek, Juha Kuokkala, Veronika Laippala, Alessandro Lenci, Krister Lindén, Nikola Ljubešić, Olga Lyashevskaya, Teresa Lynn, Aibek Makazhanov, Catalina Maranduc, Héctor Martínez Alonso, Anna Missilä, Verginica Mititelu, Yusuke Miyao, Simonetta Montemagni, Shinsuke Mori, Hanna Nurmi, Petya Osenova, Lilja Øvrelid, Elena Pascual, Marco Passarotti, Jussi Piitulainen, Barbara Plank, Prokopis Prokopidis, Loganathan Ramasamy, Wolfgang Seeker, Mojgan Seraji, Maria Simi, Kiril Simov, Arne Skjæerholt, Aaron Smith, Jan Štěpánek,Takaaki Tanaka, Francis Tyers, Sumire Uematsu, Veronika Vincze, Rob Voigt, Jonathan Washington