THE BIG PICTURE

Basic Assumptions

• Linguistics is the empirical science that studies language (or linguistic behavior)

• Linguistics proposes theories (models) that can be verified or falsified against linguistic data

• Computational linguistics is the branch of linguistics that uses computational models

• Natural language processing (NLP) is the engineering equivalent of computational linguistics (as civil engineering is to physics)

The Big PictureEmpirical Matter

Formalisms•Data structures•Formalisms•Algorithms•Distributional Models

Maud expects there to be a riot*Teri promised there to be a riotMaud expects the shit to hit the fan*Teri promised the shit to hit the

Linguistic Theory

??

?

?

Empirical Matter: Language and/or Linguistic Behavior

Grammaticality Judgments

CorporaPsycholinguistic Experiments

BrainScans

Maud expects there to be a riot*Teri promised there to be a riotMaud expects the shit to hit the fan*Teri promised the shit to hit the fan

Empirical Matter: Language and/or Linguistic Behavior

Grammaticality Judgments

CorporaPsycholinguistic Experiments

BrainScans

Maud expects there to be a riot*Teri promised there to be a riotMaud expects the shit to hit the fan*Teri promised the shit to hit the fan

(?)

Underlying Empirical Object of Study

• What is linguistics really “about”?o The brain (cognitive science)o Language as an abstract structure

(structuralism)

The Big PictureEmpirical Matter

Formalisms•Data structures•Formalisms•Algorithms•Distributional Models

Maud expects there to be a riot*Teri promised there to be a riotMaud expects the shit to hit the fan*Teri promised the shit to hit the

Linguistic Theory

??

?

or

Mathematical Formalisms (1)

Data Structures

• Phrase-structure trees

• Dependency trees

• Dags• …

Formalisms• CFG• TAG• Dependency

grammars• Unification

grammars• …

Algorithms• Chart parsing

• Bottom-up• Top-down• …

• Deterministic parsing• LR• …

• Generation• …

Distributional Models

• Probabilistic CFG

• Probabilistic TAG

• …

Mathematical Formalisms (2)

• Exist in formal computer science, mathematics, statistics…

• Exist independently of natural language• Do not on their own attempt to model or

explain natural language• Do not on their own succeed in modeling

or explaining natural language

The Big PictureEmpirical Matter

Formalisms•Data structures•Formalisms•Algorithms•Distributional Models

Maud expects there to be a riot*Teri promised there to be a riotMaud expects the shit to hit the fan*Teri promised the shit to hit the

Linguistic Theory

??

or

Linguistic Theory

• Phonetics: articulated sounds• Phonology: how do sounds form minimal

meaning units (morphemes)?• Morphology: how do morphemes form

words?• Syntax: how do words form utterances?• Semantics: what is the meaning of

utterances?• Pragmatics: in what context do we use

which utterance?

Goal of Syntactic Theory (1)

• Goal (version 1): formulate theory of how words form utterances (sentences, in written language)

• Goal (version 2): formulate theory of how words in linear sequence combine to form utterances

• Utterance represented by non-linear structure (e.g., a tree)

Goal of Syntactic Theory (2)

• Goal (version 3): formulate theory of how words in linear sequence correspond to structures

• Assumption: semantics interprets this structure as meaning -- in particular, predicate-argument structure

• Goal (version 4): formulate theory of how words in linear sequence correspond to predicate-argument structures

Goal of Syntactic Theory (3)

• What is predicate-argument structure?

seem

like

applesJohn

John seems to like apples

• Deep dependency-like structure!

Goal of Syntactic Theory (4)

• So what role does phrase-structure play?• “Augmented” representation of linear

orderS

S

Vi

Vi

NP

NP

NP

V

V

Johnseems

like apples

t

Goal of Syntactic Theory (5)

• Goal (version 5, final for us): formulate theory of how phrase structure of sentences relates to their deep dependency

• Goal (version 5 – dependency theories): formulate theory of how surface dependency of sentences relates to their deep dependency

The Big PictureEmpirical Matter

Formalisms

•Data structures•Formalisms•Algorithms•Distributional Models

Maud expects there to be a riot*Teri promised there to be a riotMaud expects the shit to hit the fan*Teri promised the shit to hit the

or

Linguistic Theory

?

theory of

Components of a Syntactic Theory (1)

• Definition of surface representationo Choice of data structure/formalism/…o List of node labels, rules, etc.

• Definition of deep representationo Choice of data structure/formalism/…o List of node labels, rules, etc.

• Description of correspondenceo Choice of formal mechanismo List of rules (?)

Components of a Syntactic Theory (2)

• Formal Frameworko Definition of surface representation

Choice of data structure/formalism/…o Definition of deep representation

Choice of data structure/formalism/…o Description of correspondence

Choice of formal mechanism

• Linguistic Contento Definition of surface representation

List of node labels, rules, etc.o Definition of deep representation

List of node labels, rules, etc.o Description of correspondence

List of rules (?)

The Big PictureEmpirical Matter

Formalisms

•Data structures•Formalisms•Algorithms•Distributional Models

Maud expects there to be a riot*Teri promised there to be a riotMaud expects the shit to hit the fan*Teri promised the shit to hit the

or

Linguistic Theory

Content• Surface representation (eg, ps)• Deep representation (eg, dep)• Correspondence

uses

theory of

Note on Competence vs

Performance

• Performance: human sentence processing o Language useo Interacts with other parts of cognition: memory,

emotions, etco Studied in psychology, data from experiments

• Competence: human knowledge of syntax that allows performanceo What we have been and will be talking about in this

course, largelyo Studied in linguistics, data from grammaticality

judgments and corpora• Distinction debatable

Content of aSyntactic Theory (1)

• Defeasible predictive theory:o Have theoryo Needs to be able to make predictions

(=deductions)o Predictions need to be verifiable or falsifiable

against empirical mattero When prediction is falsified, theory needs to be

changed Formal framework And/or linguistic content

• “Hypothetico-deductive method” (Popper)

Content of aSyntactic Theory (2)

• What exactly is being predicted?o Set of allowable surface representations:

Is predicted sentence in the language?

o AND correspondence between surface representation and deep representation:

Is the predicted correspondence plausible?

• What is scope of theory?o One language: descriptive theoryo All languages: explanatory theory (Chomsky)

Descriptive Theory

• Theory for one language, which is fixed

• Predicts what surface structures (i.e., strings) are grammatical

• Predicts, for a given grammatical string (and its surface representation) its deep representation

Explanatory Theory (1)

• Need to predict, given a language, what its surface structures and corresponding deep structures are

• Need a parameterized theory• Chomsky (1981, etc):

o Principles: things that hold for all languageso Parameters: values differ for different languages

• “Principles-and-paremeters” type theory also used by other researchers (TAG, HPSG, LFG)

Explanatory Theory (2)

Formalism LinguisticContent

Principles • CFG with slash categories• Head percolation algorithm• …

• Set of nonterminal symbols• Some rules for head percolation algorithm• …

Parameters

• Number of slashes allowed per nonterminal• …

• Whether or not VP is used is standard declarative clause• Rules of CFG• …

Example

Linguistic Theories and Empiricial Matter

• What is predicted? How can the theory be falsified?o Behavior of observable data

• What is the theory “about”?o Descriptive theory: language as

structureo Explanatory theory: presumably,

cognition

The Big Picture

Final, for now

Empirical Matter

Formalisms

•Data structures•Formalisms•Algorithms•Distributional Models

Maud expects there to be a riot*Teri promised there to be a riotMaud expects the shit to hit the fan*Teri promised the shit to hit the

or

Linguistic Theory

Content• Surface representation (eg, ps)• Deep representation (eg, dep)• Correspondence

uses

descriptivetheory is

about

explanatorytheory is about

predicts

The Big Picture

Final, for now

Empirical Matter

Formalisms

•Data structures•Formalisms•Algorithms•Distributional Models

Maud expects there to be a riot*Teri promised there to be a riotMaud expects the shit to hit the fan*Teri promised the shit to hit the

or

Linguistic Theory

Content• Surface representation (eg, ps)• Deep representation (eg, dep)• Correspondence

uses

descriptivetheory is

about

explanatorytheory is about

predicts

In course so far

The Big Picture

Final, for now

Empirical Matter

Formalisms

•Data structures•Formalisms•Algorithms•Distributional Models

Maud expects there to be a riot*Teri promised there to be a riotMaud expects the shit to hit the fan*Teri promised the shit to hit the

or

Linguistic Theory

Content• Surface representation (eg, ps)• Deep representation (eg, dep)• Correspondence

uses

descriptivetheory is

about

explanatorytheory is about

predicts

In rest of course