04/18/23 CPSC503 Winter 2007 1

CPSC 503Computational Linguistics

Lecture 10Giuseppe Carenini

04/18/23 CPSC503 Winter 2007 2

Knowledge-Formalisms Map(including probabilistic formalisms)

Logical formalisms (First-Order Logics)

Rule systems (and prob. versions)(e.g., (Prob.) Context-Free

Grammars)

State Machines (and prob. versions)

(Finite State Automata,Finite State Transducers, Markov Models)

Morphology

Syntax

PragmaticsDiscourse and

Dialogue

Semantics

AI planners

04/18/23 CPSC503 Winter 2007 3

Next three classes

• What meaning is and how to represent it

• How to map sentences into their meaning

• Meaning of individual words (lexical semantics)

• Computational Lexical Semantics Tasks– Word sense disambiguation– Word Similarity– Semantic Labeling

04/18/23 CPSC503 Winter 2007 4

Today 16/10

• Semantics / Meaning /Meaning Representations

• Linguistically relevant Concepts in FOPC / POL

• Semantic Analysis

04/18/23 CPSC503 Winter 2007 5

SemanticsDef. Semantics: The study of the meaning of words,

intermediate constituents and sentences

Def1. Meaning: a representation that expresses the linguistic input in terms of objects, actions, events, time, space… beliefs, attitudes...relationships

Def2. Meaning: a representation that links the linguistic input to knowledge of the world

Language independent!

04/18/23 CPSC503 Winter 2007 6

Semantic Relations involving Sentences

Paraphrase: have the same meaning• I gave the apple to John vs. I gave John the apple• I bought a car from you vs. you sold a car to me• The thief was chased by the police vs. ……

Same truth conditions

Entailment: “implication”• The park rangers killed the bear vs. The bear is dead• Nemo is a fish vs. Nemo is an animal

Contradiction: I am in Vancouver vs. I am in India

04/18/23 CPSC503 Winter 2007 7

Meaning Structure of Language

• How does language convey meaning?– Grammaticization

– Display a partially compositional semantics

– Display a basic predicate-argument structure (e.g., verb complements)

– Words

04/18/23 CPSC503 Winter 2007 8

GrammaticizationConcept Affix

• Past• More than one• Again• Negation

• -ed• -s• re-• in-, un-, de-

Words from Nonlexical categories• Obligation• Possibility• Definite, Specific• Indefinite, Non-specific• Disjunction• Negation• Conjunction

• must• may• the• a• or• not• and

04/18/23 CPSC503 Winter 2007 9

Common Meaning Representations

FOL

Semantic Nets

FramesConceptual Dependency

I have a car

04/18/23 CPSC503 Winter 2007 10

Requirements for Meaning Representations

e.g, • Does Maharani serve vegetarian food? -> Yes• What restaurants are close to the ocean?-> C and Monks

• Sample NLP Task: giving advice about restaurants– Accept queries in NL– Generate appropriate responses by

consulting a KB

04/18/23 CPSC503 Winter 2007 11

Verifiability (in the world?)

• Example: Does LeDog serve vegetarian food?• Knowledge base (KB) expressing our world

model (in a formal language)

• Convert question to KB language and verify its truth value against the KB content

Yes / No / I do not know

04/18/23 CPSC503 Winter 2007 12

Canonical Form

Paraphrases should be mapped into the same representation.

• Does LeDog have vegetarian dishes?• Do they have vegetarian food at LeDog?• Are vegetarian dishes served at LeDog?• Does LeDog serve vegetarian fare?• ……………

04/18/23 CPSC503 Winter 2007 13

How to Produce a Canonical Form

• Words have different senses – food ___– dish ___|____one overlapping meaning sense– fare ___|

• Meaning of alternative syntactic constructions are systematically related server thing-being-served– [S [NP Maharani] serves [NP vegetarian dishes]] thing-being-served server [S [NP vegetarian dishes] are served at [NP Maharani]]

04/18/23 CPSC503 Winter 2007 14

Inference and Expressiveness• Consider a more complex request

– Can vegetarians eat at Maharani?– Vs: Does Maharani serve vegetarian food?

• Why do these result in the same answer?

• Inference: System’s ability to draw valid conclusions based on the meaning representations of inputs and its KB

• serve(Maharani,VegetarianFood) => CanEat(Vegetarians,At(Maharani))

Expressiveness: system must be able to handle a wide range of subject matter

04/18/23 CPSC503 Winter 2007 15

Non Yes/No Questions

• Example: I'd like to find a restaurant where I can get vegetarian food.

• Indefinite reference <-> variable serve(x,VegetarianFood)

• Matching succeeds only if variable x can be replaced by known object in KB.

What restaurants are close to the ocean?-> C and Monks

04/18/23 CPSC503 Winter 2007 16

Meaning Structure of Language

• How does language convey meaning?– Grammaticization

– Display a partially compositional semantics

– Display a basic predicate-argument structure (e.g., verb complements)

– Words

04/18/23 CPSC503 Winter 2007 17

Predicate-Argument Structure

• Subcategorization frames specify number, position, and syntactic category of arguments

• Examples: give NP2 NP1, find NP, sneeze []

• Represent relationships among concepts

• Some words act like arguments and some words act like predicates:– Nouns as concepts or arguments: red(ball)– Adj, Adv, Verbs as predicates: red(ball)

04/18/23 CPSC503 Winter 2007 18

Semantic (Thematic) Roles

• Semantic Roles: Participants in an event– Agent: George hit Bill. Bill was hit by George– Theme: George hit Bill. Bill was hit by George

Source, Goal, Instrument, Force…

This can be extended to the realm of semantics

• Verb subcategorization: Allows linking arguments in surface structure with their semantic roles• Mary gave/sent/read a book to Ming

Agent Theme Goal• Mary gave/sent/read Ming a book Agent Goal Theme

04/18/23 CPSC503 Winter 2007 19

Non-verbal predicate-argument structures

• Semantic (Selectional) Restrictions: Constrain the types of arguments verbs take– George assassinated the senator– *The spider assassinated the fly

Selectional Restrictions

• A Spanish restaurant under the bridge

Under(SpanishRestaurant,

bridge)

04/18/23 CPSC503 Winter 2007 20

First Order Predicate Calculus (FOPC)

• FOPC provides sound computational basis for verifiability, inference, expressiveness…– Supports determination of truth– Supports Canonical Form– Supports compositionality of meaning– Supports question-answering (via variables)– Supports inference– Argument-Predicate structure

04/18/23 CPSC503 Winter 2007 21

Today 16/10

• Semantics / Meaning /Meaning Representations

• Linguistically relevant Concepts in FOPC / POL

• Semantic Analysis

04/18/23 CPSC503 Winter 2007 22

Linguistically Relevant Concepts in FOPC

• Categories & Events (Reification)• Representing Time• Beliefs (optional, read if relevant to your project)

• Aspects (optional, read if relevant to your project)

• Description Logics (optional, read if relevant to your project)

04/18/23 CPSC503 Winter 2007 23

Categories & Events

• Events: eg. “Make a reservation”– Reservation (Speaker,Joe’s,Today,8PM,2)– Problems:

• Determining the correct number of roles• Representing facts about the roles associated

with an event• Ensuring that all and only the correct

inferences can be drawn

• Categories:– VegetarianRestaurant (Joe’s) - relation vs. object– MostPopular(Joe’s,VegetarianRestaurant)

Reification– ISA (Joe’s,VegetarianRestaurant)– AKO (VegetarianRestaurant,Restaurant)

04/18/23 CPSC503 Winter 2007 24

MUC-4 Example

INCIDENT: DATE 30 OCT 89 INCIDENT: LOCATION EL SALVADOR INCIDENT: TYPE ATTACK INCIDENT: STAGE OF EXECUTION ACCOMPLISHED INCIDENT: INSTRUMENT ID INCIDENT: INSTRUMENT TYPEPERP: INCIDENT CATEGORY TERRORIST ACT PERP: INDIVIDUAL ID "TERRORIST" PERP: ORGANIZATION ID "THE FMLN" PERP: ORG. CONFIDENCE REPORTED: "THE FMLN" PHYS TGT: ID PHYS TGT: TYPEPHYS TGT: NUMBERPHYS TGT: FOREIGN NATIONPHYS TGT: EFFECT OF INCIDENTPHYS TGT: TOTAL NUMBERHUM TGT: NAMEHUM TGT: DESCRIPTION "1 CIVILIAN"HUM TGT: TYPE CIVILIAN: "1 CIVILIAN"HUM TGT: NUMBER 1: "1 CIVILIAN"HUM TGT: FOREIGN NATIONHUM TGT: EFFECT OF INCIDENT DEATH: "1 CIVILIAN"HUM TGT: TOTAL NUMBER

On October 30, 1989, one civilian was killed in a reported FMLN attack in El Salvador.

04/18/23 CPSC503 Winter 2007 25

Subcategorization frames• I ate• I ate a turkey sandwich• I ate a turkey sandwich at my desk• I ate at my desk• I ate lunch• I ate a turkey sandwich for lunch• I ate a turkey sandwich for lunch at my

desk

no fixed “arity”!

04/18/23 CPSC503 Winter 2007 26

Reification Again

• Reification Advantages:– No need to specify fixed number of

arguments for a given surface predicate– No more roles are postulated than

mentioned in the input– Logical connections among related

examples are specified

“I ate a turkey sandwich for lunch” w: Isa(w,Eating) Eater(w,Speaker)

Eaten(w,TurkeySandwich) MealEaten(w,Lunch)

04/18/23 CPSC503 Winter 2007 27

Representing Time• Events are associated with points or

intervals in time.• We can impose an ordering on distinct

events using the notion of precedes.

• Temporal logic notation: (w,x,t) Arrive(w,x,t)

• Constraints on variable tI arrived in New York(t) Arrive(I,NewYork,t) precedes(t,Now)

04/18/23 CPSC503 Winter 2007 28

Interval Events• Need tstart and tend

“She was driving to New York until now”

tstart,tend ,e, i ISA(e,Drive) Driver(e, She)

Dest(e, NewYork) IntervalOf(e,i)Endpoint(i, tend) Startpoint(i, tend)

Precedes(tstart,Now) Equals(tend,Now)

04/18/23 CPSC503 Winter 2007 29

Relation Between Tenses and Time• Relation between simple verb

tenses and points in time is not straightforward

• Present tense used like future:– We fly from Baltimore to Boston at 10

• Complex tenses:– Flight 1902 arrived late– Flight 1902 had arrived late

04/18/23 CPSC503 Winter 2007 30

Reference Point

• Reichenbach (1947) introduced notion of Reference point (R), separated out from Utterance time (U) and Event time (E)

• Example:– When Mary's flight departed, I ate lunch– When Mary's flight departed, I had eaten

lunch

• Departure event specifies reference point.

04/18/23 CPSC503 Winter 2007 31

Today 16/10

• Semantics / Meaning /Meaning Representations

• Linguistically relevant Concepts in FOPC / POL

• Semantic Analysis

04/18/23 CPSC503 Winter 2007 32

Semantic Analysis

Syntax-drivenSemantic Analysis

Sentence

Literal Meaning

DiscourseStructure

Meanings of words

Meanings of grammatical structures

Context

Common-SenseDomain knowledge

Intended meaning

FurtherAnalysis

INFERENCE

04/18/23 CPSC503 Winter 2007 33

Compositional Analysis

• Principle of Compositionality– The meaning of a whole is derived from

the meanings of the parts

• What parts?– The constituents of the syntactic parse

of the input

• What could it mean for a part to have a meaning?

04/18/23 CPSC503 Winter 2007 34

Compositional Analysis: Example

• AyCaramba serves meat

),()^,()^( MeateServedAyCarambaeServereServinge

04/18/23 CPSC503 Winter 2007 35

Augmented Rules

• Augment each syntactic CFG rule with a semantic formation rule

• The class of actions performed by f will be quite restricted.

)}.,....({... 11 semsemfA nn • Abstractly

• i.e., The semantics of A can be computed from some function applied to the semantics of its parts.

04/18/23 CPSC503 Winter 2007 36

Simple Extension of FOL: Lambda Forms

– Lambda-reduction: variables are bound by treating the lambda form as a function with formal arguments

)(),( yCountryyxyInx

)())((

SallyPSallyxxP

)(),())((),(

yCountryyBCyInBCyCountryyxyInx

)(),( yCountryyBCyIn

)(),())((),(

CANADACountryCANADABCInCANADAyCountryyBCyIn

)(xxP– A FOL sentence with variables in it that are to be bound.

04/18/23 CPSC503 Winter 2007 37

Augmented Rules: Example

– PropNoun -> AyCaramba– MassNoun -> meat

• Attachments{AyCaramba}{MEAT}

assigning constants• Easy parts…

copying from daughters up to mothers.

– NP -> PropNoun– NP -> MassNoun

• Attachments{PropNoun.sem}{MassNoun.sem

}

04/18/23 CPSC503 Winter 2007 38

Augmented Rules: Example

• Verb -> serves

• {VP.sem(NP.sem)}• {Verb.sem(NP.sem)

),(^),(^)(

xeServedyeServereServingeyx

Semantics attached to one daughter is applied to semantics of the other

daughter(s).• S -> NP VP• VP -> Verb NP

lambda-form

04/18/23 CPSC503 Winter 2007 39

Example

• S -> NP VP• VP -> Verb NP• Verb -> serves• NP -> PropNoun• NP -> MassNoun• PropNoun -> AyCaramba• MassNoun -> meat

• {VP.sem(NP.sem)}• {Verb.sem(NP.sem)

• {PropNoun.sem}• {MassNoun.sem}• {AC}• {MEAT}

),()^,()^( xeServedyeServereServingeyx

MEAT

MEAT

…….

y y

AC

04/18/23 CPSC503 Winter 2007 40

Full story more complex• To deal properly with quantifiers

– Permit lambda-variables to range over predicates. E.g.,

)(. xPxP

)yMenuyHad(e,

)xRestaurantxeHaver

eeHaving

)(

)(,(

)(

– Introduce complex terms to remain agnostic about final scoping

04/18/23 CPSC503 Winter 2007 41

• Similarly to PP attachment, number of possible interpretations exponential in the number of complex terms

Solution: Quantifier Scope Ambiguity

• likelihood of different orderings• Mirror surface ordering• Domain specific knowledge

• Weak methods to prefer one interpretation over another:

04/18/23 CPSC503 Winter 2007 42

Attachments for a fragment of English (Sect. 18.5)

• Sentences• Noun-phrases• Verb-phrases• Prepositional-phrases

Based on “The core Language Engine” 1992

04/18/23 CPSC503 Winter 2007 43

Integration with a Parser• Assume you’re using a dynamic-

programming style parser (Earley or CYK).

• Two basic approaches– Integrate semantic analysis into the

parser (assign meaning representations as constituents are completed)– Pipeline… assign meaning representations to complete trees only after they’re completed

04/18/23 CPSC503 Winter 2007 44

Pros and Cons• Integration

– use semantic constraints to cut off parses that make no sense

– assign meaning representations to constituents that don’t take part in any correct parse

• Pipeline– assign meaning representations only to

constituents that take part in a correct parse

– parser needs to generate all correct parses

04/18/23 CPSC503 Winter 2007 45

Next Time

• Read Chp. 19 (Lexical Semantics)

04/18/23 CPSC503 Winter 2007 46

Non-Compositionality

• Unfortunately, there are lots of examples where the meaning of a constituent can’t be derived from the meanings of the parts

- metaphor, (e.g., corporation as person)– metonymy, (??)– idioms, – irony, – sarcasm, – indirect requests, etc

04/18/23 CPSC503 Winter 2007 47

English Idioms

• “buy the farm” • “bite the bullet” • “bury the hatchet” • etc…

• Lots of these… constructions where the meaning of the whole is either – Totally unrelated to the meanings of the parts

(“kick the bucket”)– Related in some opaque way (“run the show”)

04/18/23 CPSC503 Winter 2007 48

The Tip of the Iceberg– “Enron is the tip of the iceberg.”NP -> “the tip of the iceberg” {….}

– “the tip of an old iceberg”– “the tip of a 1000-page iceberg”– “the merest tip of the iceberg”

NP -> TipNP of IcebergNP {…}TipNP: NP with tip as its head IcebergNP NP with iceberg as its head

04/18/23 CPSC503 Winter 2007 49

Handling Idioms– Mixing lexical items and grammatical

constituents– Introduction of idiom-specific constituents– Permit semantic attachments that introduce

predicates unrelated with constituents

NP -> TipNP of IcebergNP {small-part(), beginning()….}

TipNP: NP with tip as its head IcebergNP NP with iceberg as its head