Download - 6/3/2015CPSC503 Winter 20071 CPSC 503 Computational Linguistics Lecture 10 Giuseppe Carenini
04/18/23 CPSC503 Winter 2007 1
CPSC 503Computational Linguistics
Lecture 10Giuseppe Carenini
04/18/23 CPSC503 Winter 2007 2
Knowledge-Formalisms Map(including probabilistic formalisms)
Logical formalisms (First-Order Logics)
Rule systems (and prob. versions)(e.g., (Prob.) Context-Free
Grammars)
State Machines (and prob. versions)
(Finite State Automata,Finite State Transducers, Markov Models)
Morphology
Syntax
PragmaticsDiscourse and
Dialogue
Semantics
AI planners
04/18/23 CPSC503 Winter 2007 3
Next three classes
• What meaning is and how to represent it
• How to map sentences into their meaning
• Meaning of individual words (lexical semantics)
• Computational Lexical Semantics Tasks– Word sense disambiguation– Word Similarity– Semantic Labeling
04/18/23 CPSC503 Winter 2007 4
Today 16/10
• Semantics / Meaning /Meaning Representations
• Linguistically relevant Concepts in FOPC / POL
• Semantic Analysis
04/18/23 CPSC503 Winter 2007 5
SemanticsDef. Semantics: The study of the meaning of words,
intermediate constituents and sentences
Def1. Meaning: a representation that expresses the linguistic input in terms of objects, actions, events, time, space… beliefs, attitudes...relationships
Def2. Meaning: a representation that links the linguistic input to knowledge of the world
Language independent!
04/18/23 CPSC503 Winter 2007 6
Semantic Relations involving Sentences
Paraphrase: have the same meaning• I gave the apple to John vs. I gave John the apple• I bought a car from you vs. you sold a car to me• The thief was chased by the police vs. ……
Same truth conditions
Entailment: “implication”• The park rangers killed the bear vs. The bear is dead• Nemo is a fish vs. Nemo is an animal
Contradiction: I am in Vancouver vs. I am in India
04/18/23 CPSC503 Winter 2007 7
Meaning Structure of Language
• How does language convey meaning?– Grammaticization
– Display a partially compositional semantics
– Display a basic predicate-argument structure (e.g., verb complements)
– Words
04/18/23 CPSC503 Winter 2007 8
GrammaticizationConcept Affix
• Past• More than one• Again• Negation
• -ed• -s• re-• in-, un-, de-
Words from Nonlexical categories• Obligation• Possibility• Definite, Specific• Indefinite, Non-specific• Disjunction• Negation• Conjunction
• must• may• the• a• or• not• and
04/18/23 CPSC503 Winter 2007 9
Common Meaning Representations
FOL
Semantic Nets
FramesConceptual Dependency
I have a car
04/18/23 CPSC503 Winter 2007 10
Requirements for Meaning Representations
e.g, • Does Maharani serve vegetarian food? -> Yes• What restaurants are close to the ocean?-> C and Monks
• Sample NLP Task: giving advice about restaurants– Accept queries in NL– Generate appropriate responses by
consulting a KB
04/18/23 CPSC503 Winter 2007 11
Verifiability (in the world?)
• Example: Does LeDog serve vegetarian food?• Knowledge base (KB) expressing our world
model (in a formal language)
• Convert question to KB language and verify its truth value against the KB content
Yes / No / I do not know
04/18/23 CPSC503 Winter 2007 12
Canonical Form
Paraphrases should be mapped into the same representation.
• Does LeDog have vegetarian dishes?• Do they have vegetarian food at LeDog?• Are vegetarian dishes served at LeDog?• Does LeDog serve vegetarian fare?• ……………
04/18/23 CPSC503 Winter 2007 13
How to Produce a Canonical Form
• Words have different senses – food ___– dish ___|____one overlapping meaning sense– fare ___|
• Meaning of alternative syntactic constructions are systematically related server thing-being-served– [S [NP Maharani] serves [NP vegetarian dishes]] thing-being-served server [S [NP vegetarian dishes] are served at [NP Maharani]]
04/18/23 CPSC503 Winter 2007 14
Inference and Expressiveness• Consider a more complex request
– Can vegetarians eat at Maharani?– Vs: Does Maharani serve vegetarian food?
• Why do these result in the same answer?
• Inference: System’s ability to draw valid conclusions based on the meaning representations of inputs and its KB
• serve(Maharani,VegetarianFood) => CanEat(Vegetarians,At(Maharani))
Expressiveness: system must be able to handle a wide range of subject matter
04/18/23 CPSC503 Winter 2007 15
Non Yes/No Questions
• Example: I'd like to find a restaurant where I can get vegetarian food.
• Indefinite reference <-> variable serve(x,VegetarianFood)
• Matching succeeds only if variable x can be replaced by known object in KB.
What restaurants are close to the ocean?-> C and Monks
04/18/23 CPSC503 Winter 2007 16
Meaning Structure of Language
• How does language convey meaning?– Grammaticization
– Display a partially compositional semantics
– Display a basic predicate-argument structure (e.g., verb complements)
– Words
04/18/23 CPSC503 Winter 2007 17
Predicate-Argument Structure
• Subcategorization frames specify number, position, and syntactic category of arguments
• Examples: give NP2 NP1, find NP, sneeze []
• Represent relationships among concepts
• Some words act like arguments and some words act like predicates:– Nouns as concepts or arguments: red(ball)– Adj, Adv, Verbs as predicates: red(ball)
04/18/23 CPSC503 Winter 2007 18
Semantic (Thematic) Roles
• Semantic Roles: Participants in an event– Agent: George hit Bill. Bill was hit by George– Theme: George hit Bill. Bill was hit by George
Source, Goal, Instrument, Force…
This can be extended to the realm of semantics
• Verb subcategorization: Allows linking arguments in surface structure with their semantic roles• Mary gave/sent/read a book to Ming
Agent Theme Goal• Mary gave/sent/read Ming a book Agent Goal Theme
04/18/23 CPSC503 Winter 2007 19
Non-verbal predicate-argument structures
• Semantic (Selectional) Restrictions: Constrain the types of arguments verbs take– George assassinated the senator– *The spider assassinated the fly
Selectional Restrictions
• A Spanish restaurant under the bridge
Under(SpanishRestaurant,
bridge)
04/18/23 CPSC503 Winter 2007 20
First Order Predicate Calculus (FOPC)
• FOPC provides sound computational basis for verifiability, inference, expressiveness…– Supports determination of truth– Supports Canonical Form– Supports compositionality of meaning– Supports question-answering (via variables)– Supports inference– Argument-Predicate structure
04/18/23 CPSC503 Winter 2007 21
Today 16/10
• Semantics / Meaning /Meaning Representations
• Linguistically relevant Concepts in FOPC / POL
• Semantic Analysis
04/18/23 CPSC503 Winter 2007 22
Linguistically Relevant Concepts in FOPC
• Categories & Events (Reification)• Representing Time• Beliefs (optional, read if relevant to your project)
• Aspects (optional, read if relevant to your project)
• Description Logics (optional, read if relevant to your project)
04/18/23 CPSC503 Winter 2007 23
Categories & Events
• Events: eg. “Make a reservation”– Reservation (Speaker,Joe’s,Today,8PM,2)– Problems:
• Determining the correct number of roles• Representing facts about the roles associated
with an event• Ensuring that all and only the correct
inferences can be drawn
• Categories:– VegetarianRestaurant (Joe’s) - relation vs. object– MostPopular(Joe’s,VegetarianRestaurant)
Reification– ISA (Joe’s,VegetarianRestaurant)– AKO (VegetarianRestaurant,Restaurant)
04/18/23 CPSC503 Winter 2007 24
MUC-4 Example
INCIDENT: DATE 30 OCT 89 INCIDENT: LOCATION EL SALVADOR INCIDENT: TYPE ATTACK INCIDENT: STAGE OF EXECUTION ACCOMPLISHED INCIDENT: INSTRUMENT ID INCIDENT: INSTRUMENT TYPEPERP: INCIDENT CATEGORY TERRORIST ACT PERP: INDIVIDUAL ID "TERRORIST" PERP: ORGANIZATION ID "THE FMLN" PERP: ORG. CONFIDENCE REPORTED: "THE FMLN" PHYS TGT: ID PHYS TGT: TYPEPHYS TGT: NUMBERPHYS TGT: FOREIGN NATIONPHYS TGT: EFFECT OF INCIDENTPHYS TGT: TOTAL NUMBERHUM TGT: NAMEHUM TGT: DESCRIPTION "1 CIVILIAN"HUM TGT: TYPE CIVILIAN: "1 CIVILIAN"HUM TGT: NUMBER 1: "1 CIVILIAN"HUM TGT: FOREIGN NATIONHUM TGT: EFFECT OF INCIDENT DEATH: "1 CIVILIAN"HUM TGT: TOTAL NUMBER
On October 30, 1989, one civilian was killed in a reported FMLN attack in El Salvador.
04/18/23 CPSC503 Winter 2007 25
Subcategorization frames• I ate• I ate a turkey sandwich• I ate a turkey sandwich at my desk• I ate at my desk• I ate lunch• I ate a turkey sandwich for lunch• I ate a turkey sandwich for lunch at my
desk
no fixed “arity”!
04/18/23 CPSC503 Winter 2007 26
Reification Again
• Reification Advantages:– No need to specify fixed number of
arguments for a given surface predicate– No more roles are postulated than
mentioned in the input– Logical connections among related
examples are specified
“I ate a turkey sandwich for lunch” w: Isa(w,Eating) Eater(w,Speaker)
Eaten(w,TurkeySandwich) MealEaten(w,Lunch)
04/18/23 CPSC503 Winter 2007 27
Representing Time• Events are associated with points or
intervals in time.• We can impose an ordering on distinct
events using the notion of precedes.
• Temporal logic notation: (w,x,t) Arrive(w,x,t)
• Constraints on variable tI arrived in New York(t) Arrive(I,NewYork,t) precedes(t,Now)
04/18/23 CPSC503 Winter 2007 28
Interval Events• Need tstart and tend
“She was driving to New York until now”
tstart,tend ,e, i ISA(e,Drive) Driver(e, She)
Dest(e, NewYork) IntervalOf(e,i)Endpoint(i, tend) Startpoint(i, tend)
Precedes(tstart,Now) Equals(tend,Now)
04/18/23 CPSC503 Winter 2007 29
Relation Between Tenses and Time• Relation between simple verb
tenses and points in time is not straightforward
• Present tense used like future:– We fly from Baltimore to Boston at 10
• Complex tenses:– Flight 1902 arrived late– Flight 1902 had arrived late
04/18/23 CPSC503 Winter 2007 30
Reference Point
• Reichenbach (1947) introduced notion of Reference point (R), separated out from Utterance time (U) and Event time (E)
• Example:– When Mary's flight departed, I ate lunch– When Mary's flight departed, I had eaten
lunch
• Departure event specifies reference point.
04/18/23 CPSC503 Winter 2007 31
Today 16/10
• Semantics / Meaning /Meaning Representations
• Linguistically relevant Concepts in FOPC / POL
• Semantic Analysis
04/18/23 CPSC503 Winter 2007 32
Semantic Analysis
Syntax-drivenSemantic Analysis
Sentence
Literal Meaning
DiscourseStructure
Meanings of words
Meanings of grammatical structures
Context
Common-SenseDomain knowledge
Intended meaning
FurtherAnalysis
INFERENCE
04/18/23 CPSC503 Winter 2007 33
Compositional Analysis
• Principle of Compositionality– The meaning of a whole is derived from
the meanings of the parts
• What parts?– The constituents of the syntactic parse
of the input
• What could it mean for a part to have a meaning?
04/18/23 CPSC503 Winter 2007 34
Compositional Analysis: Example
• AyCaramba serves meat
),()^,()^( MeateServedAyCarambaeServereServinge
04/18/23 CPSC503 Winter 2007 35
Augmented Rules
• Augment each syntactic CFG rule with a semantic formation rule
• The class of actions performed by f will be quite restricted.
)}.,....({... 11 semsemfA nn • Abstractly
• i.e., The semantics of A can be computed from some function applied to the semantics of its parts.
04/18/23 CPSC503 Winter 2007 36
Simple Extension of FOL: Lambda Forms
– Lambda-reduction: variables are bound by treating the lambda form as a function with formal arguments
)(),( yCountryyxyInx
)())((
SallyPSallyxxP
)(),())((),(
yCountryyBCyInBCyCountryyxyInx
)(),( yCountryyBCyIn
)(),())((),(
CANADACountryCANADABCInCANADAyCountryyBCyIn
)(xxP– A FOL sentence with variables in it that are to be bound.
04/18/23 CPSC503 Winter 2007 37
Augmented Rules: Example
– PropNoun -> AyCaramba– MassNoun -> meat
• Attachments{AyCaramba}{MEAT}
assigning constants• Easy parts…
copying from daughters up to mothers.
– NP -> PropNoun– NP -> MassNoun
• Attachments{PropNoun.sem}{MassNoun.sem
}
04/18/23 CPSC503 Winter 2007 38
Augmented Rules: Example
• Verb -> serves
• {VP.sem(NP.sem)}• {Verb.sem(NP.sem)
),(^),(^)(
xeServedyeServereServingeyx
Semantics attached to one daughter is applied to semantics of the other
daughter(s).• S -> NP VP• VP -> Verb NP
lambda-form
04/18/23 CPSC503 Winter 2007 39
Example
• S -> NP VP• VP -> Verb NP• Verb -> serves• NP -> PropNoun• NP -> MassNoun• PropNoun -> AyCaramba• MassNoun -> meat
• {VP.sem(NP.sem)}• {Verb.sem(NP.sem)
• {PropNoun.sem}• {MassNoun.sem}• {AC}• {MEAT}
),()^,()^( xeServedyeServereServingeyx
MEAT
MEAT
…….
y y
AC
04/18/23 CPSC503 Winter 2007 40
Full story more complex• To deal properly with quantifiers
– Permit lambda-variables to range over predicates. E.g.,
)(. xPxP
)yMenuyHad(e,
)xRestaurantxeHaver
eeHaving
)(
)(,(
)(
– Introduce complex terms to remain agnostic about final scoping
04/18/23 CPSC503 Winter 2007 41
• Similarly to PP attachment, number of possible interpretations exponential in the number of complex terms
Solution: Quantifier Scope Ambiguity
• likelihood of different orderings• Mirror surface ordering• Domain specific knowledge
• Weak methods to prefer one interpretation over another:
04/18/23 CPSC503 Winter 2007 42
Attachments for a fragment of English (Sect. 18.5)
• Sentences• Noun-phrases• Verb-phrases• Prepositional-phrases
Based on “The core Language Engine” 1992
04/18/23 CPSC503 Winter 2007 43
Integration with a Parser• Assume you’re using a dynamic-
programming style parser (Earley or CYK).
• Two basic approaches– Integrate semantic analysis into the
parser (assign meaning representations as constituents are completed)– Pipeline… assign meaning representations to complete trees only after they’re completed
04/18/23 CPSC503 Winter 2007 44
Pros and Cons• Integration
– use semantic constraints to cut off parses that make no sense
– assign meaning representations to constituents that don’t take part in any correct parse
• Pipeline– assign meaning representations only to
constituents that take part in a correct parse
– parser needs to generate all correct parses
04/18/23 CPSC503 Winter 2007 45
Next Time
• Read Chp. 19 (Lexical Semantics)
04/18/23 CPSC503 Winter 2007 46
Non-Compositionality
• Unfortunately, there are lots of examples where the meaning of a constituent can’t be derived from the meanings of the parts
- metaphor, (e.g., corporation as person)– metonymy, (??)– idioms, – irony, – sarcasm, – indirect requests, etc
04/18/23 CPSC503 Winter 2007 47
English Idioms
• “buy the farm” • “bite the bullet” • “bury the hatchet” • etc…
• Lots of these… constructions where the meaning of the whole is either – Totally unrelated to the meanings of the parts
(“kick the bucket”)– Related in some opaque way (“run the show”)
04/18/23 CPSC503 Winter 2007 48
The Tip of the Iceberg– “Enron is the tip of the iceberg.”NP -> “the tip of the iceberg” {….}
– “the tip of an old iceberg”– “the tip of a 1000-page iceberg”– “the merest tip of the iceberg”
NP -> TipNP of IcebergNP {…}TipNP: NP with tip as its head IcebergNP NP with iceberg as its head
04/18/23 CPSC503 Winter 2007 49
Handling Idioms– Mixing lexical items and grammatical
constituents– Introduction of idiom-specific constituents– Permit semantic attachments that introduce
predicates unrelated with constituents
NP -> TipNP of IcebergNP {small-part(), beginning()….}
TipNP: NP with tip as its head IcebergNP NP with iceberg as its head