computational semantics

Computational Semantics

Torbjörn LagerDepartment of LinguisticsUppsala University

NLP1 - Torbjörn Lager 2

Computational Semantics

Compositional, logical semantics* Computational lexical semantics Word sense disambiguation* Text categorization Information extraction Information retrieval


Logical Semantics Example

John laughed laughed'(j)

Nobody laughed x[laughed'(x)]

But this is just translation! What's semantic about that?


What Is the Name of This Business?

Truth conditional semantics Model theoretic semantics Logical semantics Formal semantics Compositional semantics Syntax-driven semantic analysis

Compositional, logical, truth conditional, model theoretic semantics ....


An Important Tradition

We use language to talk about the world

Semantics is something that relates sentences (or utterances) of language and the outside world

There are other ideas about meaning, but in this tradition we don't believe in them!

Natural language

The outside world


Truth Conditional Semantics

Meaning = Truth conditions Examples:

"John whistles" is true iff John whistles

"John visslar" is true iff John whistles

"Ogul fautu seq" is true iff...

Natural language

The outside world


Model Theoretic Semantics

We don't know what the world is really like, so let's talk about a model of the world instead

Such a model does (usually) consists of individuals, sets of individuals, functions and relations. i.e the sort of things set theory talks about

Truth becomes truth relative to a model

Natural language

Model

The outside world


Compositional Semantics

The Compositionality Principle: The meaning of the whole is a

function of the meaning of the parts and the mode of combining them.

The meaning of a complex expression is uniquely determined by the meaning of its constituents and the syntactic construction used to combine them.

Natural language

Model

The World


Truth Conditional, Model Theoretic and Compositional Semantics Combined

A simple model M: Domain:

{John, Mary, Paul} Interpretation:

Names: "John" refers to John , "Mary" refers to Mary, etc.

Verbs: "whistles" refers to {John, Paul}

Example "John whistles" is true in

M iff the individual in M referred to as "John" is an element in the set of individuals that "whistles" refer to.

Richard Montague (1970): "I reject the contention that an important theoretical difference exists between formal and natural languages"


Translational Semantics

Account for the meanings of natural language utterances by translating them into another language.

It could be any language, but only if this language has a formal semantics are we done.

Natural language

Logical Form Language

Model

The World


Why Logical Semantics?

Account for ambiguity "every man loves a woman"

Allow evaluation e.g. by database lookup

Allow logical inference Every man who whistles is happy John is a man John whistles Therefore: John is happy


Applications of Logical Semantics

NLU systems Semantics + 'World knowledge' --> 'understanding'

Information Extraction Machine translation

LF as (part of an) interlingua

Dialogue Systems


Grammar and Logical Form

S -> NP VP[S] = [VP]([NP])

NP -> john[NP] = j

VP -> whistles[VP] = x[whistles'(x)]

[john whistles] = whistles'(j)

cf. The principle ofcompositionality

Have same truth conditions


Beta Reduction (Lambda Conversion)

[S] = [VP]([NP]) [NP] = j [VP] = x[whistles'(x)]

Beta reduction rule: u() where every occurrence of u in is replaced by

x[whistles'(x)](j) application whistles'(j) reduction


Grammar and Logical Form

S -> NP VP[S] = [NP]([VP])

NP -> john[NP] = P[P(j)]




From Logical Form to Truth Conditions

whistles'(j) is true iffthe individual (in the model) denoted by 'j' has the property denoted by 'whistles'

cf. "John whistles" is true iff John whistles


Beta Reduction (Lambda Conversion)

[S] = [NP]([VP]) [NP] = P[P(j)] [VP] = x[whistles'(x)]

Beta reduction rule: u() where every occurrence of u in is replaced by

P[P(j)](x[whistles'(x)]) application x[whistles'(x)](j) reduction whistles'(j) reduction


A Larger Example

S -> NP VP [S] = [NP]([VP])

NP -> DET N [NP] = [DET]([N])

DET -> every[DET] = Q[P[z[Q(z) P(z)]]]

N -> man [N] = x[man'(x)]


[every man whistles} = z[man'(z) whistles'(z)]


A Larger Example (cont'd)

[S] = [NP]([VP]) [NP] = [DET]([N]) [DET] = Q[P[z[Q(z) P(z)]]] [N] = x[man'(x)] [VP] = x[whistles'(x)]

Q[P[z[Q(z) P(z)]]](x[man'(x)]) application P[z[x[man'(x)](z) P(z)]] reduction P[z[man'(z) P(z)]] reduction P[z[man'(z) P(z)]](x[whistles'(x)]) application z[man'(z) x[whistles'(x)](z)] reduction z[man'(z) whistles'(z)] reduction


Alternative Ontologies

[john whistles] =

ex[isa(e,WhistleEvent) agent(e,x) named(x,"john")]



Alternative Meaning Representation Formalisms

Discourse Representation Theory

[every man whistles} =

man(x) whistles(x)

x


Semantics Research

How to design a nice meaning representation language? Ontology? How to treat a particular NL construct?

in a compositional way? and end up with correct truth conditions? and still be elegant and clean?

How to deal with things like time and events, propositional attitude reports, reference to non-existent individuals, etc.?

How to solve a particular semantic puzzle? How to design a nice syntax-semantics interface? How to design a nice semantics-pragmatics interface? What is the role of inference in semantic processing? How to account for things like presuppositions?


Semantic Puzzles

Why is "Every man loves a woman" ambiguous, but not "Every man loves Mary"?

What's wrong with the following argument: "Nothing is better than a long and prosperous life. A ham sandwich is better than nothing. Therefore, a ham sandwich is better than a long and prosperous life."

The morning star = the evening star, still"John believes that Venus is the morning star" may be true, and at the same time, "John believes that Venus is the evening star", may be false.

Everything written on this slide is false.

Word Sense Disambiguation

Torbjörn LagerDepartment of LinguisticsUppsala University


Example: Senses of "interest"

From the LDOCE

1. readiness to give attention, 2. quality of causing attention to be given3. activity, subject, etc., which one gives time and

attention to4. advantage, advancement, or favour5. a share (in a company, business, etc.)6. money paid for the use of money


WSD Examples

At the same time, the drop in interest rates since the spring has failed to revive the residential construction industry.

Cray Research will retain a 10% interest in the new company, which will be based in Colorado Springs.

Although that may sound like an arcane maneuver of little interest outside Washington, it would set off a political earthquake.


Why Word Sense Disambiguation?

Well, compositional logical semantics doesn't deal with word meaning, so...

Machine translation A non-disambiguated Russian translation of: “The spirit is

willing but the flesh is weak” gave “The vodka is good but the meat is rotten”

Information Retrieval When searching the web for info about companies buying

shares in other companies, you don’t want to retrieve the information about interest rates.

Provide clues to pronounciation "banan" -> BAnan or baNAN


Approaches to WSD

Deep (but brittle) WSD 'Selectional restriction'-based approaches Approaches based on general reasoning with 'world

knowledge'

Shallow and robust WSD Machine learning approaches

Supervised learning Unsupervised learning Bootstrapping approaches

Dictionary-based approaches Various combinations of methods


Machine Learning Approaches

Training data in the form of annotated corpora Decide on features on which to condition Preprocessing Steps

Context trimming Stemming/Lemmatizing Part-of-speech tagging Partial Parsing

Use a machine learning algorithm Enter the training-test cycle


Dictionary-Based Approaches

Lesk (1986)

Find the dictionary definition that overlaps most with the definitions for the words in the ambiguous word’s context.

Problem 1: A lot of computation. Problem 2: Definitions are usually too short


Lesk Example

Lexicon entries:

pine 1 kinds of evergreen tree with needle-shaped leaves

2 waste away through sorrow or illness

cone 1 solid body which narrows to a point

2 something of this shape whether solid or hollow

3 fruit of certain evergreen tree

Example: .... pine cone ...

The third sense for "cone" is selected here, since two of the (content) words in its entry, "evergreen" and "tree", overlap with words in the entry for "pine", and this doesn't happen for the other senses.


SENSEVAL-2: Second International Workshop on Evaluating Word Sense Disambiguation Systems

The purpose of SENSEVAL is to evaluate the strengths and weaknesses of WSD programs with respect to different words, different varieties of language, and different languages.

Two types of task over 12 languages. In the "all-words" task, the evaluation is on almost all of the content

words in a sample of texts. In the "lexical sample" task, first we sample the lexicon, then we find

instances in context of the sample words and the evaluation is on those instances only.

All-words: Czech, Dutch, English, Estonian Lexical sample: Basque, Chinese, Danish, English, Italian, Japanese,

Korean, Spanish, Swedish About 35 teams participated, submitting over 90 systems.


The Swedish Lexical Sample Task

40 lemmas: 20 nouns, 15 verbs and 5 adjectives. Together representing 145 senses and 304 sub-senses. 8,718 annotated instances were provided as training

material 1,527 unannotated instances were provided for testing. A lexicon - the GLDB (Gothenburg Lexical Database) -

complete with morphological information, definitions, language examples, etc., was also available.


Demo

http://www.ling.gu.se/~lager/Home/pwe_ui.html

computational semantics

Documents

nlp1 torbjrn lagerwhat

vpnpnp john np

natural languagethe

utterances of language

examplejohn whistles

truth conditionsexamples

truth relative

set of individuals