Semantics of Paragraphs

Wlodek Zadrozny, Karen Jensen

Computational Linguistics 17 (2) pp171-209 (1991)

(as (mis-)interpreted by Peter Clark)

Why study paragraphs?• Paragraph: “unit of thought”• Can study:

– Coherence– Anaphora resolution– Discourse structure, (e.g. use of connectives like “but”,

“however”, “therefore”, …)

• Proposes three-level logical theory– Object level: a model of the paragraph itself– Reference level: background knowledge

• E.g., from dictionaries and encyclopedias

– Metalevel: constraints on the object level representation

Example paragraph

“In the summer of 1347 a merchant ship returning from the Black Sea entered the Sicilian port of Messina bringing with it the horrifying disease that came to be known as the Black Death. It struck rapidly. Within twenty-four hours of infection and the appearance of the first small black pustule came an agonizing death. The effect of the Black Death was appalling. In less than twenty years half the population of Europe had been killed, the countryside devastated, and a period of optimism and growing economic welfare had been brought to a sudden and catastrophic end.”

“In the summer of 1347 a merchant ship returning from the Black Sea entered the Sicilian port of Messina bringing with it the horrifying disease that came to be known as the Black Death. It struck rapidly. Within twenty-four hours of infection and the appearance of the first small black pustule came an agonizing death. The effect of the Black Death was appalling. In less than twenty years half the population of Europe had been killed, the countryside devastated, and a period of optimism and growing economic welfare had been brought to a sudden and catastrophic end.”

But few grammatical devices for paragraph cohesion!- no connectives- only two anaphoric pronouns (“it”)

“In the summer of 1347 a merchant ship returning from the Black Sea entered the Sicilian port of Messina bringing with it….”

Against knowledge-poor methods…

• “it” = the ship or the port?

• Surface syntax & selectional restrictions don’t suffice – need background knowledge

– Dictionary can supply this

• Also: want to establish shared-word relationships between “disease”, “Black Death”, “infection”, “death”, “killed”, “end”.

– Dictionary can supply these connections too

“A ship entered the port of Messina bringing with it….”

For example…

Bring: To convey, lead, carry or cause to come along… Ship: A large boat for carrying people or goods…Port: 1. Harbour… 2. An opening in the side of a ship…

Longman’s Dictionary:= ?

- There’s a “short connection” between “bring” and “ship”(PS reminiscent of Thesaurus-based search)

- Suggests “it” = “ship” is most coherent interpretation

The Structure of Background Knowledge• A set of logical “theories”

– One for each sense of each word– Theories ordered according to “preference”– Theories for different word senses can’t exist together (i.e., are inconsistent)

• These can be obtained automatically from dictionary definitions.

sh1: “Ship: A large boat for carrying people or goods at sea.” ship(x) large-boat(x);

y carry(x,y) & (people(y) goods(y))

sh1: “Ship: A large boat for carrying people or goods at sea.” ship(x) large-boat(x);

y carry(x,y) & (people(y) goods(y))

sh2: “Ship: A large aircraft or space vehicle.” ship(x) large-aircraft(x) space-vehicle(x)

e1: “Enter: To come into a place.” enter(x,y) come-in(x,y); place(y); …

e2: “Enter: To join a group, typically of professionals.” enter(x,y) join(x,y) & group(y); professionals(y); …

Background Knowledge: Example

Partial ordering of interpretation preference: e1 >enter e2 > unknownsh1 >ship sh2 > unknown

NB use of “ambiguous predicates”e.g. ship(x)!

How to Use Background Knowledge1. Sentence logical form (“object level theory”)

“Entering the port, a ship brought a disease”

enter(x1,x2) & ship(x1) & port(x2) & bring(x3,x4) & disease(x4)

“ambiguous predicates”

How to Use Background Knowledge2. For each predicate (was word), add in one background theory

(sense) for that word

one interpretation of the sentence

enter(x1,x2) & ship(x1) & port(x2) & bring(x3,x4) & disease(x4)

;;; sh1: “Ship: A large boat for carrying people or goods at sea.”ship(x) large-boat(x); y carry(x,y) & (people(y) goods(y))

;;; e1: “enter: come into a place.” enter(x,y) come-in(x,y); place(y); …

An example of an interpretation:

How to Use Background Knowledge3. Search the space of interpretations to find the “best”:

- discount interpretations which are inconsistent

e.g. x = boat & x = aircraft

- dominance: prefer the more plausible theories

- coherence: prefer interpretations with “coherence”

- coreference: maximize coreference links

- metalevel constraints: should be satisfied

“Entering the port, a ship brought a disease”

enter(x1,x2) & ship(x1) & port(x2) & bring(x3,x4) & disease(x4)

e1

e2

sh1

sh2

p1d1b1

b2

a a

a a

a “Ship: large boat”

“Ship: aircraft or space vehicle”

“Enter: to come in”

“Enter: to join”

1. Dominance

“Entering the port, a ship brought a disease”

enter(x1,x2) & ship(x1) & port(x2) & bring(x3,x4) & disease(x4)

e1

e2

sh1

sh2

p1d1b1

b2

a a

a a

a “Ship: large boat”

“Ship: aircraft or space vehicle”

“Enter: to come in”

“Enter: to join”

Interpretation 1:

1. Dominance

“Entering the port, a ship brought a disease”

enter(x1,x2) & ship(x1) & port(x2) & bring(x3,x4) & disease(x4)

e1

e2

sh1

sh2

p1d1b1

b2

a a

a a

a “Ship: large boat”

“Ship: aircraft or space vehicle”

“Enter: to come in”

“Enter: to join”

Interpretation 2:

1. Dominance

“Entering the port, a ship brought a disease”

enter(x1,x2) & ship(x1) & port(x2) & bring(x3,x4) & disease(x4)

e1

e2

sh1

sh2

p1d1b1

b2

a a

a a

a “Ship: large boat”

“Ship: aircraft or space vehicle”

“Enter: to come in”

“Enter: to join”

All interpretations = all (consistent) paths

1. Dominance

Or to put it another way...

“Entering the port, a ship brought a disease”

{Coming into/Joining} the {harbor}, a {ship/large aircraft} {carried/caused} a {illness}.

Possible interpretations are all permutations of:

Sentence: “Entering the port, a ship brought a disaster.”

b2: “Bring: to cause”

d1: “Disaster: causes a harm”

2. Coherence• Look for “coherence links” (shared predicates)

between (but not within) theories• More c-links more coherent• Consider:

C-link!

(compared with: b1: “Bring: To carry”)

Sentence: “Entering the port, a ship brought a disaster.”

b2: “Bring: to cause” d1: “Disaster: causes a harm”

2. Coherence (cont)• word senses which “connect together” are

preferred• the coherence of words in a sentence is based on

the coherence of their meaning

2. Coherence (cont)

• Do we iterate, to look at definitions of terms in definitions etc.?– Zadrozny and Jensen: No, psychologically implausible.

Paragraphs! The Topic of a Paragraph

• A topic is one or more predicates, e.g. ship(), port(),..• That for each sentences in the paragraph, the sentence

– mentions the topic

– or, uses a word whose definition mentions the topic

– or, where the topic definition mentions the word

– or, refers back to a sentence which mentioned the topic

• Qn: Can one odd sentence “destroy” a topic?

;;; “Entering the port, a ship brought a disease” enter(x1,x2) & ship(x1) & port(x2) & bring(x3,x4) & disease(x4)

For example...

“In the summer of 1347 a merchant ship returning from the Black Sea entered the Sicilian port of Messina bringing with it the horrifying disease that came to be known as the Black Death. It struck rapidly. Within twenty-four hours of infection and the appearance of the first small black pustule came an agonizing death….

Disease: illness causing infection

Topic: the Black Death disease, i.e., disease(), as

cf. incoherent: “John took a train from Paris to Istanbul. He likes spinach.”

3. Maximize coreference/anaphora resolution

• Maximize the number of equalities that can be “plausibly inferred”… = ??

• For example:

– “bringing with it the disease” disease(d)

– “Disease: An illness caused by an infection.” disease(d) infection(i)

– “Within twenty four hours of infection…” infection(i)

– Zadrozny & Jensen: This equality i=i’ cannot be proven, but may be reasonably

assumed as disease illness -causes infection.

• Really means “connect using some background knowledge”

• NB: Here equating constants, not relating predicates

• Too open-ended? Restrict to just synonyms

“A ship entered the port bringing with it the disease known as the Black Death”

“Port: A harbor”

“Bring: To carry”“Disease: An illness caused by an infection”

“Enter: something comes into a place”

“It struck rapidly” “Strike (example): They were struck by an illness”

= coreferences= coherence links (c-links)

Putting it all together:The (best) p-model for the paragraph

4. Other Metalevel Contraints

• Avoid redundancy (penalize theories with redundant rules in)– “the captain is worried because the officer can open his safe”

– Redundant if “his” = “the officer”, so penalize this interpretation

• Gricean Cooperative Principles:– Quantity (don’t say too much or too little)

– Quality (be true)

– Relation (be relevant)

– Manner (be orderly)

• Connectives: “but” implies exceptions will follow – “the yacht is cheap, but it is elegant”

– Here prefer a definition/implication which is violated by the second part of the sentence

Again help decide which possible theory to prefer