systematic mismatches across annotations alan lee and aravind joshi institute for research in...

Systematic Mismatches Across Annotations

Alan Lee and Aravind Joshi

Institute for Research in Cognitive Science & Department of Computer and Information

Science, University of Pennsylvania

ULA Workshop,

U of Colorado, Boulder

March 2008

Preliminaries…

• We observe that certain annotated features of the Penn Discourse Treebank 2.0 (PDTB) do not match up neatly with annotations at the syntactic level.

• What do certain mismatches suggest for linguistic theory? How do we get from syntax to discourse?

• How does this affect NLP applications?

Outline

1. Attribution spans

2. Parallel Connectives

3. AltLex

4. Polarity and Determinacy

Outline

1. Attribution spans

2. Parallel Connectives

3. AltLex

4. Polarity and Determinacy

Attribution SpansAttribution Spans

Relation between agents and abstract objects (discourse relations or their arguments)

Annotation: Text Spans and Four features (source, type, polarity, determinacy). More on the features later.

The company says it is talking with several prospects.attribution

There have been no orders for the Cray-3 so far, though the company says it is talking with several prospects. Discourse semantics: contrary-to-expectation relation between “there being no

orders for the Cray-3” and “there being a possibility of some prospects”.

Sentence semantics: contrary-to-expectation relation between “there being no orders for the Cray-3” and “the company saying something”.

S

SBAR-ADV

IN S

NP VP

have been no Orders for the Cray-3

ThereVP

though

the company

says it is talkingWith severalprospects

NP VP

V S

Discourse argumentsSyntactic arguments

Although takeover experts said they doubted Mr. Steinberg will make a bid by himself, the application by his Reliance Group Holdings Inc. could signal his interest in helping revive a failed labor-management bid.

Discourse semantics: contrary-to-expectation relation between “Mr. Steinberg not making a bid by himself” and “the RGH application signaling his bidding interest”.

Sentence semantics: contrary-to-expectation relation between “experts saying something” and “the RGH application signaling Mr. Steinberg’s bidding interest”.

SBAR-ADV

Although

takeoverexperts

said

Mr. Steinberg will make a bidby himself

the application by his RGH Inc.

SBAR

IN S

NP-SBJ

could

signal

his interest in helping revivea failed labor-management bid

NP-SBJ

VPMD

VP

VB NP

VBD

S

VP

NP-SBJ VP

VBDthey

doubted

SBAR

• Mismatches occur with other relations as well, such as causal relations:

Investors are nervous about the issue because they say the company's ability to meet debt payments is dependent on too many variables, including the sale of assets and the need to mortgage property to retire some existing debt.

Discourse semantics: causal relation between “investors being nervous” and “problems with the company’s ability to meet debt payments”

Sentence semantics: causal relation between “investors being nervous” and “investors saying something”!

How to address mismatch?

• One possibility - treat attribution as a different layer of structure in discourse. (and also in syntax?)

• This has the effect of reducing the complexity of the discourse structure.

Discourse Graphbank (Wolf & Gibson 2005)

1. Farm prices in October edged up 0.7% from September

2. as raw milk prices continued their rise,3. the Agriculture Department said.4. Milk sold to the nation's dairy plants and dealers

averaged $14.50 for each hundred pounds,5. up 50 cents from September and up $1.50 from

October 1988,6. the department said.

1 2 3 4 5

1-2 4-5

attr

sim

elab

ce elab

attr

6

ce - cause/effect; elab - elaboration; sim - similiarity; attr - atribution

1 2 3,attr 4 5

1-2 4-5

elab

ce elab6,attr

ce - cause/effect; elab - elaboration; [ sim - similiarity; attr - atribution ]

Residual issues

Even if B.A.T receives approval for the restructuring, the company will remain in play, say shareholders and analysts, though the situation may unfold over the next 12 months, rather than six.

Does attribution scope over the entire relation, or just Arg1?

Guideline: in case of doubt, attribute to the Writer

Arg1: attributed to shareholders and analystsRel and Arg2: attributed to Writer

• Attribution cannot always be excluded by default

Advocates said the 90-cent-an-hour rise, to $4.25 an hour by April 1991, is too small for the working poor, while opponents argued that the increase will still hurt small business and cost many thousands of jobs.

Residual issues

What implications does this have for the approach of treating attribution as an independent layer of discourse?

Outline

1. Attribution spans

2. Parallel Connectives

3. AltLex

4. Polarity and Determinacy

Parallel ConnectivesParallel Connectives

Either he wasn’t being real in the past or he isn’t being real right now. (1549)

You’ve either got a chair or you don’t. (2428)

If the answers to these questions are affirmative, then these institutional investors are

likely to be favorably disposed toward a specific poison pill. (0275)

Parallel connectives are annotated discontinuously

In the PDTB, both parts of a parallel connective are treated as equally prominent (no hierarchical relationship)

Either he wasn’t being real in the past or he isn’t being real right now. (wsj_1549)

S

S SCCCC

Either he wasn’t beingreal in the past

or he isn’t beingreal right now

In Penn Treebank, the treatment of a parallel connective dependson its position within sentence. When “Either” is sentence-initial, both “either” and “or” are annotated as CC.

You’ve either got a chair or you don’t. (wsj_2428)

S

S SCC

You

oryou don’t

This is not possible when “either” is sentence-medial. Here, “either” is treated as an RB and “or” is as a CC.

NP-SBJ VP ADVP VP

‘ve got a chair

either

RB

How to represent parallel connective? DL-TAG approach: elementary discourse tree with

two lexical anchors (DC = discourse clause)

Either DC or DCbecauseDC DC

DCDC

But question remains: how to transition from syntactic structure to discourse structure?

Outline

1. Attribution spans

2. Parallel Connectives

3. AltLex

4. Polarity and Determinacy

Alternative LexicalizationAlternative Lexicalization(AltLex)(AltLex)

A discourse relation is inferred between two sentences which do not contain an Explicit connective, but insertion of an Implicit connective leads to redundancy. This is because the relation is alternatively lexicalized by some non-connective expression:

Under a post-1987 crash reform, the Chicago Mercantile Exchange wouldn’t permit the December S&P futures to fall further than 12 points for a half hour. AltLex = (consequence) That caused a brief period of panic seeling of stocks on the Big Board.

Discourse Connectives and Discourse Connectives and Syntactic ConstituencySyntactic Constituency

• Most explicit connectives correspond to syntactic constituencies. E.g. (“because” IN, “but” CC, “as a result” PP, etc.)

• Some small exceptions with parallel connectives, as we

have seen.

AltLex expressions often do not correspond to syntactic constituencies.

Under a post-1987 crash reform, the Chicago Mercantile Exchange wouldn’t permit the December S&P futures to fall further than 12 points for a half hour. AltLex = (consequence) That caused a brief period of panic selling of stocks on the Big Board.

S

NP-SBJ VP

VBDDT DT PP-LOC

That caused a brief period

of panicselling…..

For a list of AltLex expressions annotated in

the PDTB:

http://www.seas.upenn.edu/~pdtb/altlex-strings.txt

Or search using PDTB Browser (shameless

plug) :

http://www.seas.upenn.edu/~pdtb/PDTBAPI/pdtbbrowser.jnlp

Outline

1. Attribution spans

2. Parallel Connectives

3. AltLex

4. Polarity and Determinacy

Attribution FeaturesAttribution Features

Attribution is annotated on relations and arguments, with FOURFeatures.

Source: encodes the different agents to whom proposition is attributed• Wr: Writer agent• Ot: Other non-writer agent• Arb: Generic/Atbitrary non-writer agent• Inh: Used only for arguments; attribution inherited from relation

Type: encodes different types of Abstract Objects• Comm: Verbs of communication• PAtt: Verbs of propositional attitude• Ftv: Factive verbs• Ctrl: Control verbs• Null: Used only for arguments with no explicit attribution

Polarity vs DeterminacyPolarity vs DeterminacyPolarity: Indicates narrow scope of surface negated attributions.(Neg-raising, Klima 1964). Marked as “Neg” when neg-raisingoccurs. “Null” otherwise.

John doesn’t think the book fell (> John thinks thebook didn’t fall)

Determinacy: Attributions rendered indeterminate in certaincontexts. Marked as “Indet”, or “Null” otherwise.

John didn’t say the book fell (> no lowering of negation)

Only a certain class of verbs can have negative polarity,i.e. induce neg-raising. Verbs of Propositional Attitude (PAtt)have this behavior, but not others.

Polarity vs Determinacy…Polarity vs Determinacy…

I don’t believe they have the culture to adequately service high-net-worth individuals. (0927)

Discourse semantics: I believe they DO NOT have the culture to adequately service high-net-worth individuals. (0927)

Negation of “expect” is lowered onto the argument. The attribution is marked as negative polarity.

• Note that the attribution event of “expecting” did occur (is determinate).

Polarity vs Determinacy…Polarity vs Determinacy…

It didn’t say if it’s earlier results were influenced significantly by nonrecurring elements. (1711)

• Negation of “say” is NOT lowered onto the argument. The attribution is marked as indeterminate.

• The attribution event (of “saying”) did not actually occur.

At Syntactic Level…At Syntactic Level…At which level should discrepancy in the “polarity” vs “determinacy” type ofnegation be captured?

- In PropBank, negations of attribution verbs are uniformly marked as anegative feature for the adjunct feature “ARGM”. - In TimeML, they contain a polarity feature of “Neg”.

I don’t BELIEVE they have the culture to adequately service high-net-worth individuals.

ARG1 IARG2 they have the culture…ARGM Neg (PropBank) No Neg for lower predicate “have”POLARITY Neg (TimeML)

Should the negation be marked as ARGM for the lower predicate(“have”) instead?

It didn’t SAY if it’s earlier results were influenced significantlyby nonrecurring elements.

ARG1 It ARG2 if it’s earlier results were influenced significantly

by nonrecurring elements ARGM Neg (PropBank)POLARITY Neg (TimeML)

“Saying” event is indeterminate. Does this still count as an event?How to order this temporally?

At Syntactic Level…At Syntactic Level…

Some questions…

• How much of discourse is “projected” from syntax?

• Is there a need for a different architecture, different building blocks?

• How are these issues manifested cross-linguistically? Currently, discourse annotation work being done for Hindi, Turkish, Czech and Finnish (possibly).