“downstepped contours in the given/new distinction”
DESCRIPTION
On the Role of Prosody in Structuring Discourse October 5, 2005 - Berlin, Germany. “Downstepped contours in the given/new distinction”. Agust ín Gravano Spoken Language Processing Group Columbia University, New York. Participants in this project. - PowerPoint PPT PresentationTRANSCRIPT
“Downstepped contours in the given/new distinction”
Agustín Gravano
Spoken Language Processing Group
Columbia University, New York
On the Role of Prosody in Structuring DiscourseOctober 5, 2005 - Berlin, Germany
2
Participants in this project
Columbia University (New York)Julia HirschbergStefan BenusAgustín Gravano
Northwestern University (Chicago)Gregory WardElisa Sneed
Agustín Gravano - Columbia University
3
1. Introductiona) ToBI
b) Discourse structure (Grosz & Sidner ’86)
c) Information status (Prince ’92)
d) Meaning of intonational contours
e) The downstepped contours
2. Boston Directions Corpusa) Description of the corpus
b) Downstep and discourse structure
c) Downstep and information status
3. Games Projecta) Description of the corpus
b) Ongoing and future research
Agustín Gravano - Columbia University
4
1. Introductiona) ToBI
b) Discourse structure (Grosz & Sidner ’86)
c) Information status (Prince ’92)
d) Meaning of intonational contours
e) The downstepped contours
2. Boston Directions Corpusa) Description of the corpus
b) Downstep and discourse structure
c) Downstep and information status
3. Games Projecta) Description of the corpus
b) Ongoing and future research
Agustín Gravano - Columbia University
5
To(nes and)B(reak)I(ndices)• Prosody annotation convention.
• Two tones: H and L, which may be combined (e.g. H+L)
• Devised originally for Standard American English, but ToBI standards also proposed for Japanese, German, Italian, Spanish, British, Australian English,....
• 4 tiers:
– orthographic tier: words
– break-index tier: degrees of junction
– tonal tier: pitch accents, phrase accents, boundary tones
– miscellaneous tier: disfluencies, non-speech sounds, etc.
Agustín Gravano - Columbia University
6
Discourse Structure (G&S ’86)
• Series of discourse segments, defined in terms of the speaker’s intentions: the discourse segment purpose (DSP).
• Let a, b: DSP,– a satisfaction-precedes b
iff a must first be achieved in order for b to succeed;– a dominates b
iff fulfilling b partly fulfills a.
Barbara Grosz & Candace Sidner, 1986. “Attention, intentions, and the structure of discourse.” Computational Linguistics 12(3): 175-204.
Agustín Gravano - Columbia University
Information Status (Prince ’92)
Ellen Prince, 1992. “The ZPG letter: Subjects, definiteness, and information-status.” In Discourse Description: Diverse Analyses of a Fund Raising Text, S. Thompson & W. Mann (eds.), 295-325, Philadelphia: John Benjamins B.V.
Agustín Gravano - Columbia University
Discourse {• Given
• New
Hearer{• Given• Inferrable• New
8
Multiple “meanings” of intonational contours
• “Declarative” contours (H* L- L%)– Statements– Wh-questions
• Rise-fall-rise contours (L*+H L- H%)– Uncertainty– Incredulity
• H* Downstepped contours (H* (!H*)+ L- (L%|H%)?)– Topic beginnings or endings?– “Given” information?
Agustín Gravano - Columbia University
9
Example: H* !H* !H* !H* L-H%
Agustín Gravano - Columbia University
10
Understanding the multiple uses of contours is useful and interesting
• In most TTS systems– ‘Standard’ declarative (H* L- L%) contour
over-used – ‘Given’ information deaccented too often
• The H* (!H*)+ L- (L%|H%)? contours might be used instead, if they are appropriate
Agustín Gravano - Columbia University
11
H* (!H*)+ L- (L%|H%)? in Standard American English
• Topic structure markers (Pierrehumbert & Hirschberg ’90)– Beginning and ending of topics– Professorial tone
• Givenness (Hirschberg & Pierrehumbert ’86, Ladd ’96, Dahan et al ’02)– “This material should already be familiar to you.”– Alternates with deaccenting – when?
Agustín Gravano - Columbia University
12
1. Introductiona) ToBI
b) Discourse structure (Grosz & Sidner ’86)
c) Information status (Prince ’92)
d) Meaning of intonational contours
e) The downstepped contours
2. Boston Directions Corpusa) Description of the corpus
b) Downstep and discourse structure
c) Downstep and information status
3. Games Projecta) Description of the corpus
b) Ongoing and future research
Agustín Gravano - Columbia University
13
1. Introductiona) ToBi
b) Discourse structure (Grosz & Sidner ’86)
c) Information status (Prince ’92)
d) Meaning of intonational contours
e) The downstepped contours
2. Boston Directions Corpusa) Description of the corpus
b) Downstep and discourse structure
c) Downstep and information status
3. Games Projecta) Description of the corpus
b) Ongoing and future research
Agustín Gravano - Columbia University
14
Boston Directions Corpus
• 4 speakers
• 9 increasingly complex direction-giving tasks
• Spontaneous speech transcribed and speakers returned and read
• ~67m spon; ~50m read
15
firstenter the Harvard Square T stopand buy a tokenthenproceed to get on theinboundumRed Lineuh subwayandtake the subwayfrom Harvard Squareto Central Squareand then to Kendall Squarethen get off the T
Agustín Gravano - Columbia University
Boston Directions Corpus
16
firstenter the Harvard Square T stopand buy a tokenthenproceed to get on theinboundumRed Lineuh subwayandtake the subwayfrom Harvard Squareto Central Squareand then to Kendall Squarethen get off the T
BDC - Discourse Structure
Agustín Gravano - Columbia University
17
firstenter the Harvard Square T stopand buy a tokenthenproceed to get on theinboundumRed Lineuh subwayandtake the subwayfrom Harvard Squareto Central Squareand then to Kendall Squarethen get off the T
BDC - Information Status
Discourse Given
Agustín Gravano - Columbia University
18
firstenter the Harvard Square T stopand buy a tokenthenproceed to get on theinboundumRed Lineuh subwayandtake the subwayfrom Harvard Squareto Central Squareand then to Kendall Squarethen get off the T
BDC - Information Status
Hearer Given Hearer Inferrable
Agustín Gravano - Columbia University
19
firstenter the Harvard Square T stopand buy a tokenthenproceed to get on theinboundumRed Lineuh subwayandtake the subwayfrom Harvard Squareto Central Squareand then to Kendall Squarethen get off the T
BDC - DS Contours
Agustín Gravano - Columbia University
20
Downstep and Discourse Structure
• Distribution of use of DS contours for signaling discourse structure?
• How frequently is discourse structure conveyed using DS contours?
• Does this differ by speaking style (read vs. spontaneous speech)?
• Is there notable speaker variation in either of these?
Agustín Gravano - Columbia University
21
Use of DS contoursfor discourse position
Contour Seg Beg Seg Final Total
H* (!H*)+ L- (L%,H%)?
88 (18%) 196 (40%) 488
Agustín Gravano - Columbia University
Contour Seg Beg Seg Final Total
H* (!H*)+ L- (L%,H%)?
131 (29%) 195 (43%) 451
Spontaneous:
Read:
22
Discourse position conveyedusing DS contours
Contour Seg Beg Seg Final
H* (!H*)+ L- (L%,H%)?
88 (11%) 196 (28%)
Total 825 (100%) 693 (100%)
Agustín Gravano - Columbia University
Contour Seg Beg Seg Final
H* (!H*)+ L- (L%,H%)?
131 (18%) 195 (31%)
Total 721 (100%) 635 (100%)
Spontaneous:
Read:
23
Speaker variability
• We found high variability (both in spontaneous and read speech) in:– Overall use of DS contours– Distribution of use of DS contours– Frequency with which discourse structure is
conveyed using DS contours
• Only exception:– Speakers employ ~40% or more of their DS
contours over Segment Final phrases.
Agustín Gravano - Columbia University
24
• Are DS contours used over given information, alternating with a deaccenting strategy?
• If so, when do speakers choose one strategy over another?
• Information status in the BDC data:– at the NP level (both discourse g/n and hearer g/i/n
status),– at the word level (discourse g/n status for individual
lexical items).
• Smaller corpus: only spontaneous data labeled.
Downstep andInformation Status
Agustín Gravano - Columbia University
25
Downstep andInformation StatusHearer Given
Hearer Inferrable
Hearer New
Discourse
Given
Discourse New
All deacc 52 (5%) 6 (2%) 3 (2%) 46 (8%) 15 (2%)
So
me
acce
nt
DS416 (39%)
200 (49%) 58 (45%) 261 (44%) 413 (44%)
Other DS 48 (5%) 25 (6%) 12 (9%) 32 (5%) 53 (6%)
Other 540 (51%) 175 (43%) 57 (44%) 257 (43%) 469 (49%)
Total 1056 (100%) 406 (100%) 130 (100%) 596 (100%) 950 (100%)
Spontaneous productions only.
Agustín Gravano - Columbia University
26
Downstep and Information StatusHearer Given
Hearer Inferrable
Hearer New
Discourse
Given
Discourse New
All deacc 45 (8%) 3 (4%) 0 (0%) 44 (8%) 4 (4%)
So
me
acce
nt
DS260 (45%)
38 (54%) 3 (33%) 251 (45%) 50 (52%)
Other DS 28 (5%) 2 (3%) 2 (22%) 28 (5%) 4 (4%)
Other 244 (42%) 27 (39%) 4 (44%) 237 (42%) 38 (40%)
Total 577 (100%) 70 (100%) 9 (100%) 560 (100%) 96 (100%)
Spon - Only NPs for which all lexical elements are Given.
Agustín Gravano - Columbia University
27
• DS contours clearly dominate Hearer-Inferrables.
• DS contours are commonly used over Given information.
• Little evidence from this study that information status is a major predictor of the use of DS contours: equally likely to be used over New NPs.
Downstep andInformation Status
Agustín Gravano - Columbia University
28
1. Introductiona) ToBI
b) Discourse structure (Grosz & Sidner ’86)
c) Information status (Prince ’92)
d) Meaning of intonational contours
e) The downstepped contours
2. Boston Directions Corpusa) Description of the corpus
b) Downstep and discourse structure
c) Downstep and information status
3. Games Projecta) Description of the corpus
b) Ongoing and future research
Agustín Gravano - Columbia University
29
1. Introductiona) ToBI
b) Discourse structure (Grosz & Sidner ’86)
c) Information status (Prince ’92)
d) Meaning of intonational contours
e) The downstepped contours
2. Boston Directions Corpusa) Description of the corpus
b) Downstep and discourse structure
c) Downstep and information status
3. Games Projecta) Description of the corpus
b) Ongoing and future research
Agustín Gravano - Columbia University
30
• Elicit a corpus of spontaneous dialogue containing:– given and new NPs– topic segmentation data
Games Project - Goal
Agustín Gravano - Columbia University
31
Games Project - Design
Session:– 3 collaborative computer games.– 2 players, each with an electronic game board.– Unrestricted speech.– No visual contact between subjects.– Subjects were paid a fixed amount of money, plus
a bonus based on their performance.– Each subject participated in 2 sessions with
different partners and on different days.
Agustín Gravano - Columbia University
PLAYER 1“DESCRIBER”
PLAYER 2“SEARCHER”
Game # 1
Agustín Gravano - Columbia University
PLAYER 1“DESCRIBER”
PLAYER 2“SEARCHER”
Game # 2
Agustín Gravano - Columbia University
PLAYER 1“DESCRIBER”
PLAYER 2“SEARCHER”
Game # 3
35
• Study the relation between the choice of intonational contours and:– givenness status of NPs– syntactic position of NPs– complexity of NPs– proportion of given lexical elements in new NPs– discourse structure
Games Project - Design
Agustín Gravano - Columbia University
36
• How?– Games 1 & 2:
• Cards have increasingly more features, increasing the complexity of NPs
• Some features appear more frequently, becoming “given”.• Features appear in different sizes.
– Game 3: • Subject blinking/target image.• Objects images surrounding the target image.
• Pretests
Games Project - Design
Agustín Gravano - Columbia University
37
Games Project - Corpus
Corpus: – Recorded in a sound-proof booth at Columbia’s
Speech Lab in October 2004.– 12 sessions.– ~20 hours of spontaneous speech.– Fluent dialogues, each game with very different
characteristics. – All dialogues have already been transcribed.– Currently doing ToBI labeling.
Agustín Gravano - Columbia University
38
• Ongoing studies– Discourse Markers (okay, mm-hm, yeah, etc.)– Turn-taking– Laughter
• Future studies– Use of the downstepped contour with respect to
discourse structure and info status.– Evolution of the description of lexical entities.– Disfluencies (false repairs, self-repairs, etc.)– …
Games Project - Studies
Agustín Gravano - Columbia University
39
1. Introductiona) ToBI
b) Discourse structure (Grosz & Sidner ’86)
c) Information status (Prince ’92)
d) Meaning of intonational contours
e) The downstepped contours
2. Boston Directions Corpusa) Description of the corpus
b) Downstep and discourse structure
c) Downstep and information status
3. Games Projecta) Description of the corpus
b) Ongoing and future research
Agustín Gravano - Columbia University
40
1. Introductiona) ToBI
b) Discourse structure (Grosz & Sidner ’86)
c) Information status (Prince ’92)
d) Meaning of intonational contours
e) The downstepped contours
2. Boston Directions Corpusa) Description of the corpus
b) Downstep and discourse structure
c) Downstep and information status
3. Games Projecta) Description of the corpus
b) Ongoing and future research
Agustín Gravano - Columbia University
“Downstepped contours in the given/new distinction”
Agustín Gravano
Spoken Language Processing Group
Columbia University, New York
On the Role of Prosody in Structuring DiscourseOctober 5, 2005 - Berlin, Germany