Discourse Coherence: Concurrent Explicit and Implicit Relations

Hannah Rohde1 Alexander Johnson1 Nathan Schneider2 Bonnie Webber11 University of Edinburgh 2 Georgetown University

{Hannah.Rohde, Bonnie.Webber}@ed.ac.uk, s1668719@ed-alumni.net, nathan.schneider@georgetown.edu

Abstract

Theories of discourse coherence posit rela-tions between discourse segments as a keyfeature of coherent text. Our prior worksuggests that multiple discourse relationscan be simultaneously operative betweentwo segments for reasons not predicted bythe literature. Here we test how this jointpresence can lead participants to endorseseemingly divergent conjunctions (e.g., butand so) to express the link they see betweentwo segments. These apparent divergencesare not symptomatic of participant naïvetéor bias, but arise reliably from the concur-rent availability of multiple relations be-tween segments – some available throughexplicit signals and some via inference. Webelieve that these new results can both in-form future progress in theoretical workon discourse coherence and lead to higherlevels of performance in discourse parsing.

1 Introduction

A question that remains unresolved in work ondiscourse coherence is the nature and number of re-lations that can hold between clauses in a coherenttext (Halliday and Hasan, 1976; Stede, 2012).

Our earlier work (Rohde et al., 2015, 2016)showed that, in the presence of explicit discourseadverbials, people also infer additional discourserelations that they take to hold jointly with thoseassociated with the adverbials. For example, in:

(1) It’s too far to walk. Instead let’s take the bus.

people infer a RESULT relation in the context ofthe adverbial instead, which itself signals that thebus stands in a SUBSTITUTION relation to walking.We showed this using crowdsourced conjunction-insertion experiments (Rohde et al., 2015, 2016), inwhich participants were asked to insert into the gapbetween two discourse segments, a conjunction that

best expressed how they took the segments to be re-lated. Rohde et al. (2017) also asked participants toselect any other conjunctions that they took to con-vey the same sense as their “best” choice. (Moredetails of these experiments are given in Section 3.)

All three studies showed participants selectingconjunctions whose sense differed from that of theexplicit discourse adverbial. But Rohde et al. (2015,2016) also showed participants often selecting con-junctions that signal different coherence relationsthan those selected by other participants. And Ro-hde et al. (2017) showed participants often identi-fying very different conjunctions as conveying thesame meaning. For example, in passage (2), withthe discourse adverbial in other words, one largefraction of participants chose to insert OR, whileanother large fraction inserted SO. Since the twoare neither synonymous nor representative of thesame relation, either the participants have come upwith different analyses of the passages (Section 2)or something more surprising is at work.

(2) Unfortunately, nearly 75,000 acres of tropical forestare converted or deforested every day ______ in otherwords an area the size of Central Park disappears every16 minutes. [SO∼OR]

Rohde et al. (2017) noted other cases where dif-ferent pairs of conjunctions (e.g., BECAUSE andBUT, BUT and OR, and BECAUSE and OR) ap-pear systematically across participants and acrosspassages for particular adverbials, and speculatedon what these odd pairings may reveal, but didnot provide any empirical evidence for why thishappens. Here we present such evidence from anexperiment on three discourse adverbials (in otherwords, otherwise, and instead).

After describing related work on multiple dis-course relations (Section 2) and then our experi-mental methodology (Section 3), we step throughresults for these three adverbials. As a final pieceof evidence, we manipulate the presence and ab-sence of a fourth adverbial, after all, in order to

demonstrate that inference of the relation(s) be-tween segments in a passage is not always drivenby the presence of such an adverbial.

2 Related Work

This is not the first work on discourse coherence toacknowledge the possibility of multiple relationsholding between given discourse segments.

For example, the developers of Rhetorical Struc-ture Theory acknowledged that even experiencedRST analysts may interpret a text differently interms of the relations they take to hold (Mann andThompson, 1988, p. 265). But while RST allowsfor multiple alternative analyses of a text in termsof discourse relations, in practice, researchers work-ing in the RST framework standardly produce asingle analysis of a text, with a single relationallabeling, selecting the analysis that is “most plau-sible in terms of the perceived goals of the writer”(Mann et al., 1989, pp. 34–35). If that single an-alysis is later mapped into a different structure tosupport further processing – e.g., a binary branch-ing tree structure – the mapping does not changethe chosen relational labeling.

Multiple relations may additionally hold in theo-ries of discourse coherence that posit multiple lev-els of text analysis. For example, following Groszand Sidner (1986), Moore and Pollack (1992) char-acterized text as having both an informational struc-ture (relating information conveyed by discoursesegments) and an intentional structure (relating thefunctions of those segments with respect to whatthe speaker is trying to accomplish through thetext). The kinds of relations at the two levels aredifferent, as can be seen in the following examplefrom (Moore and Pollack, 1992, p. 540):

(3) a. George Bush supports big business.b. He’s sure to veto House Bill 1711.

At the level of intentions, (3a) aims to provide EVI-DENCE for the claim in (3b), while at an informa-tional level, (3a) serves as the CAUSE of the situa-tion in (3b). RST would force annotators to chooseonly the analysis that best reflected the perceivedgoals of the writer.

Additionally, multiple relations can hold wherethere are distinct explicit signals for distinct dis-course relations holding between a pair of segments(Cuenca and Marin, 2009; Fraser, 2013), as in:

(4) It’s too far to walk. So instead let’s take the bus.

where the conjunction so signals a RESULT relationand the adverbial instead signals that taking the busstands in an SUBSTITUTION relation to walking.

Finally, a fourth way in which the previous lit-erature has taken multiple discourse relations tohold is when a single phrase or lexico-syntacticconstruction jointly signals multiple discourse re-lations as holding over a text – for example, sinceas a subordinating conjunction may, in particularcontexts, signal both a TEMPORAL relation and aCAUSAL relation, rather than just one or the other(Miltsakaki et al., 2005).

We are aware of only two resources that allowmore than one discourse relation to be annotatedbetween two segments – the Penn Discourse Tree-Bank (PDTB; Prasad et al., 2008, 2014) and, morerecently, the BECauSE Corpus 2.0 (Dunietz et al.,2017). The PDTB allows multiple discourse re-lations of the third and fourth types noted above.It also allows them to be annotated if there is noexplicit connective between a pair of segments butannotators see more than one sense relation as link-ing them, as in the following variant of (4):

(5) It’s too far to walk. Let’s take the bus.

Here a RESULT relation can be associated with animplicit token of so between the clauses, while aSUBSTITUTION relation can be associated with animplicit token of instead. The above are the maincases in which PDTB annotates multiple relations.Relevant to this paper, the PDTB does not anno-tate implicit conjunction relations where there isalready an explicit discourse adverbial. Thus thePDTB would either ignore the implicit RESULT re-lation for (1) or (incorrectly) annotate instead in (1)as conveying both SUBSTITUTION and RESULT.

Moreover, while the PDTB has been usedin training many (but not all) discourse parsers(Marcu, 2000; Lin et al., 2014; Feng and Hirst,2012; Xue et al., 2015, 2016; Ji and Eisenstein,2014), discourse parsing has for the most part ig-nored its annotations of multiple concurrent rela-tions between clauses, except in the case of distinctexplicit connectives expressing distinct relations.Instead, they have arbitrarily taken just a single re-lation to hold, even though the relations are simplyrecorded in an a priori canonical order. This prac-tice is problematic because, for example, there maywell be a difference in the properties of segmentswhere two relations are jointly seen to hold, ver-sus those segments in which only one or the otherholds. This can result in unwanted noise in the dataand lower the reliability of whatever is induced.

While our previous studies showed anothersource of multiple discourse relations holding con-

currently between discourse segments, the workreported here explains how, in the context of mul-tiple relations, participants can take very differentconjunctions to be conveying the same relation, andwhat can change participants’ selection of a con-junction to mark the relation they infer alongsidethat conveyed by an explicit discourse adverbial.

3 Methodology

A locally crowdsourced conjunction-insertion taskprovided a proxy for labelling relations betweenadjacent discourse segments within a passage.

Our materials consisted of passages containingan explicit discourse adverbial, preceded by a gap,which effectively separated the passage into twosegments. The passages consisted of 16 with inother words, 16 with instead, 16 with after all, and48 with otherwise. Participants were asked to readeach passage and choose the conjunction(s) thatbest expressed how the two segments link together.The presentation of conjunction choices varied inorder for each participant, but always consisted ofAND, BECAUSE, BUT, OR, SO, NONE. Whilethe task admittedly encourages participants to se-lect one (or more) conjunctions, our prior workhas shown that participants are very willing to useNONE if no conjunction is appropriate. We there-fore take their insertion of a conjunction as their en-dorsement of the relation signaled by that conjunc-tion. To further control data quality, we included6 catch trials with an expected correct conjunctionlike “To be ______ not to be”.

Three of the explicit discourse adverbials thatwe chose are anaphoric: in other words, other-wise, and instead (Webber et al., 2000). Unlikeconjunctions such as AND, BECAUSE, BUT, ORand SO, they are not constrained by structure asto what they establish discourse relations with. Soa conjunction-insertion task can be used to assesslinks between the segments (see also Scholman andDemberg 2017). Our three anaphoric adverbialsshare a core meaning of ‘otherness’ via their lexicalsemantics and flexibility in the relations they canparticipate in, making them a fruitful set to com-pare. The fourth adverbial, after all, allows us totest a hypothesis that the inferred connection be-tween clauses is not driven by the adverbial alone.

These particular adverbials were selected be-cause they had yielded unexpected combinationsof conjunction insertions in our prior work (e.g.,OR/SO with in other words). This is in con-

trast to adverbials like therefore and neverthe-less, for which participants’ conjunction combi-nations could be attributed to variation in the speci-ficity of the conjunctions (SO/AND for therefore,BUT/AND for nevertheless). For our selection ofa set of conjunctions to use as proxies for relationlabels, we included all the coordinating conjunc-tions in English, as well as the subordinating con-junction BECAUSE as EXPLANATION relations arefrequent.

All participants (N=28) were monolingual na-tive English speakers who were selected followinga pre-test to measure their ability to consistentlyinsert conjunctions that captured the underlying co-herence relations in a series of passages. All gaveinformed consent. They each received £50 for theirtime. Each participant saw one of two randomlyordered lists. Passages were presented in batchesof 34, one batch per day for three days.

The materials were simplified variants of natu-rally occurring passages. Some were also manipu-lated systematically, in ways aimed at altering theavailability of different coherence relations. Pas-sages are available via the “dataset” link on thepaper in the ACL anthology, and predictions aboutthem are laid out in Sections 4.1–4.4.

4 Datasets

4.1 In other words Dataset

Rohde et al. (2016) report an OR∼SO responsesplit for in other words when participants could in-sert only their top choice of conjunction. Figure 1shows SO dominating participants’ choice in allcases, but OR showing up among their choices inall but one passage (leftmost vertical bar). Addi-tionally, several passages elicited BUT as the topchoice of some participants.

0

7

14

21

28

andbecausebeforebutorsoothernoneinstead

in other words

0

7

14

21

28

andbecausebeforebutorsoothernone

0

7

14

21

28

andbecausebeforebutorsoothernoneotherwise

�1

Figure 1: Stacked bar chart for conjunction insertions inpassages with in other words (Rohde et al., 2016). Eachvertical bar represents a passage with one response from eachparticipant (N=28, no overlap with current participants).

The in other words passages of the current ex-periment tested two linked hypotheses: The first isthat OR∼SO response splits arise from two com-ponents of the lexical semantics of the adverbialitself: its sense of an evoked alternative and itssense of a consequence via restatement, wherebythe truth of the second segment holds because itprovides a reformulated restatement of the first seg-ment’s content. For passage (2), this corresponds tothe deforestation of 75,000 acres of tropical forestentailing the disappearance of an area the size ofCentral Park every 16 minutes.

The second hypothesis is that the prevalence ofand substitutability between SO and OR in (2) de-pends on the immediately adjacency of the two seg-ments. This was suggested by participant choicesof BUT (cf. Figure 1), as well as the observationthat in other words does not always license ORvia its lexical semantics and SO via entailment, asshown in (6), where BUT has become more avail-able. Note that none of the relations conveyed bythese conjunctions (CONTRAST or CONCESSION

for BUT, DISJUNCTION for OR, CONSEQUENCE

for SO) are already conveyed by the adverbial itself,which for in other words) would be RESTATEMENT.

(6) Unfortunately, nearly 75,000 acres of tropical forest areconverted or deforested every day. I don’t know whereI heard that ______ in other words an area the size ofCentral Park disappears every 16 minutes.

We tested these hypotheses by creating mini-mal pairs of 16 passages containing in other words.The pairs varied in the presence/absence of a meta-linguistic comment intervening between the origi-nal description and its reformulation, as in (7)–(8).

(7) Typically, a cast-iron wood-burning stove is 60 percentefficient ______ in other words 40 percent of the woodends up as ash, smoke or lost heat.

(8) Typically, a cast-iron wood-burning stove is 60 percentefficient. How this is measured is unclear ______ inother words 40 percent of the wood ends up as ash,smoke or lost heat.

For each passage, participants identified theirpreferred conjunction and then any others that theytook to convey the same sense. Half the partici-pants saw a given passage with no intervening meta-linguistic comment, half with.

If our hypotheses are confirmed, it will showthat manipulating the immediately preceding seg-ment can shift participants’ preference from rela-tions associated with OR and SO (ALTERNATIVE

and CONSEQUENCE) to relations of CONTRAST orCONCESSION. This would then be evidence thatadjacency affects what coherence relations partici-pants take to be available.

0

7

14

21

28

andbecausebeforebutorsoothernoneinstead

in other words

0

7

14

21

28

andbecausebeforebutorsoothernone

0

7

14

21

28

andbecausebeforebutorsoothernoneotherwise

�1

Figure 2: Stacked bar chart for participants’ (N=28) conjunc-tion insertions in otherwise passages (Rohde et al., 2016)

4.2 Otherwise DatasetRohde et al. (2016) report surprising response splitsamongst BECAUSE∼BUT∼OR for otherwise intheir conjunction-insertion data (Figure 2). Giventhat otherwise has several different functions (de-scribed below), we hypothesize that different re-sponse splits arise from the lexical semantics ofotherwise, combined with inference as to the func-tion of the otherwise clause in a given passage.

One function of otherwise is in ARGUMENTA-TION. Here, an otherwise clause provides a reasonfor a given claim, as in (9). Another function is inENUMERATION, when the speaker first gives somepreferred or more salient options, the otherwiseclause introduces other alternative options, as in(10). A third use is in expressing an EXCEPTION toa generalization. Here, the main clause expresses ageneralization, while otherwise clause specifies anexception (disjunctive alternative) to it, as in (11).

(9) Proper placement of the testing device is an importantissue ______ otherwise the test results will be inaccu-rate.

(10) A baked potato, plonked on a side plate with sourcream flecked with chives, is the perfect accompani-ment ______ otherwise you could serve a green saladand some good country bread.

(11) Mr. Lurie and Mr. Jarmusch actually catch a shark, athrashing 10-footer ______ otherwise the action is light.

Results presented in (Rohde et al., 2017) forpassages like (9) showed participant judgments ofOR and BECAUSE, but not BUT. Passages like(10) yielded pairings of OR and BUT, but notBECAUSE. Lastly, passages like (11) yielded re-sponse splits between BUT and the less specificAND (Knott, 1996).

Note that due to overlaps in conjunction choice,some conjunctions cannot be unambiguously as-sociated with a single use of otherwise: WhileBECAUSE may unambiguously signal that a parti-cipant has inferred ARGUMENTATION, OR mightindicate inference of either ARGUMENTATION or

ENUMERATION. Thus we probe both participantchoices of connectives and (via paraphrase) the useof otherwise that they take to hold.

We chose 16 passages for each use of otherwise,based on our own category judgments. For eachpassage, we asked participants to select the con-junction that best expressed how its two segmentswere related, and then any other connectives thatthey took to express the same thing.

A paraphrase task was then used as further ev-idence for the relation participants inferred in theotherwise passages. After completing a given ses-sion’s batch of passages, participants were askedto select which of three options they took to be avalid paraphrase of the passage. Each use of other-wise was assigned a distinct paraphrase to link theleft-hand and right-hand segments (LHS, RHS).

• ARGUMENTATION: “A reason for 〈LHS〉 is〈RHS〉.”

• EXCEPTION: “Generally 〈RHS〉. An excep-tion is when 〈LHS〉.”

• ENUMERATION: “There’s more than one goodoption for 〈goal〉. They are: 〈LHS〉, 〈RHS〉.”

We also allowed participants to choose a secondparaphrase if they thought it appropriate.

4.3 Instead DatasetRohde et al. (2016) report a range of participantchoices in conjunction-insertion passages involv-ing instead (Figure 3). For passages on the leftof the figure, participants uniformly chose BUT,while the passage on the far right yielded a strongpreference for SO. Elsewhere, some chose BUTand some chose SO. (For the current experiment,we ignore the fact that AND can contingently sub-stitute for either BUT or SO as a connective in text(Knott, 1996), focussing only on passages whereparticipants explicitly choose BUT and/or SO.)

Rohde et al. (2017) report even more surpris-ing participant responses to passages such as (12),where some participants selected both BUT andSO as equally expressing how the segments in thepassage were related.(12) There may not be a flight scheduled to Loja today

______ instead we can go to Cuenca. [BUT∼SO]

Neither the inter-participant split between BUTand SO in (Rohde et al., 2016) nor the intra-participant split between them (Rohde et al., 2017)can be explained in terms of instead itself, since

0

7

14

21

28

andbecausebeforebutorsoothernoneinstead

in other words

0

7

14

21

28

andbecausebeforebutorsoothernone

0

7

14

21

28

andbecausebeforebutorsoothernoneotherwise

�1

Figure 3: Stacked bar chart for participants’ (N=28) conjunc-tion insertions in instead passages (Rohde et al., 2016)

instead simply conveys that what follows is an al-ternative to an unrealised situation in the context(Prasad et al., 2008; Webber, 2013). The currentexperiment tests the hypothesis that this BUT∼SOsplit is a consequence of inference from propertiesof the segments themselves.

To test this hypothesis, we created 16 mini-mal pairs of passages containing instead, one ofwhich emphasized the information structural par-allelism between the clauses, as in (13a), and an-other variant (13b) that de-emphasized that par-allelism in favor of a causal link implied by adownward-entailing construction such as too X(Webber, 2013). For each passage, half the partici-pants saw the parallelism variant in the conjunction-insertion task, while half saw the causal variant.

(13) a. There was no flight scheduled to Loja yesterday______ instead there were several to Cuenca.

b. There were too few flights scheduled to Loja yester-day ______ instead we went to Cuenca.

4.4 After all DatasetIn (Rohde et al., 2017), we reported aBECAUSE∼BUT response split for passagescontaining after all. We speculated that this maybe because a passage such as (14) below presentsan argument in which the second segment servesas a REASON (hence, BECAUSE) for the firstsegment, but also serves to CONTRAST with it(hence, BUT).(14) Yes, I suppose there’s a certain element of danger in it

______ (after all) there’s a certain amount of danger inliving, whatever you do.

We hypothesize that the BECAUSE∼BUT splitcannot be a consequence of the adverbial after all,which the Cambridge Dictionary indicates is “usedto add information that shows that what you havejust said is true”.1 If REASON and/or CONTRAST

1https://dictionary.cambridge.org/us/dictionary/english/after-all

A B C D E F G H I J K L M N O Pno_intervening

with_intervening

no_intervening

with_intervening

no_intervening

with_intervening

no_intervening

with_intervening

no_intervening

with_intervening

no_intervening

with_intervening

no_intervening

with_intervening

no_intervening

with_intervening

no_intervening

with_intervening

no_intervening

with_intervening

no_intervening

with_intervening

no_intervening

with_intervening

no_intervening

with_intervening

no_intervening

with_intervening

no_intervening

with_intervening

no_intervening

with_intervening

0510

# re

spon

ses

ChoiceBUT

SO

AND

BECAUSE

OR

[no connective]

Figure 4: Distribution of participants’ first choice of conjunction for passages with in other words. Each participant saw onlyone variant. Each vertical bar represents a passage with the responses from each participant, color-coded by conjunction.

are being conveyed, it can’t be a consequence ofafter all. As such, this response split must dependon the reasoning that supports the inference of co-herence between the two segments, separate fromthe adverbial itself.

We test the hypothesis that the response split isindependent of the presence or absence of after all.Starting with 16 passages that originally containedafter all, we created a variant of each passage with-out the adverbial. The conjunction insertion taskwas the same as with the other datasets.

5 Results

5.1 In other words: Inference and adjacency

Section 4.1 lays out the joint hypotheses that in-ferred relations in passages with in other wordsreflect two components of the lexical semantics ofthe adverbial (leading to the OR∼SO split) and thatthe presence of intervening material before in otherwords reduces the availability of those relations,favoring BUT instead.

Figure 4 shows the predicted pattern: Theno-intervening-content condition primarily yieldsOR/SO responses (with variation across passageson the OR-vs.-SO preference) with a relative in-crease in BUT responses in the intervening-contentcondition.2 Passage B corresponds to the pair ofexamples (2)/(6), and passage C reflects (7)/(8).

For the analysis here and in Section 5.3, a rele-vant first-choice conjunction was chosen and thebinary outcome of its insertion was modeled with amixed-effect logistic regression. Here, the insertionof OR indeed varied with the presence/absence ofintervening material (β = −1.569, p < 0.005).

We posit that increases in BUT associated withthe intervening content indicate either an interrup-tion of the meta-linguistic tangent or an intentionto signal a contrast with the negative affect of the

2For Passage P in Figure 4, participants may have linkedthe in other words clause to the intervening material itself.

tangent itself (e.g., “I don’t know where. . . ”, “frus-trating way of putting it”, “how this is measured isunclear”). We speculate that the presence of BE-CAUSE in passages with intervening content mayarise when that content implies that the situationis somehow surprising, which in turn merits ex-planation (e.g., “it’s an UNUSUAL role for her”,“their ability to actually work sensitively is perhapsQUESTIONABLE”, “it’s STRANGE to think of aplanet being born”). These hypotheses will them-selves need to be tested.

5.2 Otherwise: Inference from semanticfeatures of segments

As noted in Section 4.2, passages containing oth-erwise were used to test how semantic propertiesof the segments themselves influenced conjunctionchoice. The categorization of passages by the re-searchers (16 ARGUMENTATION, 16 EXCEPTION,16 ENUMERATION) predicts the conjunctions cho-sen by participants. In aggregate, ≈99% of re-sponses to ARGUMENTATION passages were BE-CAUSE or OR or both. ≈92% of responses to EX-CEPTION passages were BUT, AND, or both BUTand AND. And ≈98% of responses to ENUMER-ATION passages were BUT, AND, OR, or somesubset thereof. For analysis, a mixed-effect lo-gistic regression modeled the binary outcome ofBUT insertion and showed significant variationacross the three categories (p < 0.001). This mea-sure captures the difference between pairs of cate-gories: ARGUMENTATION permits BECAUSE andOR (hence BUT is rare) while ENUMERATION per-mits BUT and OR (hence BUT is present) and EX-CEPTION favors BUT (hence BUT is very frequent).All pairwise comparisons yielded a main effect ofcategory on this dependent measure (p’s < 0.001).

Turning to individual passages, participantchoices are shown in Figures 5–7. For ARGUMEN-TATION (Figure 5), the effect is uniformly strong,with all passages showing BECAUSE or OR as

A B C D E F G H I J K L M N O P

first

second first

second first

second first

second first

second first

second first

second first

second first

second first

second first

second first

second first

second first

second first

second first

second

0

10

20

# re

spon

ses

ChoiceBUT

SO

AND

BECAUSE

OR

OR,BUT

OR,SO

OR,AND

OR,BECAUSE

AND,OR,BUT

AND,OR,SO

AND,OR,SO,BUT

[no connective]

Figure 5: Distribution of first and second choice conjunctions for ARGUMENTATION otherwise. Labels such as “OR,BUT” arefor multiple second choices. Each vertical bar represents a passage with the responses from each participant, color-coded byconjunction. Enlarged B/W versions of Figures 4–8 are available via the “notes” link on the paper in the ACL anthology.

A B C D E F G H I J K L M N O P

first

second first

second first

second first

second first

second first

second first

second first

second first

second first

second first

second first

second first

second first

second first

second first

second

0

10

20

# re

spon

ses

ChoiceBUT

SO

AND

BECAUSE

OR

OR,AND

[no connective]

Figure 6: Distribution of first and second choice conjunctions for EXCEPTION otherwise. The label “OR,AND” in the legendimplies both as second choices.

A B C D E F G H I J K L M N O P

first

second first

second first

second first

second first

second first

second first

second first

second first

second first

second first

second first

second first

second first

second first

second first

second

0

10

20

# re

spon

ses

ChoiceBUT

SO

AND

BECAUSE

OR

OR,BUT

SO,OR

OR,AND

AND,OR,SO

BUT,AND

[no connective]

Figure 7: Distribution of first and second choice conjunctions for ENUMERATION otherwise. Labels in the legend such as“SO,OR” are for multiple second choices.

A B C D E F G H I J K L M N O

parallel

non_parallel

parallel

non_parallel

parallel

non_parallel

parallel

non_parallel

parallel

non_parallel

parallel

non_parallel

parallel

non_parallel

parallel

non_parallel

parallel

non_parallel

parallel

non_parallel

parallel

non_parallel

parallel

non_parallel

parallel

non_parallel

parallel

non_parallel

parallel

non_parallel

0

5

10

# re

spon

ses

ChoiceBUT

SO

AND

BECAUSE

OR

[no connective]

Figure 8: Instead passages, pairing a parallel variant and a causal variant. Each column shows the distribution of participants’first choice in the conjunction-insertion task. Each participant saw only one variant.

participants’ top choice, with OR or BECAUSEchosen as equivalent (shown in the columns la-belled “second”). For EXCEPTION (Figure 6), BUTis consistently the participants’ top choice.

There are a few deviations from this near uni-form endorsement of BUT for EXCEPTION (Fig-ure 6, passages L–P). Any hypotheses, however,would require further experimentation to test. Forexample, in passage M (see (15)) and P (see (16)),participants rarely identified any conjunction asconveying the same sense as BUT. However, whentheir top choice was BECAUSE, they also selectedOR as conveying the same sense. As noted above,BECAUSE and OR predominate with otherwiseused in ARGUMENTATION. This raises the questionof why passages M and P lead some participantsto infer ARGUMENTATION and other participants,either EXCEPTION or ENUMERATION.

(15) Democrats insist that the poor should be the priority,and that tax relief should be directed at them _____otherwise they lack a cogent vision of the needs of anew economy.

(16) He said that the proposed bill would give states moreflexibility in deciding whether they wanted to use theFederal money for outright grants to municipalities orto set up loan programs _____ otherwise it left last fall’sCongressional legislation unchanged.

Finally, though the pattern for ENUMERATION

(Figure 7) is harder to see, combinations of BUT,OR and AND predominate as participants’ topchoices, with a few tokens of BECAUSE and SO,but too few to analyse as anything but noise.

The above results reflect researcher-assigned uselabels. However, the confusion matrix in Table 1shows that on the whole, participants agree with

that assignment. The column labelled Multipleis for cases where participants offered two para-phrases. For ARGUMENTATION, at least one para-phrase always corresponded to EXCEPTION, whilefor ENUMERATION, it did so for most of these to-kens (9/14). We comment on this below.

While there was less agreement when partici-pants offered multiple paraphrases for researcher-assigned EXCEPTION, there may be too few to-kens here to draw any kind of conclusion. In anycase, the results for ARGUMENTATION and ENU-MERATION agree both across participants (in whatparaphrase they choose when they don’t choosethe researcher-assigned label) and within partici-pants (in what pairs of paraphrases they gave forthe original passage).

The above results support our hypothesis thatvariability in participants’ choice of conjunctionsfollows from both the lexical semantics of other-wise and the relation that participants infer betweenthe segments in the passage.

5.3 Instead: Inference from a singlemanipulated property

On aggregate, participants responded very differ-ently to the parallel and causal variants of insteadpassages (cf. Section 4.3). Figure 8 shows that inall cases, the parallel variant yielded more BUT re-sponses, whereas the non-parallel (causal) variantyielded significantly more SO responses (main ef-fect of (non-)parallelism: β=−7.0008, p<0.001).3

Some of these results are very strong. For exam-ple, Passage A (17) drew all BUT responses for the

3We analyzed only 15 passages for instead and after all,due to a presentation error of the 16th for these adverbials.

Participant

Researcher ARGUMENTATION ENUMERATION EXCEPTION Multiple

ARGUMENTATION 401 (91.5%) 4 25 18ENUMERATION 23 364 (81.4%) 46 14EXCEPTION 21 29 393 (87.7%) 5

Table 1: Researcher labels assigned to otherwise passages vs. labels implied by participant paraphrases

A B C D E F G H I J K L M N O

with_adverbial

no_adverbial

with_adverbial

no_adverbial

with_adverbial

no_adverbial

with_adverbial

no_adverbial

with_adverbial

no_adverbial

with_adverbial

no_adverbial

with_adverbial

no_adverbial

with_adverbial

no_adverbial

with_adverbial

no_adverbial

with_adverbial

no_adverbial

with_adverbial

no_adverbial

with_adverbial

no_adverbial

with_adverbial

no_adverbial

with_adverbial

no_adverbial

with_adverbial

no_adverbial

0

5

10

# re

spon

ses

ChoiceBUT

SO

AND

BECAUSE

[no connective]

Figure 9: Distribution of first choice in conjunction selection task for passages with after all

parallel variant in (17a) and all SO responses forthe causal variant in (17b), as did Passage B. In afew cases, however, the parallel variant drew vari-able responses, even while its causal variant drewstrong SO responses. This is true of Passage O,with parallel and causal variants in (18a–b).

(17) a. They could have been playing football in the villagegreen _____ instead they played in the street.

b. They didn’t like playing football in the village green_____ instead they played in the street.

(18) a. Smugglers nowadays don’t use overland passages_____ instead they use the seas to transport theirgoods.

b. Smugglers’ overland passages nowadays are toovisible _____ instead they use the seas to transporttheir goods.

One possible explanation is that participants var-ied in the role they assigned to the positive claim inthe second segment of (18a) – either as a reason forthe negative claim in the first segment (BECAUSE),as a contrast with that claim (BUT), or as its result(SO). Although manipulating the segment to en-hance either parallelism or causality can changeparticipant responses, it is clear that parallelismalone doesn’t guarantee contrast.

5.4 After all: Adverb adds little to inferenceFigure 9 shows participant choice of conjunctionwhen after all is present and when it is absent.Their choice is largely the same for passages A–Fand K–N, with and without the adverbial. As forpassage O, since AND can contingently substitutefor BUT (Knott, 1996), the response pattern canbe considered the same as well. A by-passage cor-relation between the rate of BUT and BECAUSEresponses across the two conditions confirms thissimilarity (R2=.70, F(1,13)=30.98, p<0.001). Theoutlier is passage G:(19) There was a testy moment driving over the George Wash-

ington Bridge when the toll-taker charged him $24 forhis truck and trailer _____ after all it was New York.

With after all, the majority of participants choseBUT as best expressing how the two segments areconnected, while without it, the majority chose BE-CAUSE. Whatever explanation we gave here wouldbe pure speculation. We trust that the fact that theother 14 passages demonstrate the predicted effectprovides sufficient evidence that splits in partici-pant responses are not simply a result of the pres-ence of a discourse adverbial.

6 Conclusion

While our previous work showed that multiple dis-course relations can hold between two segments– relations at the same semantic level, simultane-ously available to a reader – we provided no evi-dence as to what influences the particular relationsthat are taken to be available. Our current experi-ments have provided some such evidence. Specif-ically, we have shown that participant responsesto systematically manipulated passages involvingdiscourse adverbials can be explained in terms ofboth the lexical semantics of discourse adverbialsand properties of the passages that contain them.As the conjunctions chosen by participants con-vey senses that differ from those of the discourseadverbials, we also provided evidence for the simul-taneous availability of multiple coherence relationsthat arise from both explicit signals and inference.We hope the reader is now convinced that, in bothpsycholinguistic research on discourse coherenceand computational work on discourse parsing, oneneeds to identify and examine evidence for coher-ence involving more than one discourse relation.

Acknowledgments

This project has been supported by a grant fromthe Nuance Foundation. In addition, a LeverhulmeTrust Prize in Languages & Literatures to H. Rohdehas enabled her to devote more time to research.We thank Amir Zeldes and the anonymous review-ers for their helpful feedback on the paper.

ReferencesM.J. Cuenca and M.J. Marin. 2009. Co-occurrence of

discourse markers in Catalan and Spanish oral narra-tive. Journal of Pragmatics, 41(5):899–914.

Jesse Dunietz, Lori Levin, and Jaime Carbonell. 2017.The BECauSE Corpus 2.0: Annotating causality andoverlapping relations. In Proceedings of the 11thLinguistic Annotation Workshop, pages 95–104, Va-lencia, Spain.

Vanessa Wei Feng and Graeme Hirst. 2012. Text-leveldiscourse parsing with rich linguistic features. InProceedings of the 50th Annual Meeting of the Asso-ciation for Computational Linguistics, pages 60–68,Jeju Island, Korea.

Bruce Fraser. 2013. Combinations of contrastive dis-course markers in English. International Review ofPragmatics, 5:318–340.

Barbara Grosz and Candace Sidner. 1986. Attention,intention and the structure of discourse. Computa-tional Linguistics, 12(3):175–204.

Michael Halliday and Ruqaiya Hasan. 1976. Cohesionin English. Longman.

Yangfeng Ji and Jacob Eisenstein. 2014. Representa-tion learning for text-level discourse parsing. In Pro-ceedings of the 52nd Annual Meeting of the Associa-tion for Computational Linguistics, volume 1, pages13–24, Baltimore, Maryland.

Alistair Knott. 1996. A Data-driven Methodology forMotivating a Set of Coherence Relations. Ph.D. dis-sertation, Department of Artificial Intelligence, Uni-versity of Edinburgh.

Ziheng Lin, Hwee Tou Ng, and Min-Yen Kan. 2014. APDTB-styled end-to-end discourse parser. NaturalLanguage Engineering, pages 151–184.

William C. Mann, Christian M. I. M. Matthiessen, andSandra A. Thompson. 1989. Rhetorical structuretheory and text analysis. Technical Report ISI/RR-89-242, USC-ISI, Marina del Rey CA.

William C. Mann and Sandra A. Thompson. 1988.Rhetorical Structure Theory: Toward a functionaltheory of text organization. Text, 8(3):243–281.

Daniel Marcu. 2000. The theory and practice of dis-course parsing and summarization. MIT Press.

Eleni Miltsakaki, Nikhil Dinesh, Rashmi Prasad, Ar-avind Joshi, and Bonnie Webber. 2005. Experi-ments on sense annotation and sense disambigua-tion of discourse connectives. In Proceedings of theFourth Workshop on Treebanks and Linguistic Theo-ries (TLT’05), Barcelona, Spain.

Johanna Moore and Martha Pollack. 1992. A problemfor RST: The need for multi-level discouse analysis.Computational Linguistics, 18(4):537–544.

Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Milt-sakaki, Livio Robaldo, Aravind Joshi, and Bon-nie Webber. 2008. The Penn Discourse TreeBank2.0. In Proceedings of the 6th International Confer-ence on Language Resources and Evaluation, pages2961–2968, Marrakech, Morocco.

Rashmi Prasad, Bonnie Webber, and Aravind Joshi.2014. Reflections on the Penn Discourse TreeBank,comparable corpora and complementary annotation.Computational Linguistics, 40(4):921–950.

Hannah Rohde, Anna Dickinson, Chris Clark, AnnieLouis, and Bonnie Webber. 2015. Recovering dis-course relations: Varying influence of discourse ad-verbials. In Proceedings of the First Workshop onLinking Computational Models of Lexical, Senten-tial and Discourse-level Semantics, pages 22–31,Lisbon, Portugal.

Hannah Rohde, Anna Dickinson, Nathan Schneider,Christopher Clark, Annie Louis, and Bonnie Web-ber. 2016. Filling in the blanks in understand-ing discourse adverbials: Consistency, conflict, andcontext-dependence in a crowdsourced elicitationtask. In Proceedings of the Tenth Linguistic Anno-tation Workshop (LAW-X), pages 49–58, Berlin, Ger-many.

Hannah Rohde, Anna Dickinson, Nathan Schneider,Annie Louis, and Bonnie Webber. 2017. Explor-ing substitutability through discourse adverbials andmultiple judgments. In Proceedings of the 12thInternational Conference on Computational Seman-tics (IWCS), Montpellier, France.

Merel Scholman and Vera Demberg. 2017. Crowd-sourcing discourse interpretations: On the influenceof context and the reliability of a connective inser-tion task. In Proceedings of the 11th Linguistic An-notation Workshop, pages 24–33, Valencia, Spain.

Manfred Stede. 2012. Discourse Processing. Morgan& Claypool Publishers.

Bonnie Webber. 2013. What excludes an alternativein coherence relations? In Proceedings of the 10thInternational Conference on Computational Seman-tics, Potsdam, Germany.

Bonnie Webber, Aravind Joshi, and Alistair Knott.2000. The anaphoric nature of certain discourse con-nectives. In Making Sense: From Lexeme to Dis-course, Groningen, The Netherlands.

Nianwen Xue, Hwee Tou Ng, Sameer Pradhan, RashmiPrasad, Christopher Bryant, and Attapol T. Ruther-ford. 2015. The CoNLL-2015 Shared Task onShallow Discourse Parsing. In Proceedings of theNineteenth Conference on Computational LanguageLearning: Shared Task, pages 1–16, Beijing, China.

Nianwen Xue, Hwee Tou Ng, Sameer Pradhan, At-tapol Rutherford, Bonnie Webber, Chuan Wang, andHongmin Wang. 2016. CoNLL 2016 Shared Task

on Multilingual Shallow Discourse Parsing. In Pro-ceedings of the 20th Conference on ComputationalNatural Language Learning – Shared Task, pages 1–19, Berlin, Germany.