the role of representation in the interpretation of

The Role of Representation in

the Interpretation of

Representational Noun Phrases

Jeffrey T. RunnerUniversity of Rochester

[email protected]

UT Austin March 27, 2006

March 27, 2006 Jeffrey T. Runner 2

Thanks to

� Collaborators

� Micah Goldwater, University of Texas at Austin

� Elsi Kaiser, University of Southern California

� Rachel S. Sussman, University of Rochester

� Michael K. Tanenhaus, University of Rochester

� Research Assistants

� Karen Bobrow, Sasha Eloi, Leslianne LaVallee, Rebekka Puderpaugh, Ryan Schmitt, Travis Thompson


Introduction

� For the past seven years we have been investigating the interpretation and structure of “picture” noun phrases (PNPs)

� NPs headed by a “representational” noun

� picture, film, photograph, novel, etc.

� May also contain a possessor phrase and PP, both of which are optional;

� Harry’s picture of Joe

� a picture of Joe, Harry’s picture, etc.


Introduction

� Our main focus has been on PNPscontaining pronouns and reflexives

� (1) a picture of him/a picture of himself

� (2) John’s picture of him/John’s picture of himself

� In this talk I’ll focus on PNPs lacking possessor phrases (as in (1))

� Reflexives in PNPs have received a lot of attention in the literature

� Ebenezer saw a picture of himself in the paper.


Introduction

� Reflexives in PNPs are able to take antecedents beyond their minimal clause

� John said that [S there was [a picture of himself] in the post office]

� (a) John was going to get even with Mary.

� (b) That picture of himself in the paper would really annoy her, as would the other stunts he had planned. (Pollard & Sag, 1992)

� Seem to be sensitive to certain pragmatic / discourse constraints. In place of (a):

� (a’) Mary was quite taken aback by the publicity Johnwas receiving.


Two proposals for PNP reflexives

� Exempt anaphor/“logophor” approach

� Binding Theory is defined to constrain reflexives with local co-arguments (Pollard & Sag, 1992; Reinhart & Reuland, 1993)

� Ebenezer saw himself

� ‘see’ ARG-ST < Ebenezeri, himselfi >

� Ebenezer saw a picture of himself

� ‘picture’ ARG-ST < himself >

� “Exempt” anaphor is not constrained by BT

� Instead sensitive to pragmatic/discourse constraints



� PRO-in-NP approach

� PNP contains null possessor which binds the reflexive (Chomsky, 1986; Davies & Dubinsky, 2003)

� Ebenezer saw a picture of himself

� Ebenezer saw [PROi picture of himselfi]

� PRO is a pronominal that picks up its reference from the discourse context

� Ebenezeri saw [PROi picture of himselfi]

� Apparent sensitivity to discourse is due to PRO, not reflexive itself



� Both approaches attribute the behavior of PNP reflexives to a structural factor� Exempt anaphor view: particular argument structure of picture noun

� PRO-in-NP view: null PRO possessor inside PNP

� Two goals� To try to tease apart the predictions of these approaches and test them experimentally (next)

� To test another possible explanation for the behavior of PNP reflexives (see way below)


Methodology

� Intuitive judgments form the foundation for collecting data for linguistic theorizing

� We aim to complement intuitive judgment data with experimentally collected data

� In some cases—especially when both structural and non-structural factors (e.g., discourse, visual context) interact—reliable and stable judgments can be difficult

� We try to manipulate structural and non-structural factors to determine the contribution of each


Exempt anaphor vs. PRO-in-NP

� Later come back to exempt anaphor approach

� First present data relevant to the PRO-in-NP approach


Kuno’s source

� Kaiser, Runner, Sussman & Tanenhaus (in press), building on observations of Kuno(1987)

� Johni told Billj about the picture of himselfi/?j on the wall

� Johni heard from Billj about the picture of himself?i/j on the wall

� Kuno’s observation: reflexive likes ‘source’of information

� Source: subject of ‘tell’, object of ‘hear’


Tenny’s perceiver

� Kaiser, Runner, Sussman & Tanenhaus (in press), building on observations of Tenny(2003)

� Johni told Billj about the picture of him?i/j on the wall

� Johni heard from Billj about the picture of himi/?j on the wall

� Tenny’s observation: pronoun likes ‘perceiver’ of information

� Perceiver: object of ‘tell’, subject of ‘hear’


Source/perceiver experiments

� Source: subject of ‘tell’, object of ‘hear’

� Perceiver: object of ‘tell’, subject of ‘hear’

� We designed two experiments to test the Kuno/Tenny observations

� Experiment 1 Picture verification

� Experiment 2 Eye-tracking

� Participants seated in front of computer monitor

� Experiment 1: had to indicate if sentence they heard matched the scene on the screen

� Experiment 2: had to click on the picture mentioned in the sentence they heard


Experiment 1 picture verification

� Participants saw scenes while listening to sentences

� Scene contained two characters below a picture of one of the them

� Sentences contained names of characters and reflexive/pronoun

� Participants indicated whether sentence matched scene by pressing ‘y’ or ‘n’ on keyboard

� Fully counterbalanced (see Kaiser et al. (in press) for details)



� Conditions

� Subject as Source/Object as Perceiver

� Peter told Andrew about the picture of {him/himself} on the wall.

� Object as Source/Subject as Perceiver

� Peter heard from Andrew about the picture of {him/himself} on the wall.



� Conditions







� “Yes” response data� Pronouns

� Preference for object (Pron Obj > Pron Subj)� Significantly modulated by source/perceiver distinction

� Pronoun prefers perceiver to source (confirming Tenny’s observation)

� Reflexives� Strong preference for subject (Refl Subj > Refl Obj)� Smaller (but significant) effect of source/perceiver distinction

� Small preference for source over perceiver (Kuno’s observation)

020406080100

Pron Subj Pron Obj Refl Subj Refl Obj

Source

Perceiver



� Discussion

� Pronouns in picture NPs do show strong preference for perceiver (Tenny)

� Reflexives in picture NPs show a weak preference for source (Kuno)

� Structural preference for subject still evident for reflexives


Experiment 2 eye-tracking

� Participants saw scenes while listening to sentences

� Scenes contained two characters and a picture of each character

� Sentences contained characters’ names and pronoun/reflexive

� Participants clicked on the picture they thought the sentence described

� We monitored their eye movements with head-mounted eye-tracker



� Conditions







� Target choice data (what participants clicked on)� Pronouns

� Slight preference for object (Pron Obj > Pron Subj)� Significantly modulated by source/perceiver distinction

� Pronoun strongly prefers perceiver to source (Tenny’s observation)

� Reflexives� Strong preference for subject (Refl Subj > Refl Obj)� Numerical effect of source/perceiver distinction

� Tiny (n.s.) preference for source over perceiver (Kuno’s observation)

020406080100

Pron Subj Pron Obj Refl Subj Refl Obj

Source

Perceiver



� Eye-movement data� Y-axis: proportion of looks to particular item

� X-axis: time in frames (30 frames/sec)

� Pronoun condition: Proportion of looks to subject picture (left)

� Reflexive condition: Proportion of looks to object picture (right)� ‘Hear’: filled circles

� ‘Tell’: open circles

0

.1

.2

.3

.4

.5

.6

.7

0 5 10 15 20 25 30 35 40 45 50 55 60 65 70

t

h

Pronouns: Proportion of looks to subject-picture

(i.e. non-BT-compatible referent).

0

.05

.1

.15

.2

.25

.3

.35

.4

.45

.5

0 5 10 15 20 25 30 35 40 45 50 55 60 65 70

t

h

Reflexives: Proportion of looks to object-picture

(i.e. non-BT-compatible referent)



� Pronoun results� More looks to subject picture on ‘hear’ trials

� =more looks to perceiver than source (Tenny)

� Reflexive results� More looks to object picture on ‘hear’ trials

� =more looks to source than perceiver (Kuno)

0

.1

.2

.3

.4

.5

.6

.7

0 5 10 15 20 25 30 35 40 45 50 55 60 65 70

t

h

Pronouns: Proportion of looks to subject-picture

(i.e. non-BT-compatible referent).

0

.05

.1

.15

.2

.25

.3

.35

.4

.45

.5

0 5 10 15 20 25 30 35 40 45 50 55 60 65 70

t

h

Reflexives: Proportion of looks to object-picture

(i.e. non-BT-compatible referent)



� Discussion� Target choice

� Pronouns in picture NPs show strong preference for perceiver (Tenny)

� Reflexives in picture NPs show strong preference for subject

� No significant effect of preference for source (Kuno)

� Eye movements� More looks to perceiver on pronoun trials

� More looks to source on reflexive trials

� Eye movements reveal source effect for reflexives, even though target choices do not


Summary

� Both experiments

� Pronouns prefer perceiver

� object of ‘tell’, subject of ‘hear’

� Reflexives show some preference for source

� subject of ‘tell’, object of ‘hear’


PRO-in-NP approach� To account for pronoun/reflexive preferences

� PRO in NP must be sensitive to source/perceiver distinction

� What would that mean?

� Since pronoun prefers perceiver and pronoun is disjoint from PRO

� =PRO disprefers perceiver (=prefers source)

� Since reflexive prefers source and reflexive is bound to PRO� =PRO prefers source

� To illustrate:

� Subject as Source (underlined)/Object as Perceiver

� Peteri told Andrewj about [PROi picture of himj /himselfi] on the wall.

� Object as Source (underlined)/Subject as Perceiver

� Peteri heard from Andrewj about [PROj picture of himi/himselfj] on the wall.


Overt possessor experiment

� To test the PRO-in-NP hypothesis we need to know how overt possessive pronouns are interpreted

� If they are interpreted as preferring source then this would provide striking corroboration of the PRO-in-NP hypothesis

� Experiment 3� Picture verification

� Preliminary results from 16 subjects

� Eye-tracking version in progress



� Conditions


� Peter told Andrew about his picture on the wall.


� Peter heard from Andrew about his picture on the wall.


PRO-in-NP predictions

� PRO prefers source


� Peteri told Andrew about hisi picture on the wall.


� Peter heard from Andrewi about hisi picture on the wall.


Results – ‘yes’ responses

� ‘told’: no preference for source (subject)

� ‘heard’: preference for perceiver (subject)

� Reverse of what PRO-in-NP predicts, if PRO and ‘his’ are similar.

0

0.2

0.4

0.6

0.8

1

told heard

subject

object



� Discussion

� Overt possessor shows no preference for source or perceiver with ‘tell’

� Overt possessor shows preference for perceiver with ‘hear’

� PRO-in-NP approach predicts possessor should prefer source

� For PRO-in-NP approach to be maintained, an explanation for the differences in interpretation assigned PRO vs. overt ‘his’ needs to be worked out.


Exempt anaphor approach

� Exempt anaphor/“logophor” approach� Binding Theory is defined to constrain reflexives with local co-arguments (Pollard & Sag, 1992; Reinhart & Reuland, 1993)

� Ebenezer saw himself� ‘see’ ARG-ST < Ebenezeri, himselfi >� Ebenezer saw a picture of himself� ‘picture’ ARG-ST < himself >

� “Exempt” anaphor is not constrained by Binding Theory� Instead sensitive to pragmatic/discourse constraints (including preference for source)

� Exempt anaphor approach blames noun argument structure for exempt anaphor behavior


Exempt anaphor approach

� Exempt anaphor approach treats picture NP reflexives as free from Binding Theory and constrained to pick up reference from the discourse—like pronouns

� PNP reflexives are like pronouns

� Can be interpreted “coreferentially” or as “bound variables”

� True reflexives are always bound variables


Ellipsis in PNPs

� Ellipsis has been used to reveal differences between bound variable and coreferential interpretations.

� “true” reflexive:� John hates himself, and so does Fred.

� Elided VP is interpreted as � ‘Fred hates himself’ (bound variable), � Not ‘Fred hates John’ (coreferential) � Elided reflexive interpreted as a bound variable only.

� PNP reflexive:� John has a picture of himself, and so does Fred.

� Elided VP is interpreted as� ‘Fred has a picture of himself’ (bound variable) or� ‘Fred has a picture of John’ (coreferential).


Statue interpretations

� Jackendoff (1992)

� Transfer of reference to representation

� Can refer to statue of Ringo Starr as “Ringo Starr”

� True referent can bind statue reflexive

� Ringo Starr saw himself at the museum.

� Statue referent cannot bind true reflexive

� Ringo Starr fell on himself. (not: The statue of RS fell onto the true RS.)


Statue interpretations

� Lidz (2001)

� Statue reflexives (“near reflexives”) can receive coreferential interpretations

� Ringo saw himself at the museum.

� Ringo saw himself at the museum and Paul did, too.

� Ringo saw the statue of Ringo and Paul saw the statue of Ringo too.

� Note: coreferential readings not usually possible with OBJ NPs.


Statue reflexive morphemes

� Lidz (2001), Reuland (2001)

� Languages sometimes have special reflexive morphemes that distinguish between “true”reflexives and reflexives that can receive statue interpretations


Statue reflexives and PNP reflexives

� Statue reflexives

� Can receive coreferential interpretations

� Are phrases that refer to representations of their antecedents.

� Have OBJ form (=are “true” direct objects)

� PNP reflexives

� Can receive coreferential interpretations

� Are phrases that refer to representations of their antecedents.

� Have PNP form (=are in picture NPs)


Unified approach?

� Question

� Could statue reflexives and PNP reflexives be the same thing?

� Could the fact that they both refer to representations of their antecedents be the reason that they both receive coreferential interpretations?

� Implications

� If so, this would have enormous implications for the treatment of PNP reflexives:

� Their behavior would not depend on the structure of the PNP itself

� either the argument structure or the null PRO possessor


PNP vs OBJ experiment

� Goldwater & Runner (2006) tested the claim that OBJ reflexives must be bound variables and that PNP reflexives can be both bound variables and coreferential anaphora

� Using ellipsis

� OBJ condition (with OBJ scene)� Mike is pointing at himself.

� PNP condition (with PNP scene)� Mike is pointing at a picture of himself.


PNP vs OBJ experiment

� 23 participants (University of Rochester undergraduates)

� Seated in front of a computer monitor

� Listened to prerecorded sentences

� Verified if displayed scene matched the sentences heard by pushing a yes or nobutton

� Three verbs used: point, tap, touch

� Fillers, counterbalancing, etc…


Ellipsis OBJ condition

� Jose is pointing at himself, and Mike is too.




Bound Variable




Bound Variable Coreferential




� Predictions� If OBJ reflexive must be bound variable, participants should reject coreferential interpretation required by Coreferential scene

� Bound variable scene: mostly ‘yes’ responses

� Coreferential scene: mostly ‘no’ responses



Ellipsis PNP condition

� Jose is pointing at a picture of himself, and Mike is too




Bound Variable



� Jose is pointing at a picture of himself, and Mike is too� Predictions

� If PNP reflexive can receive coreferential interpretation, participants should accept coreferential interpretation with Coreferential scene

� Bound variable scene: mostly ‘yes’ responses� Coreferential scene: (at least) some ‘yes’ responses



Summary of predictions

� OBJ condition (OBJ scene)� Bound variable: mostly ‘yes’

� Coreferential: mostly ‘no’

� PNP condition (PNP scene)� Bound variable: mostly ‘yes’

� Coreferential: some ‘yes’

� Results presented as proportion of coreferential interpretations� ‘yes’ to Coreferential scene

� ‘no’ to Bound Variable scene


Ellipsis results: proportion of

coreferential interpretations

0

0.05

0.1

0.15

0.2

0.25

0.3

Coref BV

PNP

OBJ

*

n.s.

p<.01




0

0.05

0.1

0.15

0.2

0.25

0.3

Coref BV

PNP

OBJ

*

n.s.

p<.01

PNP can receive coreferential interpretationOBJ receives bound variable interpretation


Summary

� Ellipsis

� Object reflexives receive bound variable interpretation

� PNP reflexives receive both bound variable and coreferential interpretations


Reference Transfer vs OBJ experiment

� In the previous experiment the linguistic and visual stimuli were correlated.

� OBJ scene went with OBJ sentence

� PNP scene went with PNP sentence

� We wanted to tease apart the effects of the linguistic and visual stimuli

� Same scenes (OBJ and PNP)

� Only OBJ sentences

� OBJ scene with OBJ sentence =OBJ condition

� PNP scene with OBJ sentence =Reference Transfer condition



� Again, using ellipsis

� OBJ condition (with OBJ scene)

� Mike is pointing at himself.

� Reference Transfer (RT) condition (with PNP scene)

� Mike is pointing at himself.



� 37 participants (University of Rochester undergraduates)

� Seated in front of a computer monitor

� Listened to prerecorded sentences

� Verified if displayed scene matched the sentences heard by pushing a yes or nobutton

� Three verbs used: point, tap, touch

� Fillers, counterbalancing, etc…


Predictions

� Exempt anaphor approach� Coreferential interpretations available because of exempt anaphor—due to noun argument structure

� Without PNP structure, few coreferentialinterpretations predicted

� Reference transfer� Coreferential interpretations due to reference transfer

� Coreferential interpretations predicted even without PNP structure




Bound Variable




� Predictions� If OBJ reflexive must be bound variable, participants should reject coreferential interpretation required by Coreferential scene

� Bound variable scene: mostly ‘yes’ responses

� Coreferential scene: mostly ‘no’ responses



Ellipsis RT condition

� Jose is pointing at himself, and Mike is too




Bound Variable



� Jose is pointing at himself, and Mike is too� Predictions (of Reference Transfer approach)

� If RT reflexive can receive coreferential interpretation, participants should accept coreferential interpretation with Coreferential scene

� Bound variable scene: mostly ‘yes’ responses� Coreferential scene: (at least) some ‘yes’ responses



Summary of Reference Transfer

predictions

� OBJ condition

� Bound variable: mostly ‘yes’

� Coreferential: mostly ‘no’

� RT condition

� Bound variable: mostly ‘yes’

� Coreferential: some ‘yes’




0

0.05

0.1

0.15

0.2

0.25

0.3

Coref BV

RT

OBJ

p<.04




0

0.05

0.1

0.15

0.2

0.25

0.3

Coref BV

RT

OBJ

p<.04

Overall, RT reflexive received more coreferentialinterpretations, regardless of scene


Experiments compared -- ellipsis

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

Coref BV

PNP

RT

Proportion of coreferential interpretations for picture scenes does not differ for PNP and RT experiments


Summary

� PNP vs. OBJ (structure + scene vary)

� PNP reflexives can receive coreferentialinterpretations

� RT vs. OBJ (scene alone varies)

� RT reflexives can receive coreferential interpretation


Discussion

� There is a difference between PNP and OBJ reflexives when structure correlated with scene (e.g., PNP structure and PNP scene)

� But, there is still a difference between the interpretation of reflexives depending on scene.� Independent of structure of NP� With reference transfer use more likely to get coreferential interpretation

� Provides preliminary evidence that structure alone is not “causing” special behavior of PNP reflexives.

� Reflexives referring to pictures are special even when not supported by PNP structure

� Undermines exempt anaphor approach


Future directions

� Currently setting up eye-tracking version of PNP vs. OBJ and RT vs. OBJ experiments (Goldwater & Runner)

� Currently running eye-tracking version of prenominal possessor experiment (Kaiser, Runner, Sussman & Tanenhaus)

� Experiments in planning stages to test multiple reflexives and pronouns in other languages, which have multiple reflexives (Runner & Kaiser--Dutch, Swedish, Finnish)


Take home message

� Some of the data needed to evaluate our syntactic theories is hard to get reliably

� Experimental approaches can help by systematically manipulating both structural and non-structural factors

� Data from different sources should complement each other, ultimately providing fuller picture of what’s going on


Selected References� Goldwater, M. & Runner, J.T. (in press). Coreferential interpretations of reflexives in picture noun phrases: an experimental approach. In P. Denis, E. McCready, A. Palmer & B. Reese (Eds.) Proceedings of the Texas Linguistics Society 8. Austin, TX: University of Texas.

� Grodzinsky, Y. & Reinhart, T. (1993). The innateness of binding and coreference. Linguistic Inquiry, 24, 69-101.

� Pollard, C. & Sag, I. (1992). Anaphors in English and the scope of Binding Theory. Linguistic Inquiry, 23, 261-303.

� Reinhart, T. & Reuland, E. (1993). Reflexivity. Linguistic Inquiry, 24, 657-720.

� Runner, J.T., Sussman, R.S. and Tanenhaus, M.K. (2003). Assignment of reference to reflexives and pronouns in picture noun phrases: Evidence from eye movements. Cognition, 81.1, B1-B13.

� Runner, J.T., R.S. Sussman, & M.K. Tanenhaus. To appear. Processing Reflexives and Pronouns in Picture Noun Phrases, Cognitive Science.

the role of representation in the interpretation of

Documents