the interaction of spatial reference frames and...

Behrmann and Plaut 1

The Interaction of Spatial Reference Frames andHierarchical Object Representations:

Evidence from Figure Copying in Hemispatial Neglect

Marlene BehrmannDepartment of PsychologyCarnegie Mellon UniversityPittsburgh, PA 15213--3890

[email protected]

and

David C. PlautDepartments of Psychology and Computer Science

Carnegie Mellon University, andCenter for the Neural Basis of Cognition

Pittsburgh, PA [email protected]

Key words: hemispatial neglect, computational account, spatial reference frames

Acknowledgements: This work was supported by grants from NIH to MB (MH54766 and MH54246), aWeston Visiting Professorship at the Weizmann Institute of Science and a James McKeen Cattell sabbaticalaward to MB, and a Fulbright Fellowship to DCP. We thank Jeffrey Beng-Hee Ho for his contribution to thiswork.

Please note: The address for correspondence until the end of June is Department of Computer Science,Weizmann Institute of Science, Rehovot, Israel, 76100. Email: [email protected]: 972-8-934-2945


Abstract

In copying or drawing a figure, patients with hemispatial neglect following right parietal-lobe

lesions typically produce an adequate representation of parts on the right of the figure while omitting the

corresponding features on the left. The neglect of information occupying contralateral locations is

influenced by multiple spatial reference frames and by the hierarchical structure of the object(s) in the

figure. The current work presents a computational characterization of the interaction among these

influences to account for the way in which neglect manifests in copying. Empirical data are initially

collected from brain-damaged and normal control subjects during two figure-copying tasks in which the

hierarchical complexity and orientation of the displays to be copied are manipulated. In the context of the

model, neglect is simulated by a “lesion” (monotonic drop-off along gradient from right to left) that can

affect performance in both object- and viewer-centered reference frames. The effect of neglect in both these

frames, coupled with the hierarchical representation of the object(s), provide a coherent account of the

copying behavior of the patients and may be extended to account for the copying performance of other

patients across a range of objects and scenes.


Introduction

Hemispatial or unilateral neglect is a visuospatial deficit, typically acquired after brain damage to

the right posterior parietal lobe, in which patients fail to perceive or act on information that appears on the

side of space opposite the lesion. Because neglect occurs more frequently and with greater severity after

right than left hemisphere lesions, we refer to neglect as left-sided in this paper. While patients with left-

sided neglect have normal intellectual abilities and intact primary motor and sensory functions, they may

not notice objects on the left, may leave food untouched on the left side of the plate, and may not shave or

bathe the left side of the body (for recent reviews, see Bisiach & Vallar, 2000; McGlinchey-Berroth, 1997;

Vallar, 1998). Neglect is thought to occur because neurons in one hemisphere have predominant, although

not exclusive representation of the contralateral side of space; removing neurons therefore impairs spatial

representations for contralateral positions to a greater extent than for ipsilateral positions (Pouget & Driver,

2000; Rizzolatti, Berti, & Gallese, 2000). The direct consequence of damaging these neurons is that

information appearing on the relative left is poorly activated in comparison with more rightward

information and hence, is neglected (Cate & Behrmann, 2001; Kinsbourne, 1987; Kinsbourne, 1993; Pouget &

Driver, 2000; Smania et al., 1998).

One of the best examples of neglect comes from the performance of patients on copying or drawing

tasks. As shown in Figure 1, during copying, patients routinely omit features on the left, while incorporating

the corresponding features on the right. This phenomenon is so typical that such tasks are frequently used to

diagnose the presence of neglect and are considered to be especially sensitive to the deficit (Black et al.,

1994). In this paper, we make use of a standard task, that of copying daisies, to explore the mechanisms

which give rise to hemispatial neglect.

INSERT FIGURE 1 APPROXIMATELY HERE

A key issue to be addressed in understanding neglect is to specify what constitutes “left”---that is,

With respect to what frame of reference is the left side defined such that information to the left of the

midline is neglected? Possible reference frames include those whose origin and axes are defined with

respect to the midline of the viewer (e.g., aligned with the gaze, head orientation or trunk of the viewer),

environment (e.g., based on landmarks such as the walls of a room, or defined gravitationally), or object

(e.g., determined by the intrinsic characteristics of objects such as principal axes of elongation or symmetry).

Under most viewing conditions, these frames are all aligned, so there is no way to evaluate which reference


frame determines the spatial coding of stimuli and of the subsequent neglect behavior. Recent evidence,

however, obtained from a host of neuropsychological studies suggests that neglect behavior is sensitive to

spatial information defined simultaneously with respect to multiple reference frames. In light of this

evidence, it becomes important to understand how information coded in these different reference frames is

integrated to yield coherent behavior.

In this paper, we examine specifically how information appearing on the left, defined in multiple

reference frames simultaneously, manifests in the patients’ figure-copying performance. We focus on two

major reference frames, one defined egocentrically or by the midline of the viewer, and the other defined

allocentrically, by the midline of an object. In addition, we explore how spatial coding is affected by the

complexity of the object being copied. We start off by reviewing current findings that support the presence

of these two forms of reference frame in neglect. Then, in the first experiment, we present empirical data on

a task in which the neglect patients copy a single daisy presented in differing orientations. To account for

these data, we formulate a computational account of the way in which the activation of spatial information,

defined in multiple reference frames, may be synthesized to subserve behavior, and we present data

obtained from such a computational demonstration. We extend the account in the second experiment, by

including empirical data on a more complex figure-copying task and by showing that the critical

assumptions underlying the computational model are sufficient to account for the neglect performance

under these more challenging conditions.

Object-centered neglect

Evidence for neglect that is defined with respect to the midline of the viewer is well-established and

not particularly controversial. For example, there is general consensus that early visual information is

encoded with respect to retinal location and gaze direction (Andersen, Essick, & Siegel, 1985; Colby &

Goldberg, 1999), and there now exist numerous studies reporting neglect for spatial information appearing

to the left of the retinal, head and/or trunk midline (for example, Bartolomeo & Chokron, 1999; Behrmann,

Ghiselli-Crippa, Sweeney, Dimatteo, & Kass, in press; Beschin, Cubelli, Della Sala, & Spinazzola, 1997;

Bisiach, Capitani, & Porta, 1985; Chokron & Imbert, 1995; Hillis & Rapp, 1998; Karnath, Schenkel, & Fisher,

1991; Kooistra & Heilman, 1989; Vuilleumier, Valenza, Mayer, Perrig, & Landis, 1999).


The more controversial question concerns the role of a reference frame centered on the midline of an

individual object such that spatial position is located with respect to a representation that depends on the

object’s extent, shape or motion. Within such a representation, the relationship of object parts is defined with

respect to each other, allocentrically and independent of the viewer’s position. Many recent studies have

examined whether neglect occurs for information appearing to the left of a midline defined by an individual

object. The result of many, although not all such studies (Farah, Brunn, Wong, Wallace, & Carpenter, 1990),

is that patients fail to report information appearing to the left of the object midline and this may be so even

when this information is located to the right of midline of the viewer and/or the environment (Behrmann &

Moscovitch, 1994; Behrmann & Tipper, 1994; Driver & Halligan, 1991; Humphreys & Riddoch, 1994a;

Pavlovskaya, Glass, Soroker, Blum, & Groswasser, 1997; Young, Hellawell, & Welch, 1992).

One of the earliest documented examples of object-based neglect is from patient NG, who had right-

sided neglect and who failed to read the rightmost letters of a word. This was true when the word was

presented vertically, in mirror-reversed format and even when she was required to spell words backwards

(Caramazza & Hillis, 1990a; Caramazza & Hillis, 1990b; Hillis, Rapp, Benzing, & Caramazza, 1998). Arguin

and Bub (1993) also showed that their patient’s inability to report a target letter in a horizontal array of four

elements depended on the object-relative position of the letter not the viewer-relative position. In a series of

studies, Humphreys, Riddoch and their colleagues have also documented object-based neglect, showing

that patients neglect letters positioned to the left of individual words (Humphreys & Riddoch, 1994a;

Humphreys & Riddoch, 1994b; Riddoch, Humphreys, Luckhurst, Burroughs, & Bateman, 1995).

Interestingly, these same patients show neglect for information on the right in multiple-stimulus displays

simultaneous with the object-based effects, providing support for accounts that posit the involvement of

multiple spatial frames and coding between- as well as within-objects (see Haywood & Coltheart, 2000, for

further discussion of neglect dyslexia and other examples of object-based findings).

Although all of the studies cited above use letters or words as stimuli, object-based neglect has also

been reported in studies that use other types of stimuli. For example, Young, Newcombe, Haan, & Ellis,

(1990) reported that their patient performed poorly at identifying the left half of chimeric faces even when

the faces were presented upside down and the relative left chimera occupied a position on the right side of

space, again suggesting that the left of the object is disadvantaged even when it appears on the right of the

viewer. The studies of Pavlovskaya, Glass, Soroker, Blum, & Groswasser (1997) and Grabowecky,


Robertson, & Treisman (1993) used geometric shapes and showed that information falling to the left of the

center of mass of an object was less well detected than information appearing to the right. These data

presuppose a computation of a center-of-mass that is specific to the object, the subsequent determination of

the object midline, and the neglect of information to the left of this midline (see also, Driver, Baylis,

Goodrich, & Rafal, 1994; Driver & Halligan, 1991). Consistent with this, using a barbell stimulus with

differently colored ends, Behrmann and Tipper (Behrmann & Tipper, 1994; Behrmann & Tipper, 1999;

Tipper & Behrmann, 1996) reported that the left of the barbell was still more poorly processed when it

appeared in the right of the viewer or of the environment. A final finding which is consistent with object-

centered coding is that, depending on the region to be searched in a visual search task, patients show neglect

defined by the borders of the relevant region (Karnath & Niemeier, 2001). When subjects searched a large

array, subtending 180 degrees, they showed significant neglect for the left side of the array. When subjects

searched only a subset of this large array constituting a 40 degree extent on the patient’s right side and

demarcated by having items in the relevant region displayed in a particular colour, patients neglected the

left of this small, right-sided segment even though this area was well searched initially.

The findings of left-right coding with respect to the object midline are also consistent with data

from studies with nonhuman primates. Both single neuron recording studies and lesion studies indicate a

neural selectivity for one side of an object (also see Reuter-Lorenz, Drain & Morais for related data from

normal subjects). For example, neural recordings obtained when monkeys saccade to the relative left or right

side of an object show directional selectivity that is independent of the retinal position of the object or the

orbital direction of the saccade (Olson, 2001; Olson & Gettner, 1996; Olson & Gettner, 1995; Olson, Gettner,

& Tremblay, 1999). Instead, this selectivity suggests that the neural coding is for a particular side of the

object, as coded intrinsically by the object (see Deneve & Pouget, 1997; Sabes, Breznen, & Andersen, 2001, for

an alternative view).

Our approach: copying objects with hierarchical representations

The goal of this paper is to explore the contribution of an object-centered spatial representation in

neglect and to examine how this might coexist with the well-established viewer-based neglect. We do so by

combining empirical and computational approaches in the context of a figure-copying task in an attempt to

determine which parts are included and which neglected by various patients. Copying has been used


previously to characterize object-centered neglect, although the findings from these studies have not been

without criticism. For example, Gainotti, Messerli, & Tissot (1972) have published illustrations depicting

neglect of the left side of several objects that were present in a scene: for example, the patient omitted the left

of a house, while copying the right of a tree that was located further to the left of the house. Marshall and

Halligan (1993) have also used a figure-copying task to show how neglect may manifest in viewer- and/or

object-based coordinates and we consider their findings in detail later.

Although the presence of object-based neglect under the conditions of figure copying is provocative,

this conclusion may not be entirely warranted (Driver & Halligan, 1991). Because drawing is a sequential

task, with each object being the sole focus of attention for some period of time, the section being drawn

becomes the entire environment, and so neglect may be determined by environment rather than object

coordinates under these conditions. It is difficult, then, to determine the contribution of an object-based

reference frame under conditions of free copying and free viewing. To circumvent this problem, we had

neglect patients copy a single daisy presented in four different orientations, as shown in Figure 2, so as to

disambiguate the left and right of the object from the left and right of other coordinate frames. It has also

been suggested that, under conditions of misorientation, it is crucial to disambiguate the intrinsic left and

right to maintain an object’s identity (for example, differentiating between the shape as a square or diamond;

Attneave, 1971) and an object-frame may be invoked here to achieve this end.


Like most natural objects, the single daisy we employ has a hierarchical structure so that parts of the

object are in themselves objects at a smaller spatial scale, and these then decompose further into their own

parts at an even smaller scale (Marr & Nishihara, 1978; Palmer, 1977). During the copying of a hierarchical

figure like this, then, a reference frame aligned with the midline of a subpart of the object serves as the

context frame for locating and drawing its subparts. Thus, the object-centered frame is not fixed throughout

the task; rather, objects are recursively decomposed and dynamically assigned to roles as objects and parts

depending on the current relevant level of the hierarchy (Hinton, 1990). Accounting for the copying

performance of neglect patients (and of normal subjects) is complicated, therefore, because, at one point in

time, the context frame may represent the spatial coordinates for copying a particular part, whereas at a

second point in time, this same part may itself define the context frame for the copying of its own subparts.

Importantly, it is commonly assumed that long-term hierarchical object representations are used to structure


drawing (Taylor & Tversky, 1992) and that these representations are the same as those that mediate

perception (Kosslyn, 1987; Van Sommers, 1989). In the case of the daisy, we assume that the hierarchical

representation is composed of three major parts (parents), each of which can be broken down into their

subparts (children) (see Figure 3). These children are decomposed further---for example, the central stem

decomposes into the oblique stems which break down further to encompass the leaves. The representation

used in this study has, in total, four levels, as illustrated in Figure 3. We did not break down simple

geometric forms into individual strokes (such as the pot or the daisy head) as we assumed that principles of

perceptual organization would be sufficiently strong to maintain the grouping and closure of primitive

elements and to resist neglect (Vuilleumier, Valenza, & Landis, 2001).

To verify that this hierarchical object representation adequately captures normal copying

performance, we had 20 normal subjects generate three copies of the target daisies presented in each of four

orientations (upright, 90o rotation to the left or right, and inverted; n=240) and we tracked the order of the

strokes. Copying performance was considered to obey the hierarchical representation if the order in which

the components were drawn followed a depth-first traversal order through the hierarchy (ignoring the order

among subparts). In other words, once a stroke within a particular subtree is drawn, all of its components

and subcomponents must be drawn before a stroke within another subtree at the same level is drawn. Any

stroke that did not adhere to this rule was counted as a violation of the hierarchy. In an analysis of variance,

with daisy orientation as a variable and mean violations per subject as the dependent measure, the mean

number of hierarchy violations was 1.3 (SD 0.84), and was not significantly affected by the orientation of the

daisy (F<1). We compared this number of violations against that obtained from 120 randomly-generated

stroke sequences (mean 17.2; SD 2.6) in a one-way ANOVA and obtained a highly reliable difference

between the distributions, F[1,238]=3953, p<.001. This difference suggests that the normal performance is

not random and, instead, is orderly and based on traversing a hierarchical representation such as the one

shown in Figure 3. As such, this supports our assumptions about the internal structure of the daisy and we

use this hierarchy in the algorithm we adopt.



Experiment 1: Neglect for misoriented single daisies and computational implementation

In this first experiment, we present copying data from patients with neglect, using the same upright

and misoriented daisies as those used for the normal subjects. We then attempt to explain the neglect by

implementing the copying performance via a conventional tree-traversal algorithm over a hierarchical data

structure representing the daisy (as in Figure 3). We do so by imposing a spatially defined lesion, analogous

to the deficit hypothesized to underlie the attentional impairment in patients with right-parietal damage and

then evaluate the performance of the model and its fit to the empirical data.

Method.

We first describe the individual subjects and the methods we used to obtain the empirical data.

Following this, we describe the methods employed for the computational simulations and present the

human and computational results together.

Subjects.

Two neglect patients participated in this experiment. The presence of neglect was initially defined

by performance on a bedside battery consisting of line bisection, target cancellation, drawing and copying

(Black et al., 1994). Performance on this battery is measured in relation to boundaries established by control

subjects. Where performance deviates from the norm, points are awarded and then, based on the final

aggregated score, severity of neglect is determined. The total is 100 points and the normal cutoff is 5 points.

JM, a 52 year old right-handed male, suffered an extensive right middle cerebral artery infarction in

June 1992, affecting the right parietal cortex as well as the anterior portion of the thalamus. Although he

exhibited a left homonymous hemianopia initially, this had resolved by the time of this testing. JM is also

mildly hemiparetic on the left although he walks unassisted. He was self-employed as an engineer until the

time of his stroke but has not returned to work. He has participated in several other experiments

(Behrmann, Ghiselli-Crippa, & Di Matteo, 2001; Behrmann et al., in press; Philbeck, Behrmann, Black, &

Ebert, 2000) and the reader is referred to those papers for additional biographical and lesion details. JM

obtained a neglect score of 69/100, indicative of neglect in the moderate to severe range.

GS is a 65 year old right-handed male who was admitted to hospital in early January 1996 following

a history of hypertension and an incident of left upper extremity weakness and nausea. A follow-up CT scan


ten days later indicated a resolving hemorrhagic lesion of the right parietal lobe with mass effect and

decreased attenuation extending anteriorly to the frontal lobe, consistent with edema. He exhibited

moderate hemineglect (41/100) on bedside testing two months later as part of this study. Although he had a

left temporal field cut initially, this had resolved by the time of testing and he was not hemiparetic.

Procedure for human subjects.

The target picture of an individual daisy, centered on a sheet of paper, and a blank sheet of paper,

were placed in front of the subject, with the latter in closer proximity to the subject. The center of the page

was initially aligned with the midline of the subject’s head, eyes, trunk and table although the midlines

likely shifted during the copying task as the subject moved his eyes, head or trunk. The subject was

instructed to copy the daisy using the dominant (right) hand, to take as long as necessary to do so and to

indicate when the task was complete. There were four targets, each containing a single daisy in a different

orientation (upright, 90 degree left rotation, inverted, 90 degree right rotation) and each picture was

presented twice, for a total of eight pictures per subject.

Procedure for computational implementation.

We instantiated the copying task in a computational simulation to explore the implications of a

spatial impairment in object- and viewer-centered reference frames. We adopted the hierarchical

representation depicted in Figure 3 and supported by the data from the normal subjects, and implemented it

as a conventional tree data structure, in which each node in the tree corresponded to a particular part of the

daisy. The node for a part contained information on its location in the object-centered frame defined by its

parent. Specifically, the object-centered frame for a part was oriented and centered on its parent, with a

scale defined by the horizontal extent of the parent (with x-coordinates ranging between +/-1). The viewer-

centered frame was always upright, centered on the page, and scaled by the horizontal extent of the daisy.

Thus, for instance, the rightmost petal in the upright daisy has a viewer-centered x-coordinate of about 0.5

(i.e., the horizontal position of its center is about half way between the midline of the daisy and the tip of the

right leaf) and an object-centered x-coordinate of about 2.0 (i.e., its horizontal distance from the center of its

parent, the circle, about twice the radius of the circle). For a misoriented daisy, the viewer-centered positions

of parts changed accordingly but their object-centered positions remained the same.


For a particular orientation of daisy, the probability that a part would be included and drawn in a

particular frame was assumed to be a monotonically-increasing function of its horizontal position in the

frame (Figure 4). The specific (exponential) form of this function is not critical as it influences only

quantitative aspects of the results; slightly different functions have similar consequences and the actual

function probably differs from patient to patient in any event (Mozer, 2001). Importantly, the assumption of

a left-right gradient is consistent with views of neglect in which there is a weak-to-strong representation

from left to right. This gradient not only fits with existing views of neglect (Kinsbourne, 1977; Kinsbourne,

1994) and the neural underpinnings but has also been successfully adopted in most computational models

of neglect (Monaghan & Shillcock, 1998; Mozer & Behrmann, 1990; Pouget & Driver, 2000). Notice that, with

the function we have adopted, the probability of drawing a part is near 1.0 on the right side of the frame,

about 0.9 at the midline, and drops off sharply towards the left of the frame. The overall likelihood that a

part is drawn was assumed to be a weighted average of its separate probabilities in the viewer-centered

frame and in the object-centered frame---the effects of different relative weightings are explored below. This

assumption emerges from the finding that neglect in different reference frames appears to be additive rather

than multiplicative (Behrmann & Tipper, 1999). All else being equal, the effect of neglect is generally

stronger in the object-centered frame than in the viewer-centered frame because the former is defined more

locally (i.e., parts typically fall outside the +/-1 frame defined by the horizontal extent of their parent).


A depth-first tree traversal algorithm was used to determine the neglect pattern. At every node, the

probability that the corresponding part is drawn is calculated based on its viewer-centered (assumed to

remain fixed) and object-centered (defined relative to its parent) coordinates. We assumed that if a part is

not drawn, then none of its subparts would be drawn. Thus, the probability of a part being drawn is the

product of the probability of its parent being drawn and its own local probability based on its relative

positions in the viewer- and object-centered frames. The order of traversal among children of the same

parent was irrelevant. The outcome of the tree traversal was that every part was assigned a probability of

being drawn based on the orientation of the daisy and the particular weightings of the viewer- and object-

centered frames. Once the probabilities are calculated, the program generates a coarse (piecewise linear)

graphical rendition of the daisy and superimposes the probabilities on it. We present these graphical


renditions. In addition, to evaluate the fit to the patient data, we establish a threshold such that those parts

whose probability falls below the threshold are omitted in the final rendition. We can then compare the

actual ‘drawings’ of the model with those produced by the patient.

Results and Discussion.

To understand the boundary conditions of the implementation, we first explored the individual

contribution of the viewer- and object-centered frame. To do so, we calculated the probability of each part

being drawn for daisies in all four orientations---up, left, down, and right, and initially the weighting of

either the viewer- or object-centered effect was set at 1 and the other effect was set at 0. Because the

misoriented, but not upright, daisy allows for the decoupling of the viewer- and object-centered effects,

Figure 5 illustrates the independent contribution of viewer-centered neglect and of object-centered neglect in

a left-facing daisy. The numbers superimposed on the daisy indicate the probability of each feature being

drawn, calculated according to the algorithm described above. It is important to recognize that the

probability of a part being drawn is contingent on the probability of its parent being drawn---if the parent or

containing objects is omitted, so is the child. The probabilities for the subparts such as the petals and leaves,

therefore, reflect the conditional probability of parent and child both being drawn and are subsequently

always lower than the probability of the parent alone.


As is evident from this figure, when the viewer-centered influence is 1.0 with no object-centered

influence (Figure 5a), information on the viewer-centered left has a fairly low probability of being drawn,

with the probability of the daisy head at 0.75 and the petal that occupies the leftmost position at 0.38. It is

interesting to note that while the daisy head has a .75 probability, the petals to the relative right of the daisy

head, defined by the viewer, have a lower probability (.62 and .63) because their probabilities are contingent

on the daisy head. Thus, even when the gradient is imposed purely egocentrically, there is still some

influence of the object structure on performance. The effect of inheritance is even more dramatically

observed in the right panel. When the viewer-centered effect is set to have no influence and neglect arises

solely within the object-centered frame (Figure 5b), information to the right of the canonical midline of the

daisy has a high probability of being drawn (approximately 0.94) whereas the petals and leaf on the left of


the intrinsic axis have a very low probability of being drawn (approximately 0.24). The leaf on the canonical

left stem has a probability of 0.06 both because it is conditional on its parent stem being drawn and because

it occupies the most extreme left position in the object-centered frame. Of note, then, is that the neglect is

more marked in the object-centered than the viewer-centered case. We now explore the implications of these

effects for the human performance and determine whether these reference frames and combinations thereof

can provide an account of the individual patient’s copying.

Both patients showed neglect in their copying of the upright daisy. Note that because the standard

copying task confounds the influences of reference frames centered on the viewer, the environment, and the

object, we cannot determine the individual contribution of these different reference frames to performance.

We turn, then, to the performance of the patients on the misoriented daisies and discuss JM’s data first

followed by those of GS.

Figure 6a presents examples of JM’s copy of one of each target daisy. In order to account for his

performance, we selected coefficients that we thought would best reproduce the findings; the relative

weightings of viewer- and object-centered neglect selected were 0.6 and 0.4, respectively. The resultant

numerical values are shown for each part in Figure 6b and, in Figure 6c, we display the output of the model

when a threshold of .57 is applied to the data to reflect which features would be neglected. Note that we

depict the targets with rounded leaves, as in Figures 2 and 3, and the output of the model with more

rectangular leaves in order to differentiate between the two.


As can be seen from Figure 6, JM’s data are reasonably well captured by this mixture of object- and

viewer-centered neglect. The upright daisy produced by the model is a close match to his copy with the

exception of the left stem/leaf. Of more interest are the misoriented daisies. The left-facing daisy reflects the

combination of the viewer- and object-based neglect as petals to the viewer left and object left are omitted.

Oddly, the daisy does not contain the upper-right petal (Figure 6a second from left). As it turns out, JM

initially drew this petal and then erased it, removing a small part of the circle along with it. The output of

the model is a reasonable match showing the omission of petals to the left in both frames although, again,

JM includes the leaf on the left but the model does not. The match between model and patient on the


inverted daisy is good, aside from the discrepant object-left leaf again, and reflects very little neglect. When

the left of the object appears on the right of the viewer, and vice versa, the decrement for the left of the object

is balanced by the strength of the right of the viewer and there is apparent compensation for the neglect.

This pattern arises again from a combination of object and viewer-centered and is similar to the finding that

when the left of the object appears on the right of the viewer, patients are better able to detect it than when

the left of the object is on the left of the viewer (Behrmann & Tipper, 1994).

Thus far, the output of the model does a fairly good job of accounting for JM’s performance with the

exception of the leaf on the left, an issue that we return to later. A discrepancy between the model and

patient, however, is observed on the right-facing daisy (Figure 6a extreme right). JM omits petals on the left

of the daisy head, defined by the viewer-frame, but the rest of the daisy is included. The model, on the other

hand, omits the left stem/leaf, as above, but retains all the petals. A possible explanation for this

discrepancy concerns the order of drawing. JM drew the daisy head first and because the daisy head, in

isolation, is completely symmetrical and has no intrinsic axis, the orientation of the daisy head alone is

ambiguous. Note that, under this condition, there is no other information on the page such as stem or pot to

constrain the reference frame. Given the absence of constraints, the petals on the left of the daisy head may

be defined as object-left and/or viewer-left initially and then neglected. Once the patient moves on to copy

the remaining features of the daisy, the orientation is anchored and the stem and pot can then define the

coordinates. Although this interpretation is speculative, at present, and we do not account for the temporal

order and ambiguity effects in our current implementation, we show below that this pattern is rather

commonly observed when patients draw the daisy head first. It is less common, however, when the daisy

head is not drawn first, lending support to this particular interpretation.

Having established a set of co-efficients that lead to a reasonably similar reproduction of JM’s

performance and identifying those aspects which the model fails to explain, we go on to examine whether a

similar approach can be applied to the second case, GS. Figure 7 below contains one of GS’s copies of each of

the daisies in the different orientations, along with the display depicting the probabilities associated with

individual parts and the rendition of the model using a threshold value. For GS, we use a .25 and .75

weighting of the viewer- and object-frame and the threshold for the final rendition is .55 (close to .57 for JM).



In his copy of the upright daisy, GS demonstrates marked neglect. Interestingly, he shows

contrapositioning of the left branch and leaf (Halligan, Marshall, & Wade, 1992a; Halligan et al., 1992a;

Halligan, Marshall, & Wade, 1992b; Vallar, Rusconi, & Bisiach, 1994), a pattern that is not uncommon in

neglect and is thought to reflect correct activation of object structure but with imprecise spatial positioning.

The model reproduces the figure quite well although it is not equipped to deal with this transposition.

Contrapositioning is also seen in the left-facing daisy; the object neglect is so strong here that even the

canonical left of the pot is excluded. The rendition of the model is reasonably good, again with the

exception of how it deals with the transposition of the leaf. Also, as mentioned previously, we have not

made allowance for fragmentation of the simple elements, such as the pot itself, into its components and,

hence, we cannot reproduced the neglect of the line on the left of the pot (although this limitation would be

straightforward to remedy by increasing the depth of the hierarchical tree to include line features).

In both the inverted and right-facing daisy, GS drew the daisy head first and, in both cases, petals

on the left of the daisy head are omitted. As discussed above, the absence of a constraining frame for the

symmetrical daisy head might be giving rise to the neglect of these petals. If this view is correct, when the

daisy head is not drawn first and there is a frame that constrains performance initially, the neglect for the

petals should not be as evident. Interestingly, on GS’s copy of the second right-facing daisy, he did not draw

the daisy head first (see Figure 8a) but drew the pot first followed by the stem. In direct comparison with the

same right-facing daisy in Figure 7, he now shows only mild, if any, neglect of petals from the left of the

daisy head, including 6 (rather than 4) petals here (with perhaps some contrapositioning or allowance for

positioning of the stem).

We also had the opportunity to obtain partial data from a third patient VD who was not well

enough to complete the entire experiment. On every trial, VD copied the daisy head first and omitted petals

of the daisy head, and we include her rightfacing daisy in Figure 8b. VD suffered a right middle cerebral

artery infarction at age 70 and scored 37/100 (mild to moderate neglect) on the bedside battery.

Interestingly, VD, has strong viewer-center neglect as manifest in her omission of the entire pot when it

occupies a left position in viewer-coordinates. Her pattern might be accounted for by a strong, perhaps even

sole, contribution of viewer-neglect (see Figure 5 for100% viewer-centered neglect) with the constraint of

temporal order of daisy head first. Unfortunately, we do not have the full complement of her data to

evaluate the exact fit of the model to her data.



As is evident from the above discussion, both patients JM and GS show the simultaneous effect of

view- and object-based neglect when copying upright and misoriented daisies and the implemented

algorithm, with weighting of these two frames, succeeds for the most part in accounting for their

performance. When the algorithm fails, it does so in similar ways for the two patients, and the failures are

instructive. For both patients, the model does not adequately cope with the left stem and/or leaf. The model

tends to omit the leaf whereas JM tends to preserve it as does GS, either by drawing it in on the appropriate

side or by contrapositioning it. This discrepancy between the model and patients suggests that there is

something unusual about the left stem/leaf. One possibility is that because of the relative length of the stem

and because of the symmetry of the two leaves, the stem/leaf becomes somewhat resistant to neglect. The

possible benefit afforded by perceptual organization is also relevant with regard to further fragmentation of

components of the hierarchy. We have not made allowance for neglect of strokes that comprise the pot or

that make up the petal or leaf. Omission of these strokes, however, is not very common in neglect. We take

up the issue of contrapositioning and perceptual organization further in the General Discussion.

The second discrepancy between model and patients is that of the omission of petals to the left of

the daisy head when the head is drawn first. It appears that, contrary to our assumption about absence of

ordering effects, the temporal order may be relevant, especially when the subpart to be drawn is ambiguous

in orientation and when left and right remain unconstrained. When the daisy head is drawn first, both JM,

GS and a third patient, VD, all omit the petals on the viewer-defined left. When other subparts are drawn

first, these same petals are not as strongly neglected. A clear prediction then is that, provided that the

subparts have a well-defined orientation or other subparts are drawn first, this pattern of neglect will not be

obtained. Aside from these limitations that show ways in which the patients and model diverge, the

algorithm and assumptions provides a reasonable account of the mixture of viewer- and object-centered

effects in the copying performance of two patients with hemispatial neglect and reflect the combined

influence of spatial position defined in an object- and viewer-centered reference frame.

Experiment 2: Neglect for hierarchically complex objects and computational implementation

The findings reported thus far, indicating combined effects of viewer- and object-based neglect,

were achieved through the patients’ copying of a single daisy that was misoriented to allow for the


disambiguation of the different reference frames. In this second experiment, we also demonstrate how the

combination of the different reference frames can determine the outcome of a figure-copying task. In this

case, however, we use a more complex object as the target in order to extend the account. The critical display

is a double, connected daisy that has more complicated hierarchical structure and, by virtue of this, allows

us to observe the relative contribution of the viewer- and object-effects even when the stimulus remains

upright. Figure 9b shows the double or connected daisy, made of two single unconnected daisies as shown in

Figure 9a. These displays are adapted from those used by Marshall and Halligan (1993) and their data and

findings are reported below. Ignoring the left daisy in both the connected and unconnected displays would

be indicative of pure viewer-based neglect. In contrast, omitting the left half of each daisy in the

unconnected case and the entire left daisy in the connected case (and possibly the petals on the left of the

right daisy, depending on the hierarchy) would be consistent with object-based neglect. Of course, various

mixtures of these different influences might also be observed and we explore these different patterns both

empirically and computationally.


As mentioned above, use of the more complex display allows us to examine the influence of object

representations with richer hierarchical structure on neglect. Indeed, in the first experiment, some evidence

for the importance of the object hierarchy was obtained, despite the simplicity of the single daisy. In that

case, both JM and GS omitted petals on the left of the daisy head (also VD in Figure 8) when the head was

drawn first. This suggests that the head itself, although a child in the tree structure, may be considered an

object or parent initially and its left (or the left of the head in viewer coordinates) neglected before other

subparts are drawn and can serve to anchor for a particular reference frame. In this experiment, then, we

explore the impact of object complexity on the patients’ and model performance. As before, we present the

methods for the patients first, followed by a description of the algorithm and its implementation. Following

this, we report the empirical and computational findings in an interleaved fashion.

Method.

Subject.


GS, who participated in the first experiment, also completed this study. JM was, unfortunately,

unavailable for testing in Experiment 2. We also present published data from two patients with neglect

described by Marshall and Halligan (1993).

Procedure for patients.

To produce a more complex object, we used the same daisy as in Experiment 1. In one condition, the

unconnected display, we included two of these daisies located adjacent to each other, centered on the same

page, with a 5cm space between them. Each of these is an object in itself and so we might think of this

display as reflecting two objects in a scene. Given the previous comment that we cannot reach definitive

conclusions from scene copying because of the sequential nature of the approach, we adopted the design of

Marshall and Halligan (1993) who connected the two daisies to form a single, hierarchically more complex

display. The daisy heads are of the same size in the two displays and the connected display is simply formed

via the connecting stem and pot, as shown in Figure 9b. GS completed two copies of each of these two

displays. Note that the single daisies do not have pots here.

Procedure for computational implementation

The method used here is identical to that in Experiment 1 except for the following. The object

hierarchy for the connected daisy is a simple combination of two single daisy hierarchies and there is, again,

no temporal order constraining which single daisy is drawn first. The algorithm is depth first so that a single

daisy must be completed in its entirety before the second daisy (or any other part) is begun. Using this

representation and the same horizontal gradient as we used previously, we attempted to simulate the

performance of GS on these displays. We also adopted the same mixture of weightings in the two reference

frames as we had converged on for him in Experiment 1 (.25 and .75 viewer and object weighting) and also

kept the threshold identical (at .55).

Results and Discussion.

To understand the boundary conditions of the implementation, as before, we first explored the

individual contribution of the viewer- and object-centered frame with these displays. To do so, we

calculated the probability of each part being drawn for the unconnected and the connected displays. Initially


the weighting of either the viewer- or object-centered effect was set at 1 and the other effect was set at 0.

Figure 10a shows the effect of the viewer reference frame without any influence of an object-centered frame

and Figure 10b shows the converse.


Let us consider the unconnected case first. An important difference between the two different

reference frames concerns the probabilities associated with the petals and stem/leaf complex on the left of

the right daisy. These petals and stem/leaf complex occupy a relative right position in viewer-centered

coordinates and thus have a high probability (petals: .93 - .94, stem: .94, leaf: .88) of being drawn when

performance is calculated with 100% viewer frame. In contrast, when the object-centered coordinates

determines performance, these same petals have a low probability of being drawn (.25-.36) and the leaf has

an even lower probability (.06), given that it is contingent on the stem (.25) being drawn. It is also worth

noting that in the 100% object-centered condition, the probability of the left petals and leaf being drawn is

equivalent for the daisy on the left and on the right, as performance is determined only with respect to the

daisy itself and does not take page/viewer position into account. In contrast, in the 100% viewer-centered

case, the contribution of spatial position to the probabilities associated for each part depend solely on the

left-right position with respect to the viewer. Thus, the further left a part is located, the more the probability

is lowered so that the petals on the left of the left daisy have only a .36-.42 probability of being drawn.

One further consideration in both the unconnected and connected displays is that, in the 100%

viewer-centered case, the probability of drawing the central circle of the daisy head (.97) is higher than the

probability of the petals to the right of it (.95). Indeed, it might appear counterintuitive for positions

appearing further rightward to receive lower probabilities than parts that appear to their left when

probability is purely determined by the viewer position. This effect results from the assumption that a child

(petal) will only be drawn if the parent (central circle) is drawn and this assumption, based on the

representation of the object and the hierarchy, applies independent of the reference frame. Thus, a petal will

always inherit the probability of its parent daisy-head and will have lower probability because of this

contingency. This apparent discrepancy between petal and daisy-head is remedied in the object-centered

case where petals that appear to the right of the daisy midline (in both displays and for both petals) have

higher probability than the corresponding daisy center by virtue of their rightward position in object-


centered space. This somewhat raised probability compensates for the lowered probability associated with

hierarchical inheritance.

The contrasts between solely viewer- and solely object effects become even more interesting when

we compare directly the output of the algorithm on the connected daisy to that of unconnected condition.

As is evident from the lower-left panel in Figure 10, in which the viewer-centered frame operates alone at

100%, the probabilities on the daisy head for the connected daisy are identical to those for the unconnected

daisies. This occurs because it is the absolute position of the parts, relative to the viewer, that determines the

probability while the position, relative to the object itself, has no effect. In the lower, right panel, we see the

effect of the 100% object-centered frame on the connected daisy and we consider each of the two component

daisies in turn. The probability of drawing the right daisy-head and its right petals are roughly equivalent to

the probability in the viewer-centered case. In contrast, the petals on the left of this right daisy have a low

probability of being drawn (.24-.35 versus .93-.94), compared with the viewer-centered condition, and are

closer to those in the 100% object-centered unconnected case (.25-.36).

An even more interesting contrast comes from examining the fate of the left daisy in the connected

100% object case. Here the right petals and stem/leaf have a lower probability (.68-.69) than the two single

daisy case (.99-1.00) as they occupy relative left positions in an object frame defined by the entire connected

daisy. They do, however, have a higher probability of being drawn than the corresponding petals and

stem/leaf in the 100% viewer connected daisy (.44-.55) as they are on the relative right of the frame defined

by the right daisy head and are immune to the fact that they are leftward in a viewer-defined frame.

Needless to say, the petals (.17-.25) and stem/leaf (.04/.17) on the left of the leftward daisy in the 100%

object-centered connected display have the lowest probabilities of all, falling to the left of the entire

connected display as well as to the left of the left daisy-head. These probabilities are even lower than those

in the unconnected case (upper right panel) as the petals and stem/leaf inherit their probability from their

parent, the left daisy head, which already has a leftward position in the object-centered frame, defined by

the entire connected daisy, and its own reduced probability of .69. These data show the connected daisy in

the 100% object-centered case reflects the position of the part in the object-centered frame and how the

hierarchical representation also affects the probabilities by virtue of inheritance. They contrast with the

simpler case of the viewer-centered effect where performance is more straightforwardly determined by left-

right position in viewer-centered coordinates and where only a small influence of the hierarchy is observed.


Having laid out the extreme conditions with the sole influence of one of the coordinate systems, we

can now evaluate whether the copying performance of patients can be accounted for within this framework.

Figure 11 shows the performance of patient GS on the two types of displays along with the numerical

probabilities of the parts being drawn by the model and the thresholded graphical versions, using the same

weightings (.75 viewer, .25 object) and threshold (.55) as in Experiment 1. If we consider the unconnected

condition first, the model does a reasonably good job of capturing his performance, showing neglect of the

left petals on both daisies. The variability associated the probabilities for the left stem/leaf, which gave rise

to one of the discrepancies between the model and the patient performance in Experiment 1, is also seen

here: the left stem/leaf is included on the left daisy but, suprisingly, it is omitted on the daisy to its right.

Performance on the connected daisy is also well accounted for by the model, with neglect of the left petals

on both daisies. As in the unconnected case, the left stem/leaf is variable in the patient’s performance in

that it is included on the left daisy and contrapositioned on the right. We revisit the issue of the left

stem/leaf in the final discussion.

Had we only had GS’s performance on the unconnected display, we would be unable to determine

whether the left neglect is defined by the object or the environmental position given that drawing proceeds

sequentially. Using the connected configuration, however, we can now verify that the probability of

including contralesional parts is determined not only by their viewer-centered position but that there is a

considerable contribution of the object-relative position. In fact, GS appears to show predominantly object-

centered effects, manifesting at multiple hierarchical levels. When a single daisy is the object, its left is

neglected and when a connected daisy is the object, the left at multiple hierarchical levels is affected with

even lowered probabilities found further down the hierarchy by virtue of inheriting the reduced

probabilities of the parents.


The computational results from Experiment 2 have dovetailed reasonably well overall with the

empirical findings. Based on this, we would suggest that one can discover the co-efficients that determine

the patient’s copying performance for both simpler and more complex objects as a function of the spatial

position of the parts of the display, defined in multiple reference frames. We would also suggest that the

approach we have adopted is general and can be extended to account for the performance of other patients

both on these kinds of tasks as well as on others. To explore the generalizability of the approach a little


further, we have also determined the co-efficients which replicate the performance of Marshall and

Halligan’s (1993) two patients on both the connected and disconnected displays and the graphical output of

the algorithm are shown in Figure 12 below.


Marshall and Halligan (1993) originally introduced the unconnected and connected daisy displays

as an elegant way of examining the presence of object-centered neglect and its co-existence with viewer-

centered neglect. Of relevance, they documented the performance of two different patients copying these

displays and the output of the two patients’ performance is shown in Figure 12 (above and below). Their

Patient 1 was considered to have 100% viewer-centered neglect, according to their analysis, as the entire

unconnected left daisy is ignored as is the entire left daisy of the connected display. If we adopt a threshold

of .56 (again, very close to that used thus far on our patients) on the output of the 100% viewer-centered

algorithm shown in Figure 10, we obtain a good fit to the data (see upper panel, Figure 12 ). Note that, here,

the patient omits the left stem/leaf in both displays whereas, with this threshold, the left stem/leaf survives

in the model. It is the case, however, that if we adopted a much more conservative threshold of .89, we

would eliminate the left stem/leaf from the model, mirroring the patient’s performance perfectly.

Figure 12 in the bottom panel shows the outcome of the algorithm for the second patient of Marshall

and Halligan (1993), who, on their analysis, showed a combined object/viewer neglect pattern. This

patient’s performance is best captured when the weightings used are 75% object-centered and 25% viewer-

centered, as was also the case for GS. A more conservative threshold of .75 than that used for GS, however,

yields a very good reproduction of the data. In the unconnected daisy case, the left of each single daisy is

neglected by the patient and the model. The patients includes the left stem/leaf of the right daisy but not of

the left daisy but the model neglects both. In the connected display, both the patient and model neglect the

left daisy entirely and, in addition, neglect the petals to the left of the right daisy. The patient’s

contrapositioning of the left stem/leaf on the right daisy is not reproduced by the model.

The findings from this experiment illustrate how the basic approach, in which empirical

performance is simulated in a simple computational simulation, outlined in Experiment 1, may be extended

when a more complicated display is used. The same threshold and weightings used for one patient in

Experiment 1 work well to reproduce his data in Experiment 2, testifying to the robustness of the results

from the first experiment. Additionally, the algorithm is able to account for the performance of two patients


reported by Marshall and Halligan (1993) in one of the paradigmatic examples of a figure-copying task and

the model produces a close fit to the patients’ data for both connected and unconnected displays.

General Discussion

The goal of this paper has been to explore how the figure-copying performance of patients with

hemispatial neglect might be accounted for by a simple algorithm in which the relative probability of

information being neglected or preserved is determined by spatial position. Spatial position was defined

with respect to two different reference frames, one viewer-centered and one object-centered, and we

examined how these different influences, operating alone or in combination, gives rise to patterns of

performance in a figure-copying task. In addition to investigating the effects of position in different

reference frames, we also manipulated the hierarchical complexity of the objects to be copied and explored

the impact of object complexity on neglect.

In the first experiment, we required two patients to copy a single daisy , which could appear in one

of four orientations. We had previously verified the hierarchical representation of this single daisy by

tracking the temporal order of the strokes used by normal subjects in producing such an object and showed

that the daisy consisted of three children, with each of those having children. We then explored whether a

computational algorithm which calculates the probability of a part being included in a drawing, based on

the spatial position of the part in the two reference frames (with the results combined additively) over this

hierarchical representation could reproduce the pattern of data. The match between the output of the

algorithm and the patient data was reasonably good overall and, by varying the weighting of the two

reference frames (and by applying a binary threshold), the model was able to produce very similar output to

that of the patients. In the one case, viewer- and object-centered weightings of .6 and .4, were successfully

used and, in the other, weightings of .25 and .75 were successful.

In the second experiment, we used more complex displays involving two unconnected daisies and a

single connected daisy, made by joining the two single daisies (Marshall & Halligan, 1993). By holding

constant the weightings of one of the patients from the first experiment, we were able to reproduce his

performance on these more complex displays. That we were able to show generalization of the weightings

established initially to a set of novel displays suggests that the general approach we adopted and the specific

weightings in his case are robust. Through the dynamic reassignment of elements to object or parts roles,


this same model can account for neglect of objects on the left of a multi-object scene, neglect on the left of a

single object, and neglect for features on the left of a part of a single object (for a similar view on within- and

between-object coding, see Humphreys & Riddoch, 1990; Humphreys & Riddoch, 1994b). We also showed

that we could produce a good rendition of the data from two patients copying analogous displays, reported

by Marshall and Halligan (1993).

Strengths and weaknesses of the account

Although the performance of the model was reasonably good overall, it failed consistently in some

regards and these instances are, in themselves, instructive. Perhaps the most noticeable failure concerns the

left stem/leaf. Note, however, that the inclusion or exclusion of these parts is inconsistent, even within a

single patient. In Experiment 1, GS placed both stem/leaves to the object right for the upright and left facing

daisy. In Experiment 2, he included the left stem/leaf on the left daisy in both the unconnected and

connected trials, but omitted it on the right daisy in the unconnected display and contrapositioned it in the

connected display. We also see some variability associated with this stem/leaf in patient 2 of Marshall and

Halligan (1993) in that he included the left stem/leaf on the right, but not left, daisy in the unconnected

display and contrapositioned it on the right daisy in the connected display. Under these conditions of

variability, it might be unreasonable to expect the model to reproduce the variability but the issue of

contrapositioning is an important one. This pattern, in which stimuli delivered to the contralesional side are

referred to the symmetrical location on the ipsilesional side, also termed ‘allochiria’, was recognized over a

century ago (Obersteiner, 1882), and may be observed across multiple sensory modalities (Bisiach &

Geminiani, 1991). Clearly not all patients exhibit this phenomenon, as evident in our data, and as confirmed

by Kawamura, Kirayama, Shinohara, Watanabe, & Sugishita (1987) who documented this pattern in 20 out

of 123 patients who had sustained a cerebral haemorrhage. Although it has been suggested that there is

correct activation of the contralesional information with imprecise localization, the mechanisms underlying

contrapositioning are not well understood, nor is the variability from patient to patient (Bisiach & Vallar,

2000). The failure of the model to reveal this pattern is perhaps not surprising then and this issue awaits

further clarification.

A second noticeable failure of the model is in accounting for the occasional fragmentation of

component parts (as specified in the object hierarchy). For example, in Experiment 2, on the left facing daisy,


GS omitted the left stroke of the pot, defined in object-centered coordinates. This fragmentation of

components into strokes is not very common and there are no other clear examples in the patient data

reported here. Note that patients almost never draw only the right half of the circle for the head of the

flower, nor the right part of a petal (for example, when the petal is vertical), nor do they omit the lip of the

pot (if the base is drawn), even if it occupies a position on the left of the spatial reference frame. Similarly, in

clock drawing or copying, even if patients neglect to fill in the numbers on the left of the clock, they

invariably draw the entire perimeter of the clock (see Figure 1). A possible explanation for the rarity of this

fragmentation, however, may be attributable to the apparent preservation of grouping mechanisms in these

patients. For example, Vuilleumier et al. (2001) reported that some patients are able to judge the midpoint of

illusory Kanisza stimuli despite their failure to detect the left-sided inducers in explicit matching

judgements. Several recent studies have also shown that patients with neglect remain sensitive to other

Gestalt properties of the stimulus. Thus, if a feature on the left of the object's midline can be grouped

together with a feature on the right to form a ‘good’ figure, based on principles such as good continuation,

symmetry or closure, the left-sided feature is less likely to be neglected (Ward, Goodrich, & Driver, 1994).

Similar effects are obtained when the left item can be grouped with the items on the right by color,

brightness, proximity or collinearity, for example (Gilchrist, Humphreys, & Riddoch, 1996; Mattingley,

David, & Driver, 1997). The strength of grouping, according to Gestalt heuristics, could potentially be

incorporated into the hierarchical representation adopted here. Under conditions of very severe neglect or

when the elements do not strongly comprise a more global configuration, fragmentation into lower level

strokes (and neglect thereof) would still be observed but when the neglect is less severe or when the

grouping is strong, fragmentation would be resisted.

The final difficulty encountered by the model concerns the petals on the daisy head. In Experiment

1, when the misoriented daisies were copied and the daisy head was drawn first, petals to the left of the

head were neglected. This sometimes gave rise to unusual patterns as, when the entire daisy was complete,

the omitted petals occupied a position on the right, defined within a reference frame defined by the viewer

or by the entire daisy. This pattern was evident in GS’s copy of the right facing and inverted daisy, in JM’s

right facing daisy and in the performance of a third patient, VD for whom we had only limited data. We

suggested that this pattern emerged because, when the symmetrical daisy ahead alone represents the

display, the exact reference frame is ambiguous and petals to the left are deleted. As we have suggested


previously, one possible solution to this would be to impose temporal order on the model, as in these cases,

the patients are following a ‘daisy head first’ strategy. In this case, the reference frame would be ambiguous

and the petals on the left would be associated with low probability of inclusion. Once other subparts are

included, their constrained reference frames would then have an impact in subsequently determining what

is neglected and what is preserved.

Object-based neglect revisited

One of the critical issues dealt with in this paper is the existence of a frame of reference that is

aligned with the midline of an individual object. Such a reference frame, in which spatial position of object

parts depends on the extent or shape of the object and is independent of the viewer, is particularly useful for

object recognition and would serve an important role in viewpoint independence. In some of his seminal

work on structural-description theory of object recognition, Marr (1982; Marr & Nishihara, 1978) postulated

the presence of a representation in which object parts are related directly to each other. At the outset, we

provided numerous examples from empirical studies, from both human and nonhuman primates, which

appear to support such a representation.

The existence of an object-centered representation has not, however, gone without challenge. Driver

and colleagues (Driver, 1999; Driver & Pouget, 2000), for example, have suggested that there is no need to

invoke a reference frame that is tied to an individual object. Rather, they have argued that the left and right

of an object may be coded solely from one’s initial egocentric (and viewpoint dependent) encounter with the

object. The claim is that when an object is viewed, left and right are assigned in a purely egocentric manner

in accordance with the strength of an underlying attentional gradient, akin to the one we use here but

defined with respect to the retina [(Driver, 1999); for additional evidence of an attentional gradient, see

(Kinsbourne, 1993)]. A similar claim is made by Pouget and Sejnowski in their modeling work (Pouget,

Deneve, & Sejnowski, 1999; Pouget & Sejnowski, 1997); because the left of the object always appears at the

poorer end of the gradient relative to the right of the object, both in absolute and relative egocentric space,

the ipsilesional information will always dominate over the contralesional information, which will then be

neglected.

This view suggests that object-centered coding is not necessary and that the same pattern of data

may be obtained from simply assuming an egocentric gradient. Indeed, Mozer (2001) has conducted


simulations of so-called object-centered neglect in the context of a computational model, MORSEL, which

assigns spatial position purely egocentrically (by virtue of a retinotopic attentional gradient) and does not

have any object-centered representation. He shows that this implementation can account for a host of object-

centered neglect effects (e.g., Arguin & Bub, 1993a; Driver et al., 1994; Driver & Halligan, 1991; Pavlovskaya

et al., 1997). In all of these cases, the left of the object always appears further left than the object right, both

absolutely and relatively, and so is less activated.

Perhaps a more challenging situation is that of the barbell data from Behrmann and Tipper

(Behrmann & Tipper, 1994; Behrmann & Tipper, 1999; Tipper & Behrmann, 1996) in which the left of the

object does not always appear further left than the right of the object. In this paradigm, a barbell appears on

a screen, with the left and right circles colored in blue or red (and the color remains constant for a single

subject but is counterbalanced across subjects). In the first, static condition, a position on the right or left is

probed and this position is both right and left in both viewer- and object-coordinates and serves as a

baseline against which to compare performance in the second condition. In the critical, rotating condition,

the barbell is previewed and then undergoes a rotation of 180 degrees so that the left, defined by the barbell,

appears on the right of the viewer, and the right of the barbell appears on the left of the viewer. When a

spatial position on the viewer-defined right and left is probed, both accuracy and speed of detection is

influenced by whether this position occupies a right or left position, defined by the object. Thus, when the

probe appears on the viewer’s right but is on the left of the barbell (which rotated into that side), detection is

poorer than when the position is both viewer- and object-right. Similarly, when the probe appears on the

viewer’s left, detection is better when the position occupies the right of the barbell (which rotated in)

compared to when it is both viewer- and object-left. In this barbell experiment, because the left of the barbell

does not fall further left than the right, a simple egocentric gradient cannot obviously account for the data.

Instead, Mozer (2001) simulated the findings in the following way; when the barbell appears initially, the

activation of the left and right is set by the strength of the egocentric gradient. As the barbell turns, because

of hysteresis of the system, the initial activation is pulled along with it and through covert attention, is

carried to the new location. Probing the new location (‘end state’) then yields poor performance even when

the probe appears on the right as the activation associated with that location has been carried there by the

covert tracking of the moving barbell. According to Mozer, then, these simulations demonstrate that the

results of the barbell studies do not necessarily implicate object-based representations.


An outstanding question, however, is what mechanism allows for the representation of the object

and its parts under conditions of misorientation. When objects are translated in the picture plane, the left of

the object always remains to the relative left of the right of the object but this is not true when objects are

rotated. Two potential processes have been suggested to compensate for this: Mozer (2001) suggests that

covert attentional tracking represents the left and right, defined initially egocentrically, as the objects rotate.

The second suggested process is mental rotation. For example, Buxbaum, Coslett, Montgomery, & Farah

(1996) have suggested that in the case of misoriented stimuli, the stimulus is first normalized to its upright

orientation through mental rotation, and then the relative left is neglected. According to their view, then, an

egocentric gradient can still explain the empirical results; in the case of the barbell, the patients transform the

rotated barbell to its canonical upright position and then neglect the left of the ‘upright’ barbell (i.e. defined

gravitationally or egocentrically now). They base their claim on the fact that, only when they specifically

instructed a neglect patient to do the mental transformation on the barbell paradigm, did they obtain the

object-centered results.

Both mechanisms appear to encounter problems, however. With regard to covert tracking

explanations, it is now well-established that these patients have problems directing covert (and overt)

attention contralesionally (Arguin & Bub, 1993b; Posner, Walker, Friedrich, & Rafal, 1984). Functional

imaging studies have also shown that the right parietal region plays a critical role in directing attention to

the left (Corbetta, Miezin, Shulman, & Petersen, 1993; Nobre et al., 1997) and, hence, after damage to this

region, as in the case of neglect, attentional monitoring, either covert or overt, would be compromised.

There is also the problem of how such a tracking system might operate when stimuli are static and do not

need to be tracked; for example, when a stimulus is displayed inverted, as in the daisies here, the faces in the

study by Young, Newcombe, Haan, and Ellis (1990) or the words in the study by Caramazza and Hillis

(1990a). In these cases, there is no opportunity for covert attention to carry the activation of the egocentric

gradient along with it. It is precisely under such conditions that one might then invoke a process of

normalization via mental rotation. But, the involvement of mental rotation to account for the results is in

itself problematic: unlike Buxbaum et al. (1996), Behrmann and Tipper did not explicitly instruct the patients

to perform mental rotation and yet they still obtained the critical pattern of results. Moreover, nothing in the

demands of the task (simple light detection) would have prompted patients to engage in what is generally

considered to be an effortful, time-consuming process. Furthermore, it has been repeatedly demonstrated


that the right parietal lobe plays a critical role in mental rotation (Alivasatos & Petrides, 1997; Tagaris et al.,

1997) and that, when damaged, mental rotation is significantly impaired (Farah & Hammond, 1988).

Because the neglect patients typically have extensive damage to parietal cortex, it is unlikely that they are

capable of exploiting mental rotation processes. As such, it is unlikely that object-centered effects emerge

from covert attentional tracking or from normalizing via mental rotation.

We have suggested that the results emerge from the fact that subjects represent the structure of

viewed objects in terms of a spatial coordinate system that has a midline defined by the object itself.

Following brain damage to regions that represent spatial information, the contralateral side of such a

representation is then adversely affected. We have also suggested that the use and salience of such a

representation depends importantly on the nature of the task. The notion that the frame of reference used

depends on the goals of the user or the effector required by the task is not novel and is applied in the case of

other reference frames as well (for example, see Vecera and Farah, 1994, with normal subjects). For example,

it has been suggested that the ability to attend to various locations in space depends on brain areas that are

involved in organizing goal-directed actions to them (Colby, 1998; Rizzolatti & Camarda, 1987; Snyder,

Batista, & Andersen, 1997). Thus, when a task requires eye movements, we expect to observe a robust

influence of a frame of reference that is retinocentric and gaze-dependent. Likewise, when the task requires

a reach, limb-based coordinates are invoked and are relevant. We would suggest that copying an object,

especially one that is hierarchically complex, is a particularly good example of a situation in which object-

centered representations might need to be invoked. One might also imagine that object recognition itself

requires such a representation. Indeed, as recognized by Mozer (2001), “surely, if demanded by the task,

people can mentally construct visual object-based representations” (p79). We would suggest that many

tasks employ this form of spatial representation and it is on these tasks that object-centered neglect would

be obtained.

Individual Variability

A final issue to be addressed concerns the individual variability across and within patients.

Specifically, we have obtained different co-efficients and different weightings of the two reference frames

for each subject, and the question is what determines these weightings across the different patients. There

are a number of possibilities. For example, the site of the lesion may be an important determinant of the


extent to which different reference frames are affected given that neuronal populations in different regions

of parietal cortex are responsible for coding spatial position in different coordinate frames. For example,

neurons in the lateral intraparietal sulcus in monkeys have receptive fields at locations defined relative to

the retina (and modulated by orbital position) whereas neurons in the ventral intraparietal sulcus represent

locations in a head-centered frame (Colby, 1998). Additionally, in LIP, a small proportion of neurons are

sensitive to locations defined in a reference frame tied to an object (Sabes et al., 2001). Thus, depending on

the site of the lesion, various forms of spatial coding may be disrupted. We have no clear way of verifying

the correlation between lesion site and pattern of reference-frame coding in different individuals given the

lack of spatial resolution on neuroimaging that is available for humans and so this remains speculative. It is

interesting to note, however, that there is an asymmetry between the forms of coding suggested by the

neurophysiology data, in that object-fixed effects represent a small proportion of the spatial code in LIP

whereas the stronger effects are viewer-centered. We should note that, consistent with this asymmetry, none

of the patients reported in this paper have pure object-centered neglected; the neglect is either viewer-

centered or is a mixture of viewer- and object-centered neglect (this same asymmetry was evident in

Behrmann and Tipper, 1999). A prediction from the neurophysiology data is that it would be rare, perhaps

impossible, to find a patient whose deficit reflected only object-centered neglect without any viewer-

centered neglect. The converse, however, might not be uncommon and patient VD whose data we include in

Experiment 1 and patient 1 of Marshall and Halligan (1993) from Experiment 2 appear to fit this pattern.

The final issue concerns the variability within an individual subject. We obtained two copies of each

figure from each patient whose data are reported in full in this paper. Most of our analysis, however, was

based on a single copy of each. In most cases, performance was not substantially different between the

various versions of the figures although there were occasions where neglect was somewhat milder or more

severe (for example, we show in Experiment 1 that GS showed differences in the extent of neglect in his two

renditions of the right facing daisy). We should note that there does not seem to be a reversal of the pattern

of data in any individual patient and the differences appear to be quantitative rather than qualitative.

Conclusions

This paper presents an approach to characterize systematically the behavior of a mechanism in

which hierarchical object representations and multiple reference frames interact to co-determine


performance of a system and its output under damage. The simulations are not intended to be an explicit

instantiation of the neural mechanism underlying neglect nor to parallel directly the function of parietal

lobe. The principles embodied in this work, however, are consistent with many views that argue that the

parietal lobe integrates and transforms data from one set of coordinates to another (Colby, 1991; Karnath,

1994; Stein, 1992). How the brain might actually implement a hierarchical representation and how it might

achieve the dynamic reassignment of the components to parts and wholes are difficult research issues

(although see Hinton, 1990, for a connectionist approach to these problems) and we have attempted to

address these in the context of hemispatial neglect.

The task of copying the figure of a daisy was used in this research because it is standardly used in

the clinical assessment of neglect and because much is known about the performance of neglect patients on

this task. The principles governing the joint effects of neglect in more than one reference frame, as proposed

here, however, are believed to apply more generally and this approach leads to a number of predictions and

potential constraints on a system that is thought to underlie spatial representations and its relationship to

object recognition.


References

Alivasatos, B., & Petrides, M. (1997). Functional activation of the human brain during mental rotation.Neuropsychologia, 35(2), 111-118.

Andersen, R. A., Essick, G. K., & Siegel, R. M. (1985). Encoding of spatial location by posterior parietalneurons. Science, 230, 456-458.

Arguin, M., & Bub, D. (1993a). Evidence for an independent stimulus-centered spatial reference frame froma case of visual hemineglect. Cortex, 29, 349-357 .

Arguin, M., & Bub, D. N. (1993b). Modulation of the directional attention deficit in visual neglect byhemispatial factors. Brain and Cognition, 22, 148-160.

Attneave, F. (1971). Multistability in perception. Scientific American, 225, 63-71.

Bartolomeo, P., & Chokron, S. (1999). Egocentric frame of reference: its role in spatial bias after righthemisphere lesions. Neuropsychologia, 37, 881-894.

Behrmann, M. (2000). Spatial reference frames and hemispatial neglect. In M. Gazzaniga (Ed.), The CognitiveNeurosciences (Second ed., pp. 651-666). Cambridge, MA: MIT Press.

Behrmann, M., Ghiselli-Crippa, T., & Di Matteo, I. (2001). Impaired initiation but not execution of eyemovements in patients with hemispatial neglect. Behavioral Neurology, in press.

Behrmann, M., & Moscovitch, M. (1994). Object-centered neglect in patients with unilateral neglect: Effectsof left-right coordinates of objects. Journal of Cognitive Neuroscience, 6(1), 1-16.

Behrmann, M., & Tipper, S. P. (1994). Object-based attentional mechanisms: Evidence from patients withunilateral neglect. In C. Umilta & M. Moscovitch (Eds.), Attention and Performance XV: Conscious andnonconscious information processing (pp. 351-375). Cambridge, MA: MIT Press.

Behrmann, M., & Tipper, S. P. (1999). Attention accesses multiple reference frames: Evidence from neglect.Journal of Experimental Psychology: Human Perception and Performance, 25(1), 83-101.

Bisiach, E., & Geminiani, G. (1991). Anosagnosia related to hemiplegia and hemianopia. In G. P. Prigatano &D. L. Schacter (Eds.), Awareness of deficit after brain injury (pp. 17-39). New York: Oxford University Press.

Bisiach, E., & Vallar, G. (2000). Unilateral neglect in humans. In F. Boller & J. Grafman (Eds.), Handbook ofNeuropsychology (2 ed., Vol. 1, ). North-Holland, Amsterdam: Elsevier Science.

Black, S. E., Ebert, P., Leibovitch, F., Szalai, J., Blair, N., & Bondar, J. (1994). Recovery in hemispatial neglect.Neurology, 45, A178.

Buxbaum, L. J., Coslett, H. B., Montgomery, M. W., & Farah, M. J. (1996). Mental rotation may underlieapparent object-based neglect. Neuropsychologia, 34(2), 113-126.

Caramazza, A., & Hillis, A. E. (1990a). Levels of representation, co-ordinate frames and unilateral neglect.Cognitive Neuropsychology, 13(5/6), 391-446.

Caramazza, A., & Hillis, A. E. (1990b). Spatial representation of words in the brain implied by studies of aunilateral neglect patient. Nature, 346, 267-269.

Cate, A., & Behrmann, M. (2001). Hemispatial neglect: Spatial and temporal influences. submitted manuscript.

Chokron, S., & Imbert, M. (1995). Variations of the egocentric reference among normal subjects and apatients with unilateral neglect. Neuropsychologia, 33, 703-711.


Colby, C. (1998). Action-oriented spatial reference frames in cortex. Neuron, 20, 15-24.

Colby, C. L., & Goldberg, M. E. (1999). Space and attention in parietal cortex. Annual Review of Neuroscience,22, 319-349.

Corbetta, M., Miezin, F. M., Shulman, G. L., & Petersen, S. E. (1993). A PET study of visusospatial attention.Journal of Neuroscience, 13(3), 1202-1226.

Deneve, S., & Pouget, A. (1997, ). Neural basis of object-centered representations. Paper presented at the NIPS*97,Boulder.

Driver, J. (1999). Egocentric and object-based visual neglect. In N. Burgess, K. J. Jeffery, & J. O'Keefe (Eds.),The hippocampal and parietal foundations of spatial behavior (pp. 67-89). Oxford: Oxford University Press.

Driver, J., Baylis, G. C., Goodrich, S., & Rafal, R. D. (1994). Axis-based neglect of visual shape.Neuropsychologia, 32(11), 1353-1365.

Driver, J., & Halligan, P. W. (1991). Can visual neglect operate in object-centered coordinates: An affirmativestudy. Cognitive Neuropsychology, 8, 475-496.

Driver, J., & Pouget, A. (2000). Object-centered visual neglect, or relative egocentric neglect. Journal ofCognitive Neuroscience, 12(3), 542-545.

Farah, M. J., Brunn, J. L., Wong, A. B., Wallace, M., & Carpenter, P. (1990). Frames of reference for theallocation of spatial attention: Evidence from the neglect syndrome. Neuropsychologia, 28, 335-347.

Farah, M. J., & Hammond, K. M. (1988). Mental rotation and orientation-invariant object recognition:Dissociable processes. Cognition, 29, 29-46.

Gainotti, G., Messerli, P., & Tissot, R. (1972). Qualitative analysis of unilateral spatial neglect in relation tolaterality of cerebral lesions. Journal of Neurology, Neurosurgery and Psychiatry, 35, 545-550.

Gilchrist, I. D., Humphreys, G. W., & Riddoch, M. J. (1996). Grouping and extinction: Evidence for low-levelmodulation of visual selection. Cognitive Neuropsychology, 13(8), 1223-1249.

Grabowecky, M., Robertson, L. C., & Treisman, A. (1993). Preattentive processes guide visual search :Evidence from patients with unilateral visual neglect. Journal of Cognitive Neuroscience, 5(3), 288-302.

Halligan, P., Marshall, J. C., & Wade, D. T. (1992a). Contrapositioning in a case of visual neglect.Neuropsychological Rehabilitation, 2, 125-135.

Halligan, P. W., Marshall, J. C., & Wade, D. T. (1992b). Left on the right: Allochiria in a case of left visuo-spatial neglect. Journal of Neurology, Neurosurgery and Psychiatry, 55, 717-719.

Haywood, M., & Coltheart, M. (2000). Neglect dyslexia and the early stages of visual word recognition.Neurocase, 6(1), 33-43.

Hillis, A. E., Rapp, B., Benzing, L., & Caramazza, A. (1998). Dissociable coordinate frames of unilateralneglect: "Viewer-centered" neglect. Brain and Cognition, 37, 491-526.

Hinton, G. E. (1990). Mapping part-whole hierarchies in connectionist networks. Artificial Intelligence, 46, 47-75.

Humphreys, G. W., Cinel, C., Wolfe, J., Olson, A., & Klempen, A. (2000). Fractionating the binding process:Neuropsychological evidence distinguishing binding of form from binding of surface features. VisionResearch, 40, 1569-1596.


Humphreys, G. W., & Riddoch, M. J. (1990). Interactions between object and space systems revealed throughneuropsychology. In D. E. M. a. S. Kornblum (Ed.), Attention and Performance XIV . Hillsdale, NJ: LawrenceErlbaum Associates.

Humphreys, G. W., & Riddoch, M. J. (1994a). Attention to within-object and between-object spatialrepresentations: Multiple sites for visual selection. Cognitive Neuropsychology, 11(2), 207-241.

Humphreys, G. W., & Riddoch, M. J. (1994b). Separate coding of space within and between perceptualobjects: Evidence from unilateral visual neglect. Cognitive Neuropsychology, in press.

Karnath, H. O., & Niemeier, M. (2001). The exploration of space and objects in space by neglect patients.submitted manuscript.

Kawamura, M., Kirayama, K., Shinohara, Y., Watanabe, Y., & Sugishita, M. (1987). Alloaesthesia. Brain, 110,225-236.

Kinsbourne, M. (1977). Hemi-neglect and hemisphere rivalry. In E. Weinstein & R. Friedland (Eds.), Hemi-inattention and Hemispheric Specialization: Advances in Neurology 18 (pp. 41-49). New York: Raven Press.

Kinsbourne, M. (1987). Mechanisms of unilateral neglect. In M. Jeannerod (Ed.), Neurophysiological andNeuropsychological Aspects of Spatial Neglect (pp. 69-86). Amsterdam: North Holland.

Kinsbourne, M. (1993). Orientational bias model of unilateral neglect: Evidence from attentional gradientswithin hemispace. In I. H. Robertson & J. C. Marshall (Eds.), Unilateral neglect: Clinical and ExperimentalStudies (pp. 63-86). Hove, UK: Lawrence Erlbaum.

Kinsbourne, M. (1994). Mechanisms of neglect: Implications for rehabilitation. NeuropsychologicalRehabilitation, 4, 151-153.

Kosslyn, S. M. (1987). Seeing and imagining in the cerebral hemispheres: A computational approach.Psychological Review, 94(2), 148-175.

Marr, D. (1982). Vision. San Francisco, CA: W. H. Freeman.

Marr, D., & Nishihara, H. K. (1978). Representation and recognition of the spatial organization of three-dimensional shapes. Proceedings of the Royal Society of London, Series B, 200, 269-294.

Marshall, J. C., & Halligan, P. W. (1993). Visuo-spatial neglect: A new copying test to assess perceptualparsing. Journal of Neurology, 240, 37-40.

Mattingley, J. B., David, G., & Driver, J. (1997). Pre-attentive filling in of visual surfaces in parietal extinction.Science, 275, 671-674.

McGlinchey-Berroth, R. (1997). Visual information processing in hemispatial neglect. Trends in CognitiveSciences, 1(3), 91-97.

Monaghan, P., & Shillcock, R. (1998). The cross-over effect in unilateral neglect. Brain, 121, 907-921.

Mozer, M. C. (2001). Frames of reference in unilateral neglect and visual perception: A computationalperspective. submitted manuscript.

Mozer, M. C., & Behrmann, M. (1990). On the interaction of selective attention and lexical knowledge: Aconnectionist account of neglect dyslexia. Journal of Cognitive Neuroscience, 2(2), 96-123.

Nobre, A. C., Sebestyen, G. N., Gittleman, D. R., Mesulam, M. M., Frackowiak, R. S. J., & Frith, C. D. (1997).Functional localization of the system for the visuospatial attention using positron emission tomography.Brain, 120, 515-533.


Obersteiner, H. (1882). On allochiria: A peculiar sensory disorder. Brain, 4, 153-163.

Palmer, S. E. (1977). Hierarchical structure in perceptual representation. Cognitive Psychology, 9, 441-474.

Pavlovskaya, M., Glass, I., Soroker, N., Blum, B., & Groswasser, Z. (1997). Coordinate frame for patternrecognition in unilateral spatial neglect. Journal of Cognitive Neuroscience, 9(6), 824-834.

Posner, M. I., Walker, J. A., Friedrich, F. J., & Rafal, R. D. (1984). Effects of parietal injury on covert orientingof visual attention. Journal of Neuroscience, 4, 1863-1874.

Pouget, A., Deneve, S., & Sejnowski, T. J. (1999). Frames of reference in hemineglect: A computationalperspective .

Pouget, A., & Driver, J. (2000). Relating unilateral neglect to the neural coding of space. Current Opinion inNeurobiology, 10, 242-249.

Pouget, A., & Sejnowski, T. J. (1997). A new view of hemineglect based on the response properties of parietalneurones. Philosophical Transaction of the Royal Society, 352, 1449-1459.

Reuter-Lorenz, P., Drain, M. and Hardy-Morais, C. (1996). Object-centered attentional biases in the normalbrain. Journal of Cognitive Neuroscience, 8, 6, 540-550.

Riddoch, M. J., Humphreys, G. W., Luckhurst, L., Burroughs, E., & Bateman, A. (1995). "Paradoxicalneglect": Spatial representations, hemisphere-specific activation and spatial cueing. CognitiveNeuropsychology, 12(6), 569-604.

Rizzolatti, G., Berti, A., & Gallese, V. (2000). Spatial neglect: Neurophysiological bases, cortical circuits andtheories. In F. Boller & J. Grafman (Eds.), Handbook of Neuropsychology . North-Holland, Amsterdam: ElsevierScience.

Rizzolatti, G., & Camarda, R. (1987). Neural circuits for spatial attention and unilateral neglect. In M.Jeannerod (Ed.), Neurophysiological and neuropsychological aspects of spatial neglect (pp. 289-313). NorthHolland: Elsevier Science Publishers.

Sabes, P. N., Breznen, B., & Andersen, R. A. (2001). The parietal representation of object-based saccades.submitted manuscript.

Smania, N., Martini, M., Gambina, G., Tomelleri, G., Palmara, A., Natale, E., & Marzi, C. (1998). The spatialdistribution of visual attention in hemineglect and extinction patients. Brain, 121(9), 1759-1770.

Snyder, L. H., Batista, A. P., & Andersen, R. A. (1997). Coding of intention in the posterior parietal cortex.Nature, 386, 167-170.

Tagaris, G. A., Kim, S. G., Strupp, J. P., Andersen, P., Ugurbil, K., & Georgopolous, A. P. (1997). Mentalrotation studied by functional magnetic resonance imaging at high field (4 Tesla): Performance and corticalactivation. Journal of Cognitive Neuroscience, 9(4), 419-432.

Taylor, H. A., & Tversky, B. (1992). Descriptions and depictions of environments. Memory and Cognition, 20,483-496.

Tipper, S. P., & Behrmann, M. (1996). Object-centred not scene-based visual neglect. Journal of ExperimentalPsychology: Human Perception and Performance, 22(5), 1261-1278.

Vallar, G. (1998). Spatial hemineglect in humans. Trends in Cognitive Sciences, 2(3), 87-96.

Vallar, G., Rusconi, M. L., & Bisiach, E. (1994). Awareness of contralesional information in unilateral neglect:effects of verbal cueing, tracing and vestibular information. In C. Umilta & M. Moscovitch (Eds.), Consciousand nonconscious information processing (pp. 377-391). Cambridge, MA: MIT press.


Van Sommers, P. (1989). A system for drawing and drawing-related neuropsychology. 1989, 6(2), 117-164.

Vecera, S. P. and Farah, M. J. (1994). Does visual attention select objects or locations? Journal of ExperimentalPsychology: Human Perception and Performance, 123, 2, 1-14.

Vuilleumier, P., Valenza, N., & Landis, T. (2001). Explicit and implicit perception of illusory contours inunilateral spatial neglect: behavioral and anatomical correlates of preattentive grouping mechanisms.Neuropsychologia, 39, 597-610.

Ward, R., Goodrich, S., & Driver, J. (1994). Grouping reduces visual extinction: Neuropsychological evidencefor weight-linkage in visual selection. Visual Cognition, 1(1), 101-129.

Young, A. W., Hellawell, D. J., & Welch, J. (1992). Neglect and visual recognition. Brain, 115, 51-71.

Young, A. W., Newcombe, F., Haan, E. D., & Ellis, A. (1990). Facial neglect. Neuropsychologia.


Figure legends:

Figure 1: Representative examples of left-sided neglect during figure copying task by a neglect patient.

Figure 2: Targets of copying: single daisies at four different orientations.

Figure 3. A daisy and its hierarchical representation, so that each part (“child”) of an object (“parent”) canbe considered an object in its own right.

Figure 4. Function depicting the probability of drawing a part as a function of its horizontal position. Thefunction is applied to both the viewer- and object-based reference frames.

Figure 5. The probabilities that the parts of a left-facing daisy are drawn when neglect operates (a) solely inthe viewer-centered frame, and (b) solely in the object-centered frame(s).

Figure 6. (a) Copy of daisies by JM. (b) Probability of drawing each part as function of algorithm, producedby a mixture of 0.6 viewer-centered neglect and 0.4 object-centered neglect (c) Output of model,assuming a threshold probability of drawing a part of 0.57.

Figure 7. (a) Copy of daisies by GS. (b) Probability of drawing each part as function of algorithm, producedby a mixture of 0.75 viewer-centered neglect and 0.25 object-centered neglect (c) Output of model,assuming a threshold probability of drawing a part of 0.55.

Figure 8. Copy of right-facing daisy by patients (i) GS and (ii) VD in which combined viewer and objectneglect appear.

Figure 9. Targets of unconnected and connected daisy displays.

Figure 10. The probabilities that the parts of the single unconnected daisies and connected daisy are drawnwhen neglect operates (a) solely in the viewer-centered frame (100%), and (b) solely in the object-centeredframe (100%).

Figure 11. (a) GS’s copy of unconnected and connected daisy display. (b) Probability of drawing each part asfunction of algorithm, produced by a mixture of 0.75 viewer-centered neglect and 0.25 object-centeredneglect. neglect (c) Output of model, assuming a threshold probability of drawing a part of 0.55.

Figure 12. Copy of (a) unconnected and (b) connected daisy, with output of algorithm for two patientsreported by Marshall and Halligan (1993).


Figure 1


Figure 2


Figure 3


Figure 4


Figure 5

(a) 100% viewer-centered neglect (b) 100% object-centered neglect


Figure 6

upright left-facing inverted right-facing

(a)

(b)

(c)


Figure 7upright left-facing inverted right-facing

(a)

(b)

(c)


Figure 8

(a) (b)


Figure 9

(a) (b)


Figure 10

(a) 100% viewer- centered (b) 100% object- centered


Figure 11(a)

(b)

(c)


Figure 12(a) Unconnected daisies (b) Commected daisy

Patient 1:

Model

Patient 2:

Model:(75 % object 25% viewer)

the interaction of spatial reference frames and...

Documents