chapter 13 dorsal and ventral cortical pathways for visuo ...panlab/papers/jtwks_innmjkj.pdf ·...

Chapter 13Dorsal and Ventral Cortical Pathwaysfor Visuo-haptic Shape IntegrationRevealed Using fMRI

Thomas W. James and Sunah Kim

13.1 Introduction

Visual object recognition is pervasive and central to many aspects of humanfunctioning. In adults, it seems effortless and nearly automatic. Despite the easewith which we perceive and identify objects, however, computer simulations ofobject recognition have been largely unsuccessful at mimicking human recognition.Simulations can succeed in constrained environments, but cannot match the flexibil-ity of the human system. One reason machine vision may have had limited successoutside of highly constrained contexts is that visual recognition is an extremelydifficult computational problem (Lennie, 1998). Another reason, however, may bethat computational approaches have largely restricted themselves to modeling thevisual system in isolation from other sensory and motor systems, whereas humanvisual recognition is embedded in interactions between multiple sensory systems(Clark, 1997; de Sa and Ballard, 1998). Although research of multisensory phe-nomena has a long history (Molyneux, 1688), research into the neural mechanismsof sensory processes in humans and other primates has been dominated in recentyears by investigations of unisensory visual function. This has led to a relativepaucity of empirical data from – and theoretical treatment of – other sensory sys-tems and, perhaps most importantly, interactions between multiple sensory systems.Our goals in this chapter are twofold. First, we describe an influential theoreticalperspective on the organization of the cortical visual system, the two visual streamstheory, and apply that perspective to interactions between visual and haptic objectshape processing. Second, using a new methodology, we assess neuronal conver-gence of visual and haptic inputs in regions considered part of those two separablepathways.

T.W. James (B)Department of Psychological and Brain Sciences, Cognitive Science Program, Indiana University,1101 E Tenth St., Bloomington, IN 47405, USAe-mail: [email protected]

231M.J. Naumer, J. Kaiser (eds.), Multisensory Object Perception in the Primate Brain,DOI 10.1007/978-1-4419-5615-6_13, C© Springer Science+Business Media, LLC 2010

232 T.W. James and S. Kim

13.2 Visual Cortical Pathways for Action and Perception

For almost three decades, one of the most dominant theories of visual systemorganization has been the two visual streams theory (Goodale and Milner, 1992;Ungerleider and Mishkin, 1982). There are two prominent sets of visual projec-tions in the primate cerebral cortex: the ventral stream, which arises in area V1 andprojects to the inferotemporal cortex, and the dorsal stream, which also arises inarea V1 and projects to the posterior parietal cortex (Baizer et al., 1991; Morel andBullier 1990; Ungerleider and Mishkin, 1982; Young, 1992). In the early 1990s,Goodale and Milner (Goodale and Milner, 1992; Milner and Goodale, 1995) arguedthat the ventral stream plays the major role in constructing our perceptual represen-tation of the visual world and the objects within it, while the dorsal stream mediatesthe visual control of actions that we direct at those objects. This idea was differ-ent from the initial conceptualization of “what–where” dual pathways (Ungerleiderand Mishkin, 1982). The what–where hypothesis suggested that both streams wereinvolved in perception, but different aspects of perception. The ventral (what) streamwas intimately involved in identifying objects, whereas the dorsal (where) streamwas involved in locating them in space.

In the Goodale and Milner model, the same information (that is, the same rudi-mentary features of objects) is processed by both streams, but for different purposes.In other words, the input to both streams is the same, but the outputs are different.Because the inputs are the same and the outputs are different, the calculations andtransformations that each stream performs on the input must be different. In the caseof the ventral stream, object features and parameters are transformed in such a waythat it produces our phenomenological experience of the world, allowing us to delib-erate and reason about our choice of actions. In the case of the dorsal stream, thesame object information is transformed in such a way that it is useful for controllingthose actions.

Some of the most compelling evidence for Goodale and Milner’s perception–action hypothesis has come from studies of patient DF, a young woman who sufferedirreversible brain damage in 1988 as a result of hypoxia from carbon monoxidepoisoning (Milner et al., 1991). Studies of DF’s visual abilities have shown that DFis unable to report the size, shape, and orientation of an object, either verbally ormanually. On the other hand, an analysis of her visuo-motor abilities demonstratesthat she shows normal pre-shaping and rotation of her hand when reaching out tograsp the same objects. In other words, DF is able to use visual information about thelocation, size, shape, and orientation of objects to control her grasping movements(and other visually guided movements), despite the fact that she is unable to perceiveand report those same object features. As a concrete example of this dissociation, DFcan orient her hand correctly to grasp a rectangular plaque that is placed in front ofher at different orientations, as assessed by biomechanical measurements. However,when presented with those same plaques and asked to report the orientation withoutacting on the object, she cannot do it (Goodale et al., 1991). In fact, she cannoteven judge whether a grating stimulus presented on a computer screen is vertical orhorizontal (Humphrey et al., 1995).

13 Dorsal and Ventral Cortical Pathways 233

Structural MRI of DF’s brain shows that her lesion is relatively focal and islocated bilaterally in an area known as the lateral occipital complex (LOC) (Jameset al., 2003). The LOC has been studied in healthy individuals for almost 15 years.The LOC is a region on the lateral surface of the cortex at the junction of the occip-ital and temporal lobes (Malach et al., 1995). The results of many studies haveconfirmed that the LOC produces more activation with intact objects than with anyother class of control stimuli tested. Although the mechanisms of object recogni-tion are unknown and remain the source of intense study, researchers agree that theLOC is intimately involved in visual object recognition, categorization, and naming(Grill-Spector et al., 2001; James et al., 2000; Kourtzi et al., 2003). The location ofDF’s lesion site and her inability to recognize objects suggests not only that the LOCis involved in visual object recognition but also that an intact LOC is necessary forrecognition of objects.

Although the lesion to DF’s occipito-temporal cortex was relatively focal, therewas evidence of smaller scale atrophy throughout the brain, as indicated by enlargedsulci and ventricles (James et al., 2003). Despite the abnormal structural appearanceof the atrophied regions, however, BOLD responses from those regions appearednormal. This was in dramatic contrast with the lesion sites, in which the signalresembled those from cerebral spinal fluid, that is, there was no BOLD response.One of the areas that showed striking similarity of functional response between DFand healthy control subjects was the intraparietal sulcus (IPS). In humans and otherprimates, the IPS has been implicated in planning object-directed actions and spa-tially representing the environment (Culham and Valyear, 2006; Culham et al., 2006;Frey et al., 2005; Grefkes and Fink, 2005). DF is able to use visual input to guide herobject-directed actions, such as reaching and grasping. When she performed grasp-ing actions in the MRI, BOLD activation in her IPS region resembled activation seenin healthy control subjects (James et al., 2003). Thus, DF’s case provides clear andconverging evidence from behavioral and neuroimaging measures for a separationof function between the dorsal and the ventral visual cortical streams.

The idea of two separate processing pathways, either the what–where hypothesisor the action–perception hypothesis, has been extremely influential in the study ofvision, but it has also influenced the study of other sensory systems. A dual-streamsapproach has been adopted by other researchers to explain the organization of theauditory system (Arnott et al., 2004; Hickok and Poeppel, 2004; Saur et al., 2008)and somatosensory system (Dijkerman and de Haan, 2007; James et al., 2007; Reedet al., 2005).

13.3 Converging Visual and Somatosensory Pathways

Like the visual system, the somatosensory system is organized hierarchically andpotentially into two or more separate pathways. There are at least three differ-ent two-stream hypotheses that describe the organization of the neural substratesinvolved in haptic exploration of the environment and, specifically, exploration of


Fig. 13.1 Three two-stream models of haptic object processing. Light gray arrows representvisual streams originating in posterior occipital cortex. Dark gray arrows represent somatosen-sory streams originating in post-central cortex. Solid arrows denote ventral stream projections,whereas dashed arrows denote dorsal stream projections. Transparent circles represent zones ofmultisensory convergence. Abbreviations: LOC, lateral occipital cortex; IPS, intraparietal sulcus;INS, insula; PPC, posterior parietal cortex, SII, secondary somatosensory cortex

physical objects (Dijkerman and de Haan, 2007; James et al., 2007; Reed et al.,2005). The first model (Fig. 13.1a) contends that haptic signals that code objectidentity are processed in a ventral pathway, which projects from primary somatosen-sory cortex to the inferior parietal lobe and prefrontal cortex, whereas haptic signalsthat code the spatial location of objects are processed in a dorsal pathway, whichprojects from primary somatosensory cortex to the superior parietal lobe (Reedet al., 2005). This theory is similar to the what–where hypothesis in vision, sepa-rating the processing of identity from the processing of location (Ungerleider andMishkin, 1982).

The second model (Fig. 13.1b) contends that processing of haptic signals forobject recognition and perception is accomplished by a ventral pathway, whichprojects from primary and secondary somatosensory cortex to the insula, whereasprocessing of haptic signals for motor actions is accomplished by a dorsal pathway,which projects from primary and secondary somatosensory cortex to the posteriorparietal cortex (Dijkerman and de Haan, 2007). This theory is similar to the visualaction–perception hypothesis, separating the processing of objects for perceptualjudgments and the processing of objects for actions. An important consideration inthis model is the interaction between the dorsal and the ventral pathways. Although


the visual action–perception hypothesis allows for interactions between the dorsaland the ventral streams, Dijkerman and colleagues suggest that these interactionsshould be stronger for haptic streams than for vision. A second consideration is theformation of a body representation or body sense. Representations in vision wouldtend to be of the external environment, even though the receptors are located withinthe body. Representations of one’s own body are completely internal, which maymake this type of representation different from visual representations. A third con-sideration is that the dorsal and ventral streams for vision and haptics may have sitesof convergence within the cortex, that is, the pathways may converge at specific sitesto integrate information, and these sites may be specific to dorsal and ventral streamfunction.

The third model (Fig. 13.1c) specifically addresses the convergence of two hapticand two visual streams (James et al., 2007). Haptic object processing is organizedinto two streams: a ventral stream that projects from the primary somatosensory cor-tex to the lateral occipito-temporal cortex (LOC) and a dorsal stream that projectsfrom primary somatosensory cortex to the posterior parietal cortex, specifically theintraparietal sulcus (IPS). These two cortical sites (LOC and IPS) mark pointsof convergence between the visual and the haptic streams of object processing.Convergence of the dorsal visual and haptic pathways is specialized for processingmultisensory shape cues to plan object-directed motor actions, whereas convergenceof the ventral visual and haptic pathways is specialized for processing multisensoryshape cues for object perception, which in turn allows for deliberation and reasoningabout our choice of actions directed toward those objects.

Findings to support these models come from a combination of behavioural andneuroimaging studies with patients and healthy subjects, and neurophysiologicalsingle-unit recording in nonhuman primates. In the late 1990s, a group of cross-modal haptic–visual priming studies (Easton et al., 1997a, b; Reales and Ballesteros,1999) changed conceptions in cognitive psychology of how object shape maybe represented. In these studies, previous experience with an object facilitatedsubsequent performance when naming that object. The important finding was that(in most cases) the facilitation occurred whether the sensory modalities of the initialexperience and the subsequent test matched or mismatched. The findings of thesestudies, and subsequent studies (Newell et al., 2001; Norman et al., 2004), suggestedthat a common representation of shape was used by vision and haptics.

Subsequently, several fMRI studies demonstrated that haptic object recognitionrecruited areas of putative visual cortex (Amedi et al., 2001; James et al., 2002; Reedet al., 2004; Sathian et al., 1997), suggesting an overlap of visual and haptic neuralrepresentations for objects. The visual cortical region most consistently recruitedin these studies was a sub-region of the LOC, which has been labeled by some asLOtv, for tactile–visual (Amedi et al., 2002). The visual and somatosensory systemsprocess many characteristics of objects. One of the most salient characteristics ofobjects is their shape (texture is also salient, but shape seems to be the most salient).The results of a number of studies suggest that the key characteristic of objects thatleads to recruitment of LOtv is their shape (Amedi et al., 2007; James et al., 2002;Stilla and Sathian, 2008). Some studies also suggest that shape is the most important


characteristic for IPS (Culham et al., 2006; Grefkes et al., 2002; Kitada et al., 2006;Peltier et al., 2007). Thus, the processing in LOtv (and perhaps IPS) may not be“visual” or even “visuo-haptic,” but instead may be “metamodal.” In other words,shape information may be processed in LOtv regardless of input modality (Amediet al., 2007; Pascual-Leone and Hamilton, 2001).

Behavioral data collected from patients with visual agnosia converge with theseneuroimaging findings (Feinberg et al., 1986; James et al., 2005; Morin et al., 1984;Ohtake et al., 2001). For instance, patient DF’s lesion in the occipito-temporal cortexoverlaps with the location of LOC in healthy subjects and has impaired her abilityto name or match objects visually, especially when those judgments must be madebased on an object’s shape (Humphrey et al., 1994; James et al., 2003). However,DF’s haptic object recognition ability is also impaired compared to healthy subjects.On three separate tasks, old/new recognition, sequential matching, and paired asso-ciates, DF was equally impaired when using vision or haptics (James et al., 2005).In addition to these patient lesion data, transcranial magnetic stimulation (TMS) hasbeen used to produce “transient virtual lesions” in the cortex of healthy individuals.Disrupting neural processing in the occipital cortex caused impairments on a tac-tile orientation discrimination task (Zangaladze et al., 1999). Taken together, thesebehavioral and neuroimaging findings from patients and healthy subjects convergeto suggest that the visual and haptic systems share overlapping neural substrates forobject recognition based on shape analysis (Amedi et al., 2005; James et al., 2007).The most likely candidate for that neural substrate is the LOtv, which resides in whatis considered the ventral perceptual stream of visual processing.

Even more intuitive than the convergence of vision and haptics for the recog-nition of objects is the convergence of vision and haptics for manual interactionwith objects. Haptic feedback plays a large role in the calibration of visuo-motoractions (Coats et al., 2008). Neuroimaging studies have shown that at least onearea of the intraparietal sulcus (IPS) is involved in bi-modal visuo-haptic process-ing of shape or geometric properties of objects (Bodegard et al., 2001; Culham andKanwisher, 2001; Grefkes et al., 2002; Peltier et al., 2007; Roland et al., 1998;Zhang et al., 2004). The IPS is also intimately involved in the planning and exe-cution of sensorimotor actions, including eye movements, pointing, reaching, andgrasping (Culham and Valyear, 2006; Grefkes and Fink, 2005). Of particular rele-vance are the anterior and middle aspects of the IPS. The functional significance ofthese areas is broad, including the preparation of grasping movements, sensitivityto visual or haptic input, and the processing of object shape and size. Patients withdamage to IPS can suffer from tactile apraxia, which is characterized by an inabilityto recognize objects haptically due to inappropriate use of exploratory movements(Binkofski et al., 1998, 2001; Pause, 1989). These converging lines of evidence haveled researchers to conclude that IPS is a site of convergence for several inter-relatedsensorimotor processes that rely on visual, haptic, and motor information to analyzeobject shape.

Data from nonhuman primate single-unit recordings strongly support the claimfor bi-modal visual and somatosensory processing in IPS (Buneo et al., 2002;Murata et al., 2000; Taira et al., 1990), which is considered homologous with IPS


in humans (Grefkes and Fink, 2005; Grefkes et al., 2002). Support for bi-modalprocessing in LOtv from single-unit recording data, however, is much less sure.One issue is that a nonhuman primate homologue for LOtv has not been established(Tootell et al., 2003). But, despite the issue of homology, there is some evidence thatneurons in the ventral visual stream of nonhuman primates do receive both visualand tactile inputs (Maunsell et al., 1991). Thus, the current state of research intovisual and haptic shape processing pathways in humans suggests that each sensorysystem has two functionally specialized cortical pathways, and these pathways con-verge on at least two separate cortical locations, LOtv and IPS. LOtv is involvedin visual and haptic processing of shape information for the purpose of recognitionand perception, whereas IPS is responsible for processing visual and haptic shapeinformation for the purposes of guiding object-directed actions. What has not beendirectly tested, however, is the manner in which signals from the two sensory inputsare combined or integrated in these areas.

13.4 Measuring Neuronal Convergence with BOLD fMRI

When describing the convergence of sensory inputs onto brain regions, researchersin the field of multisensory neurophysiology distinguish between two types ofconvergence: areal convergence and neuronal convergence (Meredith, 2002). Indescribing the research on convergence of visual and haptic inputs in the preced-ing sections, this distinction was not made. Areal convergence describes the casewhen different sensory inputs project to neurons in the same brain region, but do notsynapse on the exact same neurons. Because the inputs do not synapse on the sameneurons, there is no interaction or integration of the inputs. Neuronal convergence,on the other hand, describes the case when inputs from different sensory systemsproject to the same neurons. By synapsing on the same neurons, the inputs interact atthe neural level and can be integrated. Areal convergence and neuronal convergenceare relatively simple to dissociate with single-unit recording. If a neuron changes itsactivity when the animal is simultaneously stimulated through two sensory modali-ties compared to only one, then the neuron is integrating those inputs (Meredith andStein, 1983; Stein and Stanford, 2008). Because single-unit recording is difficultor impossible to perform in humans, multisensory integration in the human brainhas been investigated using functional neuroimaging techniques. Because BOLDfMRI activation is measured from clusters of voxels that represent large popula-tions of neurons, distinguishing between areal and neuronal convergence with fMRIinvokes a different set of criteria than with single units. Because fMRI is a newermethodology than single-unit recording, these criteria are not as well established.

One issue with predicting the strength BOLD activation with multisensory stim-uli is that populations of neurons in known multisensory cortical regions containunisensory as well as multisensory neurons (Barraclough et al., 2005; Beneventoet al., 1977; Hikosaka et al., 1988). Under this assumption, the null hypothesis tobe rejected is that a multisensory stimulus produces activation equivalent to the


sum of that produced by the unisensory components (Calvert et al., 2000; Laurientiet al., 2005). This is because the combination stimulus should excite the unisensoryneurons at least as effectively as the component stimuli. Only if the combinationstimulus produces more activation than this additive null hypothesis (“superadi-tivity”), do the results imply an interaction between sensory streams. Based onthe known neurophysiology, the most likely interpretation of an interaction is thatthere is a third pool of multisensory neurons in the population, in addition to thetwo unisensory pools. To be clear, this hypothetical third pool of neurons includesany neuron with a response that is not considered unisensory. Thus, the third poolincludes all types of multisensory neurons, including those that show multisen-sory enhancement and those that show suppression, those with linear or additiveresponses with multisensory stimuli, and those with nonlinear responses with mul-tisensory stimuli. Known sites of multisensory convergence, such as the superiortemporal sulcus (STS) for audio-visual stimuli, have many different types of mul-tisensory neurons combined with unisensory neurons. Thus, a site like STS shouldproduce a pattern of activation that rejects the null hypothesis of only two pools ofunisensory neurons. In practice, though, known sites of multisensory convergencelike STS rarely show statistically significant evidence of superadditivity with BOLDfMRI signals (Beauchamp, 2005b; Stevenson et al., 2007). Thus, other factors mustplay a role in determining whether or not BOLD fMRI measurements can detect thepresence of a pool of multisensory neurons.

It is generally understood that BOLD fMRI measurements lack a natural zerovalue or well-defined baseline (Raichle et al., 2001; Stark and Squire, 2001).Because of this constraint, BOLD is considered a “relative” measure of neural acti-vation, rather than an absolute measure. In other words, only differences in BOLDresponse between conditions are meaningful, not the absolute levels. It is possiblethat the use of absolute BOLD values in the calculation of an additive criterionhas led to a lack of consistency across studies assessing multisensory integration ofspecific brain regions.

The influence of an arbitrary baseline on the superadditive criterion is illustratedgraphically in Fig. 13.2. The two top graphs show raw BOLD data collected in twodifferent experiments. Because this is a simulation, we can make the data in the twoexperiments the same, except for one important factor, which is that experimentersfor the different experiments have chosen to use different baselines (or have doneso unknowingly). In Experiment 1, the baseline is slightly below the “true base-line” (natural zero), and in Experiment 2, the baseline is slightly above the “truebaseline.” Raw BOLD activation for the multisensory stimulus condition (VH) issimulated based on a neural population composed of only unisensory visual andunisensory haptic neurons. Thus, the result of both experiments should be to fail toreject the null hypothesis. The established practice in fMRI is to convert raw BOLDvalues into percent signal change values by subtracting the value of the baselinecondition and then dividing by it. The bottom two graphs in Fig. 13.2 show the datafrom the top two graphs after undergoing this transformation. Recall that the onlydifference between experiments was the different baseline activation; therefore, dif-ferences between the left and right bottom graphs are due only to a difference in the


Fig. 13.2 The influence of baseline activation on the absolute additive criterion. (a) and (b) Theheight of stacked bars indicates the contribution of different factors to the raw BOLD signal. Darkgray bars indicate an arbitrary value added during the reconstruction of MRI images. Horizontaland vertical lines indicate the contribution of neural activity from visual (V) or haptic (H) sensorychannels, respectively. The light gray bar is the BOLD activation produced by the “baseline” con-dition. (c) and (d) Percent BOLD signal change calculated based on the different baseline activationvalues shown in (a) and (b), respectively. Signal change values are proportional to the differencebetween the total height of the stacked bars and the dotted line indicating the level of baseline acti-vation. The Sum (V,H) is the absolute additive criterion. (c) Superadditivity and (d) Subadditivity

baseline. The signal change values in the two graphs are clearly different. Both ofthese experiments would reject the null hypothesis, but the effects are in the oppositedirection. More alarming is that rejecting the null hypothesis would reveal nothingabout the underlying neural populations, but is completely dependent on the activa-tion of the baseline condition. It is possible that this type of influence may explainthe inconsistency in results from different research groups using superadditivity as acriterion (Beauchamp, 2005a, b; Beauchamp et al., 2004a, b; Laurienti et al., 2006;Peiffer et al., 2007; Stevenson and James, 2009; Stevenson et al., 2007).

Because absolute BOLD measurements produce inconsistent results when usedas a criterion for the assessment of multisensory integration, we have recently turnedto using relative differences in BOLD activation. The use of relative differencesalleviates the problem of an indeterminate baseline, because the baseline compo-nents embedded in the two measurements cancel out when a difference operation isperformed. The null hypothesis for these BOLD differences is similar to absoluteBOLD measurements and follows a similar hypothesis of additivity, that is, the null


hypothesis is the sum of the two unisensory differences. If the multisensory differ-ence differs from the null, one can infer an interaction between sensory channels inthe form of a third pool of multisensory neurons using the same logic applied to thesuperadditive null hypothesis. The benefit of using differences, however, is that theyare not susceptible to changes in baseline.

BOLD differences can be calculated across any manipulation of the stimulusor task that produces a systematic, monotonic change in BOLD activation. Forinstance, BOLD differences have been successfully used with audio-visual stim-uli for which the signal-to-noise ratio (SNR) was varied (Stevenson and James,2009). In that study, unisensory audio and visual stimuli produced less BOLD acti-vation in the superior temporal sulcus (STS) as stimuli were degraded by loweringthe SNR. Multisensory stimuli also showed less activation with reduced SNR, butthe decrease in activation was not as large as predicted by the null hypothesis.That is, the BOLD difference for multisensory stimuli was less than the sum ofthe two unisensory differences. This effect on multisensory BOLD activation wascalled inverse effectiveness, because it resembled an effect often seen in single-unitrecordings taken from multisensory regions. As stimuli are degraded, they are lesseffective at stimulating unisensory and multisensory neurons. Because the relativemultisensory gain increases as effectiveness decreases, the effect is called inverseeffectiveness (Meredith and Stein, 1986). Although a pattern of inverse effective-ness in BOLD activation does not necessarily imply that neurons in that area areinversely effective, it does imply a difference from the null hypothesis, and thus aninteraction between sensory channels.

13.5 Sites of Visuo-haptic Neuronal Convergence

Although there is considerable evidence for bi-modal visual and haptic processingof object shape in the primate brain in at least two cortical sites, LOtv and IPS, untilvery recently (Tal and Amedi, 2009) a test for neuronal convergence in humansusing fMRI had not been reported. Using the BOLD differences method describedabove, we designed an experiment to test for neuronal convergence of visual andhaptic inputs in the LOC and IPS (Kim and James in press). Our stimulus manipu-lation was to vary the level of stimulus quality. Finding a significant “difference ofdifferences” across levels of stimulus quality would confirm the presence of multi-sensory integration, even in the absence of superadditivity. Based on previous resultswith audio-visual integration using this method, we hypothesized that the directionof that difference would be in the direction predicted by inverse effectiveness, thatis, as stimulus quality was reduced, there should be a smaller drop in multisensoryactivation than predicted based on the drop in unisensory activation.

We localized ROIs using a standard method that contrasts visual objects (VO)with visual textures (VT) and haptic objects (HO) with haptic textures (HT) (Amediet al., 2001). The visual contrast is the same standard functional localizer used toisolate the LOC visually (Malach et al., 1995). The haptic contrast typically activates


a large cluster along the entire IPS. A conjunction of the two contrasts isolatesoverlapping regions that are object selective for both sensory modalities. In the ven-tral stream, the overlapping region is labeled LOtv and is found consistently withthe conjunction contrast. An overlapping region in the dorsal stream is less consis-tently found with the conjunction contrast (Amedi et al., 2005; Lacey et al., 2009),perhaps because researchers do not control the parameters of active exploration.Nevertheless, our use of the conjunction contrast did find statistically reliable clus-ters of voxels in the IPS. Figure 13.3 shows the two functional ROIs, LOtv andIPS, localized on the group-average functional data (N = 7) and superimposed ongroup-average anatomical images. The analysis used to produce the maps was aconjunction of four contrasts, HO–HT and VO–VT and HO–VT and VO–HT. Thisanalysis localizes regions that are bi-modal and object selective and that also haveequal contribution from visual and haptic conditions. Images on the left and rightof Fig. 13.3 are shown at different statistical thresholds. At the more conservativethreshold (Fig. 13.3a, c), the right-hemisphere cluster does not survive. At the moreliberal threshold (Fig. 13.3b, d), the left-hemisphere cluster is much larger than theright. The same left-hemisphere bias for haptic or visuo-haptic object processinghas been shown in previous studies; however, the significance of the possible lat-eralization of functional is unknown. We wanted to analyze both the left and theright hemispheres and wanted to equate the size (i.e., number of voxels) of theleft- and right-hemisphere clusters. Due to the reliable left-hemisphere bias, this

Fig. 13.3 Bi-modal visuo-haptic object-selective regions of interest. The boundaries of the regionsof interest used in the analyses are outlined in bright yellow. The heights of the axial slices inTalairach space are indicated by the Z = labels


meant using a different statistical threshold for determining the left- and right-hemisphere ROIs. Thus, the left-hemisphere ROIs were the yellow-outlined clustersin Panels A and C and the right-hemisphere ROIs were the yellow-outlined clustersin Panels B and D. The large, non-outlined clusters in the left hemisphere in PanelsB and D were not analyzed. Importantly, the threshold for the left-hemisphere ROIswas set at a typically conservative value. A more liberal threshold was used only forthe right-hemisphere ROI. The results of these contrasts were consistent with pre-vious research suggesting that LOtv and IPS are instrumental in processing objectshape information (Amedi et al., 2005). They are also consistent with previous worksuggesting that LOtv and IPS are sites of convergence for visual and haptic sensoryinputs (James et al., 2007).

To assess inverse effectiveness and superadditivity in these ROIs, we presentedsubjects with novel objects from two categories and instructed them to perform atwo-alternative forced-choice decision. Sixteen objects were used. All objects werecreated by attaching four wooden geon-like geometric components in a standardconfiguration to provide differences in shape information (Biederman, 1987). Inthis experiment, only one of the components was diagnostic of category member-ship. Eight objects in Category 1 had a half-circle-shaped diagnostic component,and the other eight objects in Category 2 had a triangle-shaped diagnostic compo-nent (Fig. 13.4a). Each stimulus was 14 cm wide and 9.5 cm long. Texture on thestimuli was determined by the size of nylon beads that were glued to the surface.Textures and non-diagnostic features could not be used to perform the 2AFC task.The purpose of the different textures and non-diagnostic shape components was toadd complexity and variability to the psychological object similarity space and keepsubjects more interested in the task, but subjects were explicitly instructed to useonly the diagnostic shape feature to perform the task. The distribution of texturesand non-diagnostic components was the same across the two object categories.

For visual presentation, a grayscale picture of each stimulus was presented usingan LCD projector and rear-projection screen. Subjects viewed the images using arear-face mirror attached to the head coil. Visual stimuli were presented at 12◦ × 8◦

Fig. 13.4 Examples of procedures for degrading stimuli in visual and haptic conditions. Picturesof an undegraded object (a), a visually degraded object (b), and a haptically degraded object beingpalpated (c)


of visual angle. To establish different levels of salience, a fixed level of externalGaussian noise was added to the stimuli (Fig. 13.4b) and the stimulus signal contrastwas reduced to two different levels. These levels were calibrated independently foreach individual and reflected their 71 and 89% performance thresholds.

For haptic presentation, an angled “table” was placed on the subject’s midsection.An experimenter standing in the MRI room delivered tangible stimuli to the subjectby placing them on a designated spot on the table. Subjects palpated the objectswith eyes closed with both hands and were instructed not to lift the objects from thetable. To establish different levels of salience, subjects wore a pair of PVC gloves,which reduced their tactile sensitivity. Individual 71 and 89% performance thresh-olds were measured by covering objects with a different number of layers of thickfelt fabric (Fig. 13.4c). The layered fabric further reduced tactile sensitivity, butunlike the gloves, allowed the experimenter to rapidly change between performance(or sensitivity) levels.

Subjects and the experimenter both wore headphones and listened for a sequenceof auditory cues that indicated when subjects should start and stop hand move-ments and when the experimenter should switch out the stimulus. The final designhad two factors: stimulus quality and stimulus modality, with two levels of qual-ity (high and low) and three modalities (visual [V], haptic [H], and visuo-haptic[VH]). Seven subjects participated in the experiment. Imaging parameters and datapre-processing steps were standard and are described elsewhere (Kim and James inpress). Accuracy and reaction time measures showed strong effects of stimulus qual-ity. In the unisensory conditions, accuracy levels were close to 71% for low qualityand 89% for high quality, the performance levels for which the conditions werecalibrated. Reaction times were also slower for the low quality than high qualitycondition.

BOLD activations in left and right LOtv and IPS are shown in Fig. 13.5 for allconditions in the 2×3 design. The null hypothesis for the additive model is alsopresented for each level of stimulus quality and labeled S(V,H). It is apparent fromcomparing the VH stimulus condition to the null hypothesis that there is no evidenceof superadditivity in either LOtv or IPS. The VH stimulus condition produced sub-additive activation in all cases except for the high quality condition in right LOtv,which was additive. Thus, if superadditivity were the only criterion, we would inferfrom these data that LOtv and IPS do not represent sites of neuronal convergencefor visual and haptic sensory inputs.

Figure 13.6 shows the results of the new BOLD differences analysis, performedon the same data shown in Fig. 13.5. Instead of comparing absolute levels of BOLDactivation, recall that this analysis compares differences in BOLD activation. Thus,the height of the bars in Fig. 13.6 represents the difference in BOLD activationbetween high- and low-signal quality conditions. This difference is represented forvisual (�V), haptic (�H), and visuo-haptic (�VH) stimulus conditions. The nullhypothesis to be rejected is represented by the sum of unisensory differences (i.e.,S(�V, �H) bar). There is a clear difference between the �VH and S(�V, �H)for both LOtv and IPS in the left hemisphere. The difference in the right hemi-sphere is in the same direction, but is less robust. Based on this rejection of the


Fig. 13.5 BOLD percent signal change as a function of sensory condition, stimulus quality, regionof interest, and hemisphere shown for the following regions: a) left LOtv, b) right LOtv, c) left IPSand d) right IPS. S(V, H) represented the absolute additive criterion

null hypothesis, we can infer that the underlying neuronal population is not com-posed of two pools of unisensory neurons: one visual and one haptic. Based onthe neurophysiology of known multisensory areas, a likely inference is that thesetwo areas contain a mixture of unisensory and multisensory neurons (Meredith andStein, 1983, 1986).

Rejection of our null hypothesis, though, simply means that there was a differ-ence between �VH and the sum of �V and �H. A difference in either directionimplies an interaction between sensory modalities, possibly due to a third pool ofmultisensory neurons. But, the direction shown in Fig. 13.6 was unexpected. Oneof the general principles of single-unit recordings from multisensory neurons is thatmultisensory enhancement increases as the stimuli are degraded in quality. Thatis, the gain in activity with a multisensory stimulus over and above the activitywith a unisensory stimulus increases with decreasing quality. Low-quality stimuliare also less effective at stimulating unisensory and multisensory neurons. Thus,inverse effectiveness describes the phenomenon that as the effectiveness of a stimu-lus decreases, the multisensory gain increases. Given that inverse effectiveness is aknown principle of neural activity in multisensory neurons, we predicted that activ-ity in LOtv and IPS, if the null hypothesis was rejected, would also show evidence ofinverse effectiveness. However, the pattern of change shown in Figs. 13.5 and 13.6


Fig. 13.6 BOLD differences as a function of sensory condition, region of interest, and hemi-sphere shown for the following regions: a) left LOtv, b) right LOtv, c) left IPS and d) rightIPS. Differences were calculated as high quality minus low quality. S(�V,�H) represents theadditive-differences criterion. �VH represents the multisensory difference, which is compared toS(�V,�H) to establish the presence of enhanced or inverse effectiveness

is the opposite. The multisensory gain is stronger for high-quality stimuli than forlow-quality stimuli. It is important to note that the opposite direction of the effect isnot due to an indirect relation between stimulus quality and brain activation (effec-tiveness). That is, if high-quality stimuli produced less activation in these regionsthan low-quality stimuli, then that alone could make the change in gain appear to goin the opposite direction.

The multisensory gains shown in Figs. 13.5 and 13.6 are clearly stronger with thehigh-quality stimuli. This effect is the opposite of that seen with inverse effective-ness (Meredith and Stein, 1986). Thus, we suggest that this effect should be called“enhanced effectiveness” (Kim and James in press), because as the effectiveness ofthe stimuli is enhanced, the multisensory gain is also disproportionately enhanced.

Both LOtv and IPS showed evidence for integration of visual and haptic sensoryinputs and both LOtv and IPS showed enhanced effectiveness. The effect was in thesame direction in both the left and the right hemispheres, but was much strongerin the left. The results provide further evidence that the visual and haptic systemsprocess object shape through two pathways and that LOtv and IPS represent pointsof convergence for those two pathways (James et al., 2007). LOtv forms part ofthe ventral or perceptual pathway for vision and haptics, and IPS forms part of thedorsal or action pathway for vision and haptics. The results suggest that integrationof visual and haptic sensory inputs is similar in LOtv and IPS. This result may be


unexpected, given the emphasis placed on separable processes in the two pathways(Goodale et al., 1991). However, our results suggest only that the properties of mul-tisensory convergence are similar in the two regions, not that the underlying neuralprocesses that rely on that convergence are the same. Although speculative, our datamay suggest that the properties of multisensory convergence are similar in differ-ent areas of cortex. To more directly test this hypothesis, future studies of neuronalconvergence of visual and haptic sensory channels in ventral and dorsal pathwaysshould investigate both object recognition and object-directed actions.

13.6 Conclusions

We have reviewed the evidence for separable action and perception pathways inboth the visual and the haptic systems for the analysis of object shape. These sys-tems converge on at least two neural sites: one in the dorsal action pathway and onein the ventral perception pathway. We tested these sites of convergence for evidenceof multisensory integration of visual and haptic inputs. In the absence of superaddi-tivity, we found evidence for multisensory integration (neuronal convergence) usinga new method that employs relative BOLD differences instead of absolute BOLDvalues. Dorsal and ventral sites showed the same general pattern of multisensoryintegration. This suggests that integration of object shape information in the dorsaland ventral streams may occur by the same general mechanisms.

Acknowledgments This research was supported in part by the Faculty Research SupportProgram, administered by the Indiana University Office of the Vice President of Research, andin part by the Indiana METACyt Initiative of Indiana University, funded in part through a majorgrant from the Lilly Endowment, Inc.

References

Amedi A, Jacobson G, Hendler T, Malach R, Zohary E (2002) Convergence of visual and tactileshape processing in the human lateral occipital complex. Cereb Cortex 12:1202–1212

Amedi A, Malach R, Hendler T, Peled S, Zohary E (2001) Visuo-haptic object-related activationin the ventral visual pathway. Nat Neurosci 4:324–330

Amedi A, Stern WM, Camprodon JA, Bermpohl F, Merabet L, Rotman S, Hemond C, Meijer P,Pascual-Leone A (2007) Shape conveyed by visual-to-auditory sensory substitution activatesthe lateral occipital complex. Nat Neurosci 10:687–689

Amedi A, von Kriegstein K, van Atteveldt NM, Beauchamp MS, Naumer MJ (2005) Functionalimaging of human crossmodal indentification and object recognition. Exp Brain Res 166:559–571

Arnott SR, Binns MA, Grady CL, Alain C (2004) Assessing the auditory dual-pathway model inhumans. Neuroimage 22:401–408

Baizer JS, Ungerleider LG, Desimone R (1991) Organization of visual inputs to the inferiortemporal and posterior parietal cortex in macaques. J Neurosci 11:168–190

Barraclough NE, Xiao D, Baker CI, Oram MW, Perrett DI (2005) Integration of visual and auditoryinformation by superior temporal sulcus neurons responsive to the sight of actions. J CognNeurosci 17:377–391


Beauchamp MS (2005a) See me, hear me, touch me: multisensory integration in lateral occipital-temporal cortex. Curr Opin Neurobiol 15:145–153

Beauchamp MS (2005b) Statistical criteria in FMRI studies of multisensory integration.Neuroinformatics 3:93–113

Beauchamp MS, Argall BD, Bodurka J, Duyn JH, Martin A (2004a) Unraveling multisensoryintegration: patchy organization within human STS multisensory cortex. Nat Neurosci 7:1190–1192

Beauchamp MS, Lee KE, Argall BD, Martin A (2004b) Integration of auditory and visualinformation about objects in superior temporal sulcus. Neuron 41:809–823

Benevento LA, Fallon J, Davis BJ, Rezak M (1977) Auditory--visual interaction in single cells inthe cortex of the superior temporal sulcus and the orbital frontal cortex of the macaque monkey.Exp Neurol 57:849–872

Biederman I (1987) Recognition-by-components: a theory of human image understanding. PsycholRev 94:115–147

Binkofski F, Dohle C, Posse S, Stephan KM, Hefter H, Seitz RJ, Freund HJ (1998) Human anteriorintraparietal area subserves prehension: a combined lesion and functional MRI activation study.Neurology 50:1253–1259

Binkofski F, Kunesch E, Classen J, Seitz RJ, Freund HJ (2001) Tactile apraxia: unimodal aprac-tic disorder of tactile object exploration associated with parietal lobe lesions. Brain 124:132–144

Bodegard A, Geyer S, Grefkes C, Zilles K, Roland PE (2001) Hierarchical processing of tactileshape in the human brain. Neuron 31:317–328

Buneo CA, Jarvis MR, Batista AP, Andersen RA (2002) Direct visuomotor transformations forreaching. Nature 416:632–636

Calvert GA, Campbell R, Brammer MJ (2000) Evidence from functional magnetic resonanceimaging of crossmodal binding in the human heteromodal cortex. Curr Biol 10:649–657

Clark A (1997) Being there. MIT Press, CambridgeCoats R, Bingham GP, Mon-Williams M (2008) Calibrating grasp size and reach distance:

interactions reveal integral organization of reaching-to-grasp movements. Exp Brain Res189:211–220

Culham JC, Cavina-Pratesi C, Singhal A (2006) The role of parietal cortex in visuomotor control:what have we learned from neuroimaging? Neuropsychologia 44:2668–2684

Culham JC, Kanwisher NG (2001) Neuroimaging of cognitive functions in human parietal cortex.Curr Opin Neurobiol 11:157–163

Culham JC, Valyear KF (2006) Human parietal cortex in action. Curr Opin Neurobiol 16:205–212de Sa VR, Ballard DH (1998) Perceptual learning from cross-modal feedback. In: Goldstone RL,

Schyns PG, Medin DL (eds) Psychology of learning and motivation, vol 36. Adademic Press,San Diego, CA

Dijkerman HC, de Haan EH (2007) Somatosensory processes subserving perception and action.Behav Brain Sci 30:189–201; discussion 201–139

Easton RD, Greene AJ, Srinivas K (1997a) Transfer between vision and haptics: memory for 2-Dpatterns and 3-D objects. Psychonomic Bulletin and Review 4:403–410

Easton RD, Srinivas K, Greene AJ (1997b) Do vision and haptics share common representations?Implicit and explicit memory within and between modalities. J Exp Psychol: Learn, Mem, Cogn23:153–163

Feinberg TE, Gonzalez Rothi LJ, Heilman KM (1986) Multimodal agnosia after unilateral lefthemisphere lesion. Neurology 36:864–867

Frey SH, Vinton D, Norlund R, Grafton ST (2005) Cortical topography of human anterior intra-parietal cortex active during visually guided grasping. Brain Res Cogn Brain Res 23:397–405

Goodale MA, Milner AD (1992) Separate visual pathways for perception and action. TrendsNeurosci 15:20–25

Goodale MA, Milner AD, Jakobson LS, Carey DP (1991) A neurological dissociation betweenperceiving objects and grasping them. Nature 349:154–156


Grefkes C, Fink GR (2005) The functional organization of the intraparietal sulcus in humans andmonkeys. J Anat 207:3–17

Grefkes C, Weiss PH, Zilles K, Fink GR (2002) Crossmodal processing of object features inhuman anterior intraparietal cortex: an fMRI study implies equivalencies between humans andmonkeys. Neuron 35:173–184

Grill-Spector K, Kourtzi Z, Kanwisher N (2001) The lateral occipital complex and its role in objectrecognition. Vision Res 41:1409–1422

Hickok G, Poeppel D (2004) Dorsal and ventral streams: a framework for understanding aspectsof the functional anatomy of language. Cognition 92:67–99

Hikosaka K, Iwai E, Saito H, Tanaka K (1988) Polysensory properties of neurons in the ante-rior bank of the caudal superior temporal sulcus of the macaque monkey. J Neurophysiol 60:1615–1637

Humphrey GK, Goodale MA, Corbetta M, Aglioti S (1995) The McCollough effect revealsorientation discrimination in a case of cortical blindness. Curr Biol 5:545–551

Humphrey GK, Goodale MA, Jakobson LS, Servos P (1994) The role of surface informationin object recognition: studies of a visual form agnosic and normal subjects. Perception 23:1457–1481

James TW, Culham JC, Humphrey GK, Milner AD, Goodale MA (2003) Ventral occipitallesions impair object recognition but not object-directed grasping: an fMRI study. Brain 126:2463–2475

James TW, Humphrey GK, Gati JS, Menon RS, Goodale MA (2000) The effects of visual objectpriming on brain activation before and after recognition. Curr Biol 10:1017–1024

James TW, Humphrey GK, Gati JS, Servos P, Menon RS, Goodale MA (2002) Haptic study ofthree-dimensional objects activates extrastriate visual areas. Neuropsychologia 40:1706–1714

James TW, James KH, Humphrey GK, Goodale MA (2005) Do visual and tactile object representa-tions share the same neural substrate? In: Heller MA, Ballesteros S (eds) Touch and blindness:psychology and neuroscience. Lawrence Erlbaum, Mahwah, NJ

James TW, Kim S, Fisher JS (2007) The neural basis of haptic object processing. Can J Exp Psychol61:219–229

Kim S, James TW (in press) Enhanced effectiveness in visuo-haptic object-selective brain regionswith increasing stimulus salience. Hum Brain Mapp

Kitada R, Kito T, Saito DN, Kochiyama T, Matsumura M, Sadato N, Lederman SJ (2006)Multisensory activation of the intraparietal area when classifying grating orientation: afunctional magnetic resonance imaging study. J Neurosci 26:7491–7501

Kourtzi Z, Tolias AS, Altmann CF, Augath M, Logothetis NK (2003) Integration of local featuresinto global shapes: monkey and human fMRI studies. Neuron 37:333–346

Lacey S, Tal N, Amedi A, Sathian K (2009) A putative model of multisensory object representation.Brain Topogr 21:269–274

Laurienti PJ, Burdette JH, Maldjian JA, Wallace MT (2006) Enhanced multisensory integration inolder adults. Neurobiol Aging 27:1155–1163

Laurienti PJ, Perrault TJ, Stanford TR, Wallace MT, Stein BE (2005) On the use of superadditivityas a metric for characterizing multisensory integration in functional neuroimaging studies. ExpBrain Res 166:289–297

Lennie P (1998) Single units and visual cortical organization. Perception 27:889–935Malach R, Reppas JB, Benson RR, Kwong KK, Jiang H, Kennedy WA, Ledden PJ, Brady TJ,

Rosen BR, Tootell RB (1995) Object-related activity revealed by functional magnetic resonanceimaging in human occipital cortex. Proc Natl Acad Sci U S A 92:8135–8139

Maunsell JH, Sclar G, Nealey TA, DePriest DD (1991) Extraretinal representations in area V4 inthe macaque monkey. Vis Neurosci 7:561–573

Meredith MA (2002) On the neuronal basis for multisensory convergence: a brief overview. BrainRes Cogn Brain Res 14:31–40

Meredith MA, Stein BE (1983) Interactions among converging sensory inputs in the superiorcolliculus. Science 221:389–391


Meredith MA, Stein BE (1986) Visual, auditory, and somatosensory convergence on cells insuperior colliculus results in multisensory integration. J Neurophysiol 56:640–662

Milner AD, Goodale MA (1995) The visual brain in action. Oxford University Press, Oxford, UKMilner AD, Perrett DI, Johnston RS, Benson RS, Jordan PJ, Heeley TR, Bettucci DW, Mortara

D, Mutani F, Terazzi R, Davidson DLW (1991) Perception and action in visual form agnosia.Brain 114:405–428

Molyneux W (1688) Letter to John Locke. In: de Beer ES (ed) The correspondence of John Locke,vol 3. Clarendon Press, Oxford

Morel A, Bullier J (1990) Anatomical segregation of two cortical visual pathways in the macaquemonkey. Vis Neurosci 4:555–578

Morin P, Rivrain Y, Eustache F, Lambert J, Courtheoux P (1984) Visual and tactile agnosia. RevNeurol 140:271–277

Murata A, Gallese V, Luppino G, Kaseda M, Sakata H (2000) Selectivity for the shape, size, andorientation of objects for grasping in neurons of monkey parietal area AIP. J Neurophysiol83:2580–2601

Newell FN, Ernst MO, Tjan BS, Bulthoff HH (2001) Viewpoint dependence in visual and hapticobject recognition. Psychol Sci 12:37–42

Norman JF, Norman HF, Clayton AM, Lianekhammy J, Zielke G (2004) The visual and hapticperception of natural object shape. Percept Psychophys 66:342–351

Ohtake H, Fujii T, Yamadori A, Fujimori M, Hayakawa Y, Suzuki K (2001) The influence ofmisnaming on object recognition: a case of multimodal agnosia. Cortex 37:175–186

Pascual-Leone A, Hamilton R (2001) The metamodal organization of the brain. Prog Brain Res134:427–445

Pause M, Kunesch E, Binkofsku F, Freund HJ (1989) Sensorimotor disturbances in patients withlesions of the parietal corex. Brain 112:1599–1625

Peiffer AM, Mozolic JL, Hugenschmidt CE, Laurienti PJ (2007) Age-related multisensoryenhancement in a simple audiovisual detection task. Neuroreport 18:1077–1081

Peltier S, Stilla R, Mariola E, LaConte S, Hu X, Sathian K (2007) Activity and effective connectiv-ity of parietal and occipital cortical regions during haptic shape perception. Neuropsychologia45:476–483

Raichle ME, MacLeod AM, Snyder AZ, Powers WJ, Gusnard DA, Shulman GL (2001) A defaultmode of brain function. Proc Natl Acad Sci U S A 98:676–682

Reales JM, Ballesteros S (1999) Implicit and explicit memory for visual and haptic objects: corss-modal priming depends on structural descriptions. J Exp Psychol: Learn, Mem, Cogn 25:644–663

Reed CL, Klatzky RL, Halgren E (2005) What vs. where in touch: an fMRI study. Neuroimage25:718–726

Reed CL, Shoham S, Halgren E (2004) Neural substrates of tactile object recognition: an fMRIstudy. Hum Brain Mapp 21:236–246

Roland PE, O′Sullivan B, Kawashima R (1998) Shape and roughness activate different somatosen-sory areas in the human brain. Proc Natl Acad Sci 95:3295–3300

Sathian K, Zangaladze A, Hoffman JM, Grafton ST (1997) Feeling with the mind’s eye.Neuroreport 8:3877–3881

Saur D, Kreher BW, Schnell S, Kummerer D, Kellmeyer P, Vry MS, Umarova R, Musso M,Glauche V, Abel S, Huber W, Rijntjes M, Hennig J, Weiller C (2008) Ventral and dorsalpathways for language. Proc Natl Acad Sci U S A 105:18035–18040

Stark CE, Squire LR (2001) When zero is not zero: the problem of ambiguous baseline conditionsin fMRI. Proc Natl Acad Sci U S A 98:12760–12766

Stein BE, Stanford TR (2008) Multisensory integration: current issues from the perspective of thesingle neuron. Nat Rev Neurosci 9:255–266

Stevenson RA, Geoghegan ML, James TW (2007) Superadditive BOLD activation in superiortemporal sulcus with threshold non-speech objects. Exp Brain Res 179:85–95


Stevenson RA, James TW (2009) Audiovisual integration in human superior temporal sulcus:Inverse effectiveness and the neural processing of speech and object recognition. Neuroimage44:1210–1223

Stilla R, Sathian K (2008) Selective visuo-haptic processing of shape and texture. Hum Brain Mapp29:1123–1138

Taira M, Mine S, Georgopoulos AP, Murata A, Sakata H (1990) Parietal cortex neurons of themonkey related to the visual guidance of hand movement. Exp Brain Res 83:29–36

Tal N, Amedi A (2009) Multisensory visual-tactile object related network in humans: insightsgained using a novel crossmodal adaptation approach. Exp Brain Res 198:165–182

Tootell RB, Tsao D, Vanduffel W (2003) Neuroimaging weighs in: Humans meet macaques in“primate” visual cortex. J Neurosci 23:3981–3989

Ungerleider LG, Mishkin M (1982) Two cortical visual systems. In: Ingle DJ, Goodale MA,Mansfield RJ (eds) The analysis of visual behavior. MIT Press, Cambridge, MA, pp 549–586

Young MP (1992) Objective analysis of the topological organization of the primate cortical visualsystem. Nature 358:152–155

Zangaladze A, Epstein CM, Grafton ST, Sathian K (1999) Involvement of visual cortex in tactilediscrimination of orientation. Nature 401:587–590

Zhang M, Weisser VD, Stilla R, Prather SC, Sathian K (2004) Multisensory cortical processing ofobject shape and its relation to mental imagery. Cogn Affect Behav Neurosci 4:251–259

chapter 13 dorsal and ventral cortical pathways for visuo ...panlab/papers/jtwks_innmjkj.pdf ·...

Documents