the ventriloquist in periphery: impact of eccentricity …...the ventriloquist in periphery: impact...

The ventriloquist in periphery: Impact of eccentricity-relatedreliability on audio-visual localization

Genevieve Charbonneau $

Centre de Recherche en Neuropsychologie et Cognition(CERNEC), Departement de Psychologie,

Universite de Montreal, Montreal, Canada

Marie Veronneau $



Colin Boudrias-Fournier $



Franco Lepore $



Olivier Collignon # $

Center for Mind/Brain Sciences (CIMeC),University of Trento, Trento, Italy



The relative reliability of separate sensory estimatesinfluences the way they are merged into a unifiedpercept. We investigated how eccentricity-relatedchanges in reliability of auditory and visual stimuliinfluence their integration across the entire frontalspace. First, we surprisingly found that despite a strongdecrease in auditory and visual unisensory localizationabilities in periphery, the redundancy gain resulting fromthe congruent presentation of audio-visual targets wasnot affected by stimuli eccentricity. This result thereforecontrasts with the common prediction that a reductionin sensory reliability necessarily induces an enhancedintegrative gain. Second, we demonstrate that the visualcapture of sounds observed with spatially incongruentaudio-visual targets (ventriloquist effect) steadilydecreases with eccentricity, paralleling a lowering of therelative reliability of unimodal visual over unimodalauditory stimuli in periphery. Moreover, at alleccentricities, the ventriloquist effect positivelycorrelated with a weighted combination of the spatialresolution obtained in unisensory conditions. These

findings support and extend the view that thelocalization of audio-visual stimuli relies on an optimalcombination of auditory and visual informationaccording to their respective spatial reliability. Alltogether, these results evidence that the external spatialcoordinates of multisensory events relative to anobserver’s body (e.g., eyes’ or head’s position) influencehow this information is merged, and thereforedetermine the perceptual outcome.

Introduction

Through the integration of information acquired bythe different senses, our brain is able to construct aunified and more robust representation of our envi-ronment. The respective reliability of distinct sensoryinputs strongly influences how they are merged in asingle coherent percept (Sumby & Pollack, 1954).Probably one of the best examples of how the

Citation: Charbonneau, G., Veronneau, M., Boudrias-Fournier, C., Lepore, F., & Collignon, O. (2013). The ventriloquist inperiphery: Impact of eccentricity-related reliability on audio-visual localization. Journal of Vision, 13(12):20, 1–14, http://www.journalofvision.org/content/13/12/20, doi:10.1167/13.12.20.

Journal of Vision (2013) 13(12):20, 1–14 1http://www.journalofvision.org/content/13/12/20

doi: 10 .1167 /13 .12 .20 ISSN 1534-7362 � 2013 ARVOReceived April 29, 2013; published October 28, 2013

mailto:[email protected]








https://sites.google.com/site/collignonlab/

https://sites.google.com/site/collignonlab/



perceptual system deals with intersensory reliability isthe ventriloquist effect (Howard & Templeton, 1966),in which the localization of auditory information isbiased in the direction of a synchronously presented,but spatially misaligned, visual stimulus. In thisperceptual situation, because the visual system typicallyprovides the more accurate and reliable spatialinformation, the brain attributes more weight to visionin localizing the audio-visual event, thus inducing avisual capture of acoustic space (Pick, Warren, & Hay,1969). This effect explains why although a movieactor’s voice comes from loud-speakers far away fromthe screen, our brain recalibrates this discrepancy togive us the false impression that sound is actuallycoming from his mouth. Such visual capture does notoccur in a rigid, hardwired manner, but follows flexiblesituation-dependent rules that allow information to becombined with maximal efficacy. Some experimentsdemonstrated that when the reliability of visual input isreduced, for example when visual information isblurred (Alais & Burr, 2004), presented at a lowperceptual threshold (Bolognini, Leo, Passamonti,Stein, & Ladavas, 2007), near the onset of a visualsaccade (Binda, Bruno, Burr, & Morrone, 2007) ordegraded with myopia-inducing lenses (Hairston,Laurienti, Mishra, Burdette, & Wallace, 2003), therelative reliability of the auditory spatial cues increasesand more weight is attributed to sounds in thelocalization of bimodal stimuli. Therefore, the ventril-oquist effect may be considered as a specific example ofnear-optimal statistical (Bayesian) combination ofvisual and auditory spatial cues, where each cue isweighted in proportion of its relative reliability, ratherthan one modality capturing the other (Alais & Burr,2004; Battaglia, Jacobs, & Aslin, 2003; Ernst &Bulthoff, 2004).

In daily life, a frequent situation inducing a decreasein visual and auditory reliability is when the same eventappears in periphery rather than in the central spatialfield relative to our sensory receptors. Auditory infor-mation is less reliably localized when it is presented inthe periphery versus in front of the head (Barfield,Cohen, & Rosenberg, 1997; Makous & Middlebrooks,1990; Nguyen, Suied, Viaud-Delmon, & Warusfel, 2009;Thurlow & Runge, 1967) due to change in interauralcues (Middlebrooks, 1991). Similarly, it is more difficultfor a human observer to localize visual inputs whenpresented in the periphery compared to the center of thevisual field because the central (foveal) part of retinacontains a greater density of cone photoreceptorscompared to the retinal periphery, leading to a morefine-grained representation of the information beingprocessed (Baizer, Ungerleider, & Desimone, 1991;Dacey & Petersen, 1992; Stephen et al., 2002).

Such systematic reduction in the ability of the humanobserver to locate peripheral auditory and visual targets

therefore affords a unique ecological opportunity toinvestigate how the reliability of sensory stimulationsimpact audio-visual integration across space (Collignon,2007). Surprisingly, if the impact of the reliability onaudio-visual integration has been massively investigatedin the central visual field, only a few studies exploredthis phenomenon throughout the frontal space. Hair-ston and collaborators (2003b) studied how the absolutelocation in space modifies the ability of an irrelevantvisual signal to influence the localization of an auditorytarget. They showed that auditory localization is moreaffected by the simultaneous presentation of a visualinput coming from the center rather than from theperiphery of the visual field. However, because theseauthors did not explore the far periphery (þ308), andbecause their paradigm was not designed to jointlyinvestigate the redundancy gain (presentation of spa-tially aligned multisensory sources) and sensory capture(presentation of spatially misaligned multisensorysources), the impact of eccentricity-related reliability onaudio-visual spatial integration remains elusive.

The goal of the present study was therefore toinvestigate further how spatial signals acquired bydifferent sensory systems are integrated in the contextof their position relative to the observer. To do so, weused a design allowing us to probe how the position ofaudio-visual targets in the entire frontal space (1808)affect the cross-modal capture and the multisensoryintegration (MSI) resulting from the presentation ofspatially incongruent and congruent audio-visualevents, respectively.

Methods

Subjects

Thirty-two right-handed participants (16 males;mean age of 24 years 63; range 20–30 years) wererecruited and participated in this experiment. None ofthe subjects reported a history of neurological orpsychiatric disorders. They all had normal hearing andnormal or corrected-to-normal vision. The study wasapproved by the Comite d’Ethique de la Recherche dela Faculte des Arts et des Sciences (CERFAS) of theUniversity of Montreal. All participants providedwritten informed consent and received financial com-pensation for their participation in the study.

Apparatus and stimuli

The entire experiment took place in a darkened,double-walled audiometric room (3.49 m ·1.94 m ·1.89 m), which was insulated by 7.5 cm width wedges on

Journal of Vision (2013) 13(12):20, 1–14 Charbonneau et al. 2

five sides (the floor was made of carpeting). Stimuli werepresented on a semi-circular perimeter of 1808 with adiameter of 150 cm. A height-adjustable chair and akeypad with two response buttons were installed at thecenter of the semicircle. The distance between theparticipant’s head and the perimeter was 75 cm. Thirteenwhite light-emitting diodes (LEDs) and 13 smallloudspeakers (Beyer Dynamics, DT-48) were positionedon this unit. A LED was placed directly above eachspeaker at eye level of the participant. Each pair,composed of a LED and a speaker, was separated by 158of visual angle. These pairs were located at 08, 6158,6308, 6458, 6608, 6758, and 6908 relative to thecentral fixation point, with positive values representingareas to the right and negative values representing areasto the left of the fixation point (Figure 1A). Anadditional red LED was used as a fixation spot placed at08, just above the central target white LED. The wholeperimeter was covered with a semi-transparent blackfabric so that the participants could not see the speakersand the LEDs when they were turned off.

Visual stimuli consisted in a 15-ms illumination of awhite LED (48 cd/m2). Auditory stimuli were 15-msbursts of white noise (5-ms rise/fall time, 5-ms plateau).The level of intensity was adjusted to 65 dB SPL at theposition of the participants’ head. Bimodal (audio-visual) stimuli consisted in the simultaneous occurrenceof a visual and an auditory stimulus. Stimuli deliverywas controlled with Presentation software (Neurobe-havioral Systems, Inc., Albany, CA).

Procedure

A pair of two stimulations (always consisting of aprobe and of a target stimulus) was presented on each

trial (see Figure 1B for an example). The firststimulation (probe) was composed of an audio-visualpair, in which the auditory and visual stimuli werealways delivered simultaneously at the same location.The probe could originate from seven differentlocations: 08, 6158, 6458, or 6758 of eccentricity frommidline. The second stimulation (target) could be (a) avisual stimulus alone, (b) an auditory stimulus alone, or(c) both visual and auditory stimuli presented simul-taneously in a congruent or incongruent fashion. Forthe congruent matching, the visual and auditory stimuliwere presented at the same location. In contrast, for theincongruent matching, the visual and auditory stimulicame from separate eccentricities. The target stimuluswas presented between 100 and 150 ms after the probeeither to its right (þ158) or to its left (�158). For theincongruent conditions, one component of the targetstimulus (e.g., visual stimulus) was presented to theright (þ158) of the probe while the other (e.g., auditorystimulus) appeared to its left (�158). Each pair ofstimuli was followed by a 1000-ms interval for responseproduction. The central red LED was lit for 500 mspreceding the occurrence of a stimulus to ensure centralfixation.

Participants were instructed to judge as accuratelyand as fast as possible whether the target stimulus,which could be an auditory, a visual, or an audio-visualstimulation, was located to the left or to the right of theprobe by pressing the appropriate buttons of thekeypad with the index (leftward response) or the middlefingers (rightward response) of their right hand. Theywere asked to localize the target stimulus based on theirinitial reaction to the stimulation even if they perceiveda conflict between the senses.

Figure 1. (A) Schematic representation of the apparatus. Thirteen LEDs and 13 small loudspeakers were positioned on a 1808 semi-

circular perimeter. (B) Schematic example of an experimental trial. Pairs consisting of a LED placed above a speaker were located at

08, 6158, 6308, 6458, 6608, 6758, and 6908 relative to the fixation point of the subject. The probe could originate from seven

different locations, either 08, 6158, 6458, or 6758 from midline, while the comparison stimulus was presented either to the right

(þ158) or/and to the left (�158) of this reference.


A total of 1,680 randomly interleaved stimuli [7(position of the probe: 08, 6158, 6458, 6758) · 2(target’s location: right, left) · 4 (modality: visual,auditory, bimodal congruent, bimodal incongruent) ·30 repetitions] were presented to each subject. Thesestimuli were displayed in 15 blocks of 112 stimulilasting approximately 4 min each. Breaks wereencouraged between blocks to maintain a high con-centration level and prevent mental fatigue.

Data analysis

In order to investigate the participants’ ability tolocalize auditory, visual, and congruent audio-visualstimuli across the external frontal space, we com-puted the sensitivity index (d 0) to obtain a measure ofthe discriminability between leftward versus right-ward targets relative to the probe and the positionbias (b). The d 0 were calculated as [(zproportion correct�zproportion incorrect)/=2] and the b as [�(zproportion correct

þ zproportion incorrect)/2] (Macmillan & Creelman,1991). Correct mean reaction times (RTs; rangingfrom 150 ms to 1000 ms) were also analyzed. Inexperiments that place equal emphasis on accuracyand processing speed, it is possible that subjectsadopt different response strategies by varying RTinversely with accuracy (and thus show speed/accuracy trade-off). Therefore, overall performancemay be best reflected by a single variable thatsimultaneously takes into account speed and accura-cy. We have therefore also calculated the InverseEfficiency (IE) scores which constitute a standardapproach to combine mean RTs and accuracymeasures of performance (Townsend & Ashby, 1978)and can be considered as ‘‘corrected reaction times’’that discount possible criterion shifts or speed/accuracy trade-offs (Collignon et al., 2008; Roder,Kusmierek, Spence, & Schicke, 2007; Spence, Shore,Gazzaniga, Soto-Faraco, & Kingstone, 2001). The IEscores were obtained by dividing response times bythe rate of correct responses separately for eachcondition (thus, a higher value indicates worseperformance).

MSI was investigated by calculating the redundancygain (RG) based on d 0, RTs and IE scores. RG wasdefined as the difference (in percent) between the meand 0, mean RTs, or IE scores obtained in the multisen-sory condition and the mean d 0, RTs, or IE scoresobtained in the best unisensory condition. RG wasmeasured separately for each participant and level ofeccentricity. Different explanations have been putforward to account for the observation of the RG. Themost commons are the race and the coactivationmodels. The race model proposes that each individualstimulus elicits an independent detection process. For

a given trial, the fastest stimulus determines theobservable RT. On average, the time to detect thefastest of several redundant signals is faster than thedetection time for a single signal. Therefore, thespeeding up of reaction time is attributable tostatistical facilitation (Raab, 1962). When the racemodel’s prediction is violated, the speedup of RTscannot be attributed to a statistical effect alone butsome kind of coactivation must have occurred. Toaccount for violations of the race model’s prediction,the coactivation model (Miller, 1982) proposes thatthe neural activations of both stimuli combine toinduce faster responses. Testing the race modelinequality is widely used as an indirect behavioralmeasure of neurophysiological integrative processesunderlying RT facilitation (see for example Girard,Pelland, Lepore, & Collignon, 2013; but also see Otto& Mamassian, 2012). To further investigate MSI, therace model inequality was evaluated (Miller, 1982)using the RMITest software, which implements thealgorithm described at length in Ulrich, Miller &Schroter (2007). This procedure involves several steps.First, cumulative distribution functions (CDFs) of theRT distributions were estimated for every participant,eccentricity, and condition (visual, auditory, andaudio-visual conditions). Second, the bounding sum ofthe two CDFs obtained from the two unimodalconditions (visual and auditory) were computed foreach participant. This measure provided an estimateof the boundary at which the race model is violated,given by Boole’s inequality. Third, percentile pointswere determined for every distribution of RT,including the estimated bound for each participant. Inthe present study, the race model inequality wasevaluated at the 2.5, 7.5, 12.5, . . . 97.5 percentilepoints of the RT distributions. Fourth, for eachpercentile, mean RTs from redundant conditions weresubtracted to the mean RTs from the bound. If thosescores were above 0, the results exceeded the racemodel prediction and therefore supported the exis-tence of a facilitation process (Miller, 1982; but seealso Otto & Mamassian, 2012). In order to quantifysensory dominance (the ’’ventriloquist effect’’) inincongruent conditions, the percentage of responsestoward the sound (i.e., when a sound is coming fromthe right side and a flash is coming from the left side ofthe probe, and the participant reports the stimulus ascoming from the right side) was subtracted from thepercentage of responses toward the flash (i.e., when asound is coming from the right side and a flash iscoming from the left side of the probe, and theparticipant reports the stimulus as coming from theleft side) for each eccentricity separately. A positivescore would therefore indicate a visual capture overaudition, while a negative score would suggest thereverse.


The relative performances obtained in visual andauditory unisensory conditions were directly com-pared by subtracting visual from auditory d 0, RTs,and IE scores and respectively dividing the scoreobtained by the auditory d 0, RTs, and IE scores, (d 0,RTs, or IE auditory � d 0, RTs, or IE visual)/d 0, RTs,or IE auditory. Positive scores therefore attest higherunisensory spatial performance in vision expressed inpercent of the unisensory auditory results. Thesescores could then be correlated with the visual captureobserved in the bimodal incongruent condition inorder to investigate whether the relative superiority ofvision over audition in unisensory conditions relatesto the ventriloquist effect.

Since there was no difference in performances for theeccentricities to the left (�158, �458, �758) and to theright (þ158, þ458, þ758) of the central fixation point,these scores were combined (see Supplementary Mate-rial for the details of the statistical analyses), at theexception of the b analysis which differ depending onwhich side the stimuli were presented. Statisticalanalyses were therefore conducted on four levels ofeccentricity, which enhance statistical power by reduc-ing the number of multiple comparisons: j08j (central);j158j (mean: �158, þ158), j458j (mean: �458, 458), j758j(mean: �758, þ758). Each result’s figure (Figure 2through Figure 6) represents the performance obtainedin the full frontal space as well as the figures for thecombined eccentricities.

Results

Localization of auditory, visual and audio-visual

congruent stimuli: General performance

Differences in general performance were analyzed bysubmitting d0, b, RTs, and IE scores to repeated-measures ANOVAs (4 [eccentricities: j08j, j158j, j458j,j758j] · 3 [modalities: auditory, visual, bimodal

Figure 2. Left panel: d0 scores (A), RTs (B), and IE scores (see Methods) (C) for the discrimination of auditory, visual, and audio-visual

stimuli as a function of the eccentricity. Right panel: relative performance for visual over auditory targets as a function of the

eccentricity based on d0 scores (D), RTs (E), and IE scores (F). Error bars denote standard errors of the mean corrected for between-

subjects variability (Cousineau, 2005).

Figure 3. b scores for the discrimination of auditory, visual, and

audio-visual stimuli as a function of the eccentricity. Positive

values reflected a leftward response bias, while negative values

represented a rightward response bias. Error bars denote

standard errors of the mean corrected for between-subjects

variability (Cousineau, 2005).


http://www.journalofvision.org/content/13/12/20/suppl/DC1


congruent]). Based on significant F values, Bonferronipost-hoc analyses were performed when appropriate.d0 (Figure 2A): We observed a main effect for the‘‘modality’’ factor, F(2, 248)¼ 153.75, p � 0.0001, g2¼0.55, reflecting inferior performance for auditorystimuli compared to both visual and bimodal stimuliand inferior performance for visual than that ofbimodal stimuli. We also obtained a main effect forthe ‘‘eccentricity’’ factor, F(3, 124) ¼ 38.69, p �0.0001, g2 ¼ 0.48. Overall, this effect revealed thatperformance steadily decreased with eccentricity, withsuperior performance when comparing eccentricitiesj08j and j458j, j158j and j458j or j458j and j758j. Finally,an ‘‘eccentricity’’ by ‘‘modality’’ interaction was alsofound, F(6, 248)¼ 3.02, p � 0.05, g2¼ 0.07, driven bythe fact that auditory performance only (not visual orbimodal) decreased between eccentricities j08j andj158j.b (Figure 3): We found a main effect for the‘‘eccentricity’’ factor, F(6, 217)¼ 7.18, p � 0.001, g2 ¼0.17), with a leftward bias for stimuli presented at theeccentricities�758,�458,�158, 08, and 158 compared to758 and for the stimuli presented at the eccentricity�758 compared to 458. There was no main effect for the‘‘modality’’ factor, F(2, 434) ¼ 0.99, p ¼ 0.37, g2 ¼0.005), and no interaction was identified between the

factors ‘‘modality’’ and ‘‘eccentricity,’’ F(12, 434) ¼1.71, p ¼ 0.06, g2 ¼ 0.05.RTs (Figure 2B): First, a main effect was found for the‘‘modality’’ factor, F(2, 248)¼ 114.94, p � 0.0001, g2¼0.48, with higher RTs for auditory stimuli compared toboth visual and bimodal stimuli and higher RTs forvisual than bimodal stimuli. We also observed a maineffect for the ‘‘eccentricity’’ factor, F(3, 124)¼ 2.76, p �0.05, g2 ¼ 0.06, revealing a general slowdown of RTswith eccentricity. Finally, an ‘‘eccentricity’’ by ‘‘mo-dality’’ interaction was found, F(6, 248) ¼ 10.11, p �0.0001, g2 ¼ 0.20, revealing that only RTs to auditorytargets were not influenced by the eccentricity of thestimuli (p ¼ 1.0 for all comparisons).IE scores (Figure 2C): We observed a main effect of thefactor ‘‘modality,’’ F(2, 248)¼ 204.58, p � 0.0001, g2¼0.62, with worst performance for auditory stimulicompared to both visual and bimodal stimuli and aworst performance for visual than bimodal stimuli. Wealso found a main effect for the ‘‘eccentricity’’ factor,F(3, 124)¼ 20.52, p � 0.0001, g2¼ 0.33, again showinga decrease of performance with eccentricity. Nointeraction was identified between the factors ‘‘modal-ity’’ and ‘‘eccentricity,’’ F(6, 248)¼ 2.65, p¼ 0.13, g2¼0.04.

Figure 4. RG (in percent) based on d0 scores (A), RTs (B), and IE scores (C) as a function of the eccentricity. (D) RG in function of the

performance for the best unisensory component based on d0, RTs, and IE scores. Error bars denote standard errors of the mean

corrected for between-subjects variability (Cousineau, 2005).


The relative unimodal performance of vision overaudition (see data analysis section for details) wascompared across the eccentricities using one-wayANOVAs. When based on d0 scores, the relativeunimodal performance of vision over audition changedacross the eccentricities, F(3, 127)¼ 6.01, p � 0.005, g2

¼ 0.13, with lower visual dominance for eccentricity j08jthan j458j (Figure 2D). If measured using the RTs, therelative performance of vision over audition signifi-cantly decreased in periphery, F(3, 127) ¼ 14.59, p �0.001, g2 ¼ 0.26), with higher visual dominance whencomparing eccentricities j08j than j458j, j158j than j458jand j458j than j758j (Figure 2E). No significant effect ofeccentricity, F(3, 127) ¼ 2.12, p ¼ 0.10, g2 ¼ 0.05) wasfound when the relative performance of vision overaudition was calculated based on IE scores, even if thescores tended to be lower at the largest eccentricity(j758j; Figure 2F).

Localization of spatially congruent audio-visualstimuli: Redundancy gain

There was no significant difference in RG through-out the different eccentricities when derived from thed0, F(3, 124)¼ 2.32, p¼ 0.08, g2¼ 0.05 (Figure 4A), orthe IE scores, F(3, 124)¼ 0.23, p ¼ 0.88, g2 ¼ 0.006

(Figure 4C). If measured using the RTs, RG showedfluctuations according to the degree of eccentricity,F(3, 124)¼ 4.55, p � 0.005, g2¼ 0.10, with higher RGfor eccentricity j08j and j158j compared to j758j (Figure4B). A positive difference (meaning a violation of therace model prediction) was observed between theredundant condition and the probabilistic bound forthe first three percentiles of the RT distribution ateccentricity j08j and for the first percentile at eccen-tricity j158j. No violation of the race model estimationwas found for the eccentricities j458j and j758j (Figure 5,see Supplementary Material for the figure representingthe results obtained in the full frontal space).

In order to further investigate the associationbetween the RG and the localization performance forunisensory information, we correlated (using Pearsonproduct-moment correlation coefficient) the RG withthe mean d0, RTs, and IE scores for the best unisensorymodality across eccentricities (Gingras, Rowland, &Stein, 2009). According to the principle of inverseeffectiveness (IE), stating that the multisensory gainproduced by the integration of separate sensoryestimates is enhanced when the reliability of thesestimuli is low (Stein & Meredith, 1993), these variablesare generally inversely correlated; that is, loweraccuracy in localizing single stimuli is associated withmaximal RG when adding another stimulus (Stanford,Quessy, & Stein, 2005; Stein & Meredith, 1993). Weobtained no significant correlation between the RG andthe best unisensory d0 (r ¼�0.33, p ¼ 0.47), RTs (r ¼�0.72, p ¼ 0.07), or IE scores (r¼�0.27, p¼ 0.55)(Figure 4D), meaning that RG was relatively constantregardless of the reliability of the best unisensorycomponent. The marginal tendency observed with RTswas in opposite direction of the predictions made fromthe IE principle: Shorter RTs in the best modality (atmore central location) were associated with a higherRG.

Localization of spatially incongruent audio-visual stimuli: The ventriloquist effect

We observed a main effect of the factor ‘‘eccentric-ity,’’ F(3, 124)¼ 17.67, p � 0.0001, g2¼ 0.30, revealinga decrease in the ventriloquist effect with eccentricity.Post-hoc comparisons demonstrated a higher visualcapture for eccentricity j08j compared to j458j and j758j,for eccentricity j158j compared to j458j and j758j, andfor eccentricity j458j compared to j758j (Figure 6A).

In order to investigate the association between theventriloquist effect and the relative reliability ofunisensory visual over auditory stimuli, we correlatedthe relative reliability of visual over auditory unimodalinformation in d0 scores, RTs, and IE scores (reflectingboth the discriminability and processing speed of the

Figure 5. Race model inequality. Test for violation of the race

model inequality (Miller, 1982; Ulrich, Miller, & Schroter, 2007).

The graph represents the difference in milliseconds (on the y-

axis) between the model prediction computed from the RTs of

each unisensory counterpart (the model bound) and the RTs

obtained in the redundant conditions. Positive values on the

graph refer to RTs that were faster than the race model

prediction. Negative values on the graph refer to RTs that were

slower than the race model prediction. The difference between

the bound and the RTs of the redundant condition are

computed for each percentile of the RT distribution (on the x-

axis).



stimuli) with the visual capture effect separately foreach eccentricity. For all eccentricities, we obtained apositive correlation between the ventriloquist effect andthe scores of visual reliability based on IE scores [j08j (r¼ 0.63, p � 0.001), j158j (r¼ 0.70, p � 0.001), j458j (r¼0.69, p � 0.001), and j758j (r¼ 0.77, p � 0.001); Figure6B], d0, [j08j (r¼ 0.42, p � 0.05), j158j (r¼ 0.21, p¼0.24), j458j (r ¼ 0.32, p ¼ 0.07), and j758j (r ¼ 0.18, p ¼0.33); Supplementary Figure S2A] and RTs [j08j (r¼0.55, p � 0.001), j158j (r¼ 0.72, p � 0.0001), j458j (r¼0.62, p � 0.0001) and j758j (r ¼ 0.56, p � 0.001;Supplementary Figure S2B], meaning that individualshaving higher visual localization abilities relative toauditory ones also show an enhanced visual capture atevery level of eccentricity.

Control experiment

In the main experiment, the probe always consistedin an audio-visual pair of stimuli while the target couldeither be a unimodal (auditory or visual) or a bimodal(audio-visual) stimulation. Therefore, the unisensorytrials were not purely unimodal as they contained abimodal probe, and the amount of noise associatedwith the two types of multisensory pairs was not equal

to that associated with the unisensory auditory andvisual pairs. In order to test if the differences weobserved between unisensory and multisensory stimu-lations was associated to the difference in the level ofnoise between those conditions, we conducted a controlexperiment in which the first stimulation of each pairwas always presented in the same modality as thesecond stimulus of the pair (either auditory, visual, oraudio-visual), instead of always being an audio-visualstimulation (Supplementary Figure S3A and S3B).

Results (and related statistics) are presented in theSupplementary Material (Supplementary Figure S4through S8) and are consistent with what was found inthe main experiment. Therefore, using a constant levelof noise for each pair of stimulation, we observed thatdespite a strong decrease in auditory and visualunisensory localization abilities in periphery (seeSupplementary Figure S4A through S4C), the RGresulting from the congruent presentation of audio-visual targets was not affected by stimuli eccentricity(see Supplementary Figure S6). We also confirmed thatthe ventriloquist effect steadily decreased with eccen-tricity (see Supplementary Figure S8A), paralleling alowering of the relative reliability of unimodal visualover unimodal auditory stimuli in periphery (seeSupplementary Figure S4D through S4F), and that it

Figure 6. (A) Ventriloquist effect as a function of the eccentricity. This score is obtained by subtracting the percentage of responses

guided by hearing to the percentage of responses guided by vision for each eccentricity. Error bars denote standard errors of the

mean corrected for between-subjects variability (Cousineau, 2005). (B) Ventriloquist effect as a function of the relative reliability

between visual and auditory targets. This figure shows a significant positive correlation between the ventriloquist effect and the

visual reliability scores for each eccentricity.













positively correlated with a weighted combination ofthe spatial resolution obtained in unisensory conditions(see Supplementary Figure S8B).

Discussion

By exploring how the spatial location of auditoryand visual targets affects both the multisensory gainand the ventriloquist effect, the current study shed newlights on how the natural eccentricity-related changes inreliability of audio-visual stimuli impact their integra-tion across the entire external frontal space.

We first examined how the position of auditory-alone and visual-alone stimuli influences our spatialdiscrimination abilities. Regarding the localization ofthe visual stimuli, we observed higher discriminability(Figure 2A) and faster RTs (Figure 2B) when theinformation was presented in the center of the frontalfield rather than in the periphery. This reduction in thereliability of visual information with eccentricity isconsistent with the fact that, through its physiologicalproperties (higher density of cones than rods in thefovea and the reverse in the periphery of the retina), thevisual system has a better spatial resolution for centralvision (Baizer et al., 1991; Dacey & Petersen, 1992;Stephen et al., 2002). When looking at the ability tolocalize the auditory stimuli, we also observed a severereduction in the discriminability of information pre-sented in the periphery (Figure 2A). This finding is inaccordance with studies showing that higher auditoryspatial resolution is achieved for information presenteddirectly in front of the head, due to the spatialdependence of the interaural difference cues (Blauert,1997; Middlebrooks, 1991). Interestingly, the RTs forthe auditory targets remained similar across all thefrontal space (Figure 2B), which therefore contrastswith the results obtained in vision where eccentricitysimilarly affected both the discriminability and the RTsof the stimuli. These results suggest that, at least interms of RTs, the detrimental effect of peripheralstimuli appearance is lower in audition than in vision(see also Figure 2E–F). This outcome might be relatedto studies reporting that saccadic reaction time towardauditory (but not visual) stimuli accelerated as afunction of target eccentricity, whereas accuracydecreased (as for visual targets) with increasingeccentricity (Gabriel, Munoz, & Boehnke, 2010; Yao &Peck, 1997), suggesting that there is a speed-accuracytrade-off in the localization of peripheral auditorytargets. The mechanisms underlying this effect stillremain elusive, but have been suggested to be related tospatial receptive field attributes of neurons in thesuperior colliculus (Zambarbieri, 2002; Zambarbieri,Beltrami, & Versino, 1995; Zambarbieri, Schmid,

Magenes, & Prablanc, 1982), as it changes with eyeposition or when the fixation point disappear before theonset of the target.

The first main objective of the current study was toexplore the impact of the eccentricity-related reliabilityof auditory and visual information on the RG. Basedon the inverse efficiency (IE) principle in MSI statingthat the multisensory gain produced by the integrationof separate sensory estimates coming from the sameevent is enhanced when the reliability of these stimuli islow (Stein & Meredith, 1993), one may have expected apositive relation between the RG and the eccentricitiesof the targets since the reliability of visual and auditoryinformation dramatically decrease when presented inperiphery (Figure 2C). Moreover, the observation ofdirect projections between caudal region of theauditory cortex and visual areas representing peripheralvisual space in monkeys (Falchier, Clavagnier, Barone,& Kennedy, 2002; Rockland & Ojima, 2003) andhumans (Beer, Plank, & Greenlee, 2011) provided aneurophysiological support to hypothesize enhancedaudio-visual integration in periphery. Surprisingly, wedid not observe a significant influence of the eccen-tricity of our targets on the RG derived from d’ or IEscores (Figure 4A and Figure 4C). Rather, we evenobserved a stronger RG for central locations with RTmeasurements (Figure 4B and Supplementary FigureS4B). In support of these results, we observed that forthe fastest latencies (percentiles) of the RT distribu-tions, the RT probability in the bimodal conditionexceeded the probabilistic sum of the RT observed inthe auditory or visual unisensory conditions (Figure 5and Supplementary Figure S7; Miller, 1982; but seealso Otto & Mamassian, 2012). This violation of therace model prediction was however not observed formore peripheral locations (j458j and j758j), whichindicates a reduced ability to integrate separate sensoryinformation when presented in the periphery of thefrontal space.

Gingras and collaborators (2009) examined theperformance of cats in localizing auditory, visual, andspatio-temporally congruent audio-visual stimuli atdifferent eccentricities of the frontal space (6458). Theyfound that the enhanced performance obtained withaudio-visual stimuli was inversely proportional to thebest single stimulus accuracy at each location, a resultwhich is consistent with the IE principle. In contrastwith these results and reinforcing the hypothesis thatthe IE principle is not observed in our experiment, wefound no correlation between the performance for thebest unisensory modality and the RG (Figure 4D).

As such, we show that IE does not strictly apply tothe multisensory enhancements seen during audio-visual localization in humans. These results thereforecontradict simple models predicting that the rules of IEgoverning MSI derived from unicellular studies in the






brainstem (see for example Rowland, Stanford, &Stein, 2007; Stein, Stanford, Ramachandran, Perrault,& Rowland, 2009) directly relates to multisensoryphenomena at the level of perception and behavior(Holmes & Spence, 2005). Actually, even at aneurophysiological level, it was already suggested thatsetting stimulus intensity such that both significantfacilitation and suppression relative to the unisensoryresponse could reasonably arise, and measuring abso-lute changes of multisensory gain as a function ofunisensory responsiveness, are parameters that havebeen shown to reduce the likelihood that the rule of IEwill be followed (Holmes, 2009).

Although IE has been shown in numerous behav-ioral studies (Bolognini, Frassinetti, Serino, & Ladavas,2005; Gondan, Niederhaus, Rosler, & Roder, 2005;Stein, Huneycutt, & Meredith, 1988), several reportsalso failed to show support for this principle (Diederich& Colonius, 2008; Kim & James, 2010; Ross et al.,2007; Ross, Saint-Amour, Leavitt, Javitt, & Foxe,2007; Stevenson, Krueger Fister, Barnette, Nidiffer, &Wallace, 2012). For example, Ross and colleagues(2007) investigated whether seeing a speaker’s articu-latory movements would influence a listener’s ability torecognize spoken words under noisy environmentalconditions, and found that the redundancy RG ishigher at intermediate auditory signal-to-noise ratios(and not at lowest signal-to-noise ratios as predicted bythe IE principle). The authors interpreted their resultsby suggesting that there is a maximal integrationwindow at intermediate signal-to-noise ratios, wherethe perceptual system can build on either the visual orthe auditory modality. These findings appear to beconsistent with those from our experiment, as RGtended to be greatest at intermediate eccentricities(j158j, j458j; see Figure 4C). A reason why these resultswere not observed from previous studies investigatingaudio-visual perception across the frontal space (Bo-lognini et al., 2005) could be that none of thempresented the information in the far periphery (.558).Therefore, these studies cut off approximately at thesame eccentricity at which we observed a maximal gain.

Another related question is whether the samemechanisms are implicated when comparing multisen-sory enhancement in the same brain receptive field (RF)or across different RFs. The vast majority of experi-ments explored the multisensory enhancement at aunique position in space while changing the level ofstimulus intensity, therefore activating a unique RFwith different level of intensity. However, in the currentstudy, we varied the strength of the sensory signal bypresenting it through different RFs having differentspatial resolution. It might therefore be interesting toinvestigate if different level of stimulus intensity (i.e.,high and low intensity) at each of the eccentricities may

induce performances compatible with the principle ofIE.

The second main interest of the current study was toexplore the impact of eccentricity-related reliability ofauditory and visual information on the ventriloquisteffect. The ventriloquist effect is a perceptual conditionwherein a visual stimulus biases (captures) the locali-zation of a spatially incongruent sound. It was firstthought to reflect a winner-take-all advantage thatfavors visual over non-visual spatial informationbecause vision typically provides the most reliableinformation for space perception (Welch & Warren,1980). However, more recent studies demonstrated thatinformation from independent sensory channels isintegrated with a near-optimal or Bayesian strategy,such that the perceived location of the sound ratherdepends on the weighted sum of the sensory cues basedon their respective reliability (Alais & Burr, 2004; Ernst& Bulthoff, 2004). In the spatial domain, Hairston et al.(2003b) examined the ability of a visual signal toinfluence the localization of an auditory target ac-cording to their eccentricity (6408 in fontal space).They found that the visual bias was strongly dependenton the location of the visual stimulus, with greatest biasin the center of visual space where the reliability of thevisual targets is the greatest. Moreover, Teramoto andcollaborators (2012) showed that the perceived direc-tion of visual motion could be modulated by thepresence of a moving sound and that the more thevisual stimuli originated from the periphery, thestronger the effect. The current study supports andextends these results by demonstrating that theventriloquist effect steadily decreases with eccentricityover the whole frontal space (Figure 6A). Moreover, weshow that this reduction of visual capture is associatedwith more similar unimodal localization performancebetween vision and audition in periphery (Figure 2D-F;Supplementary Figure S4D through S4F). It is howeverimportant to note here that regarding the relationbetween the relative unisensory performances for visualover auditory targets, we observed a speed/accuracytradeoff effect, as the performance based on d’increased with eccentricity and the opposite effect wasfound for RTs (Figure 2D through E). The combina-tion of both scores in the IE scores still indicated areduction in visual dominance as a function of theeccentricity (Figure 2F).

We also demonstrate for the first time that theventriloquist effect strongly correlates with the relativeunisensory reliability between visual over auditoryinformation for each eccentricity across the frontalspace (Figure 6B): The better the unisensory localiza-tion abilities in vision compared to audition, thestronger the ventriloquist effect. These results furtherbolster the idea that, based on the relative spatialreliability of auditory and visual unisensory informa-




tion, the ventriloquist effect results from a weightedcombination of the information gathered from the twomodalities (Alais & Burr, 2004). Audio-visual integra-tion in space is therefore not an absolute process butrelative to the location of the multisensory event fromthe observer’s body position, which location differentlyimpact on the reliability of the unisensory components.

It is hypothesized that the topography of theauditory space is constantly calibrated by retino-visuomotor feedback in order to maintain stable spatialalignment with the visual space (Lewald, 2013; 2002a,b). This was notably evidenced by studies demonstrat-ing that following an exposure period to spatiallydisparate audio-visual stimuli, the perceived location ofa sound is being translocated in the direction of thevisual stimuli to which observers had previously beenexposed, known as the ventriloquist after-effect (Can-on, 1970; Frissen, Vroomen, de Gelder, & Bertelson,2003; Lewald, 2002a, b; Radeau & Bertelson, 1974;Recanzone, 1998). Based on the results of the currentstudy, we hypothesize that the magnitude of this after-effect may vary according to the eccentricity where theaudio-visual stimuli are presented during the exposureperiod, with stronger ventriloquist after-effect in thecentral visual field where the reliability of the visualtarget is the highest.

Conclusions

The current study provides new insights into how theeccentricity of separate sensory estimates influences themerging of these inputs into a unified percept. First, wefound that the multisensory gain was not affected bystimuli reliability, contrasting with the prediction madefrom the principle of IE. This finding suggests thatmultisensory principles derived from unicellular exper-iment carried out in the superior colliculus (Stein andMeredith, 1993) are not straightforward in theirexplanation of multisensory phenomena at the level ofperception and behavior (Spence, 2013). Second, wedemonstrated that the visual capture of soundsobserved with spatially incongruent audio-visual tar-gets steadily decreased with eccentricity. Finally, weshowed that at every level of eccentricity, the ventril-oquist effect directly relates to a weighted combinationof the spatial resolution obtained in unisensoryconditions. These results evidence that the externalspatial coordinates of multisensory events relative to anobserver’s body (e.g., eyes’ or head’s position) influencehow this information is merged, and therefore deter-mine the perceptual outcome. These results might haveimplications for the design of optimal ergonomicenvironment where the attention of an observer has to

be captured by auditory, visual or audio-visual cuesdelivered at various spatial locations.

Keywords: multisensory integration, ventriloquist ef-fect, spatial localization, reliability, eccentricity

Acknowledgments

This research was supported in part by the CanadaResearch Chair Program (FL), the Canadian Institutesof Health Research (FL, GC), the Natural Sciences andEngineering Research Council of Canada (FL), and theResearch Center of the Ste-Justine University Hospital(OC).

Commercial relationships: none.Corresponding author: Olivier Collignon.Email: [email protected]: Center for Mind/Brain Sciences, University ofTrento, Italy.

References

Alais, D., & Burr, D. (2004). The ventriloquist effectresults from near-optimal bimodal integration.Current Biology, 14, 257–262.

Baizer, J. S., Ungerleider, L. G., & Desimone, R.(1991). Organization of visual inputs to the inferiortemporal and posterior parietal cortex in macaques.Journal of Neuroscience, 11(1), 168–190.

Barfield, W., Cohen, M., & Rosenberg, C. (1997).Visual and auditory localization as a function ofazimuth and elevation. The International Journal ofAviation Psychology, 7(2), 123–138.

Battaglia, P. W., Jacobs, R. A., & Aslin, R. N. (2003).Bayesian integration of visual and auditory signalsfor spatial localization. Journal of the OpticalSociety of America A: Optics, Image Science, andVision, 20, 1391–1397.

Beer, A. L., Plank, T., & Greenlee, M. W. (2011).Diffusion tensor imaging shows white matter tractsbetween human auditory and visual cortex. Exper-imental Brain Research, 213(2-3), 299–308.

Binda, P., Bruno, A., Burr, D. C., & Morrone, M. C.(2007). Fusion of visual and auditory stimuli duringsaccades: A Bayesian explanation for perisaccadicdistortions. The Journal of Neurosciences, 27(32),8525–8532.

Blauert, J. (1997). Spatial hearing: The psychophysics ofhuman sound localization. Cambridge, MA: MITPress.


Bolognini, N., Leo, F., Passamonti, C., Stein, B. E., &Ladavas, E. (2007). Multisensory-mediated audi-tory localization. Perception, 36, 1477–1485.

Bolognini, N., Frassinetti, F., Serino, A., & Ladavas, E.(2005). ‘‘Acoustical vision’’ of below thresholdstimuli: Interaction among spatially convergingaudiovisual inputs. Experimental Brain Research,160, 273–282.

Canon, L. K. (1970). Intermodality inconsistency ofinput and directed attention as determinants of thenature of adaptation. Journal of ExperimentalPsychology, 84(1), 141–147.

Collignon, O. (2007). Reinforcing the weight attributedto auditory space across saccades. The Journal ofNeuroscience, 27(46), 12438–12439.

Collignon, O., Girard, S., Gosselin, F., Roy, S., Saint-Amour, D., Lassonde, M., Lepore, F. (2008).Audio-visual integration of emotion expression.Brain Research, 1242, 126–135.

Cousineau, D. (2005). Confidence intervals in within-subject designs: A simpler solution to Loftus andMasson’s method. Tutorials in Quantitative Meth-ods for Psychology, 1(1), 42–45.

Dacey, D. M., & Petersen, M. R. (1992). Dendritic fieldsize and morphology of midget and parasolganglion cells of the human retina. Proceedings ofthe National Academy of Sciences, USA, 89(20),9666–9670.

Diederich, A., & Colonius, H. (2008). When a high-intensity ‘‘distractor’’ is better than a low-intensityone: Modeling the effect of an auditory or tactilenontarget stimulus on visual saccadic reaction time.Brain Research, 1242, 219–230.

Ernst, M. O., & Bulthoff, H. H. (2004). Merging thesenses into a robust percept. Trends in CognitiveSciences, 8(4), 162–169.

Falchier, A., Clavagnier, S., Barone, P., & Kennedy, H.(2002). Anatomical evidence of multimodal inte-gration in primate striate cortex. The Journal ofNeuroscience, 22(13), 5749–5759.

Frissen, I., Vroomen, J., de Gelder, B., & Bertelson, P.(2003). The aftereffects of ventriloquism: Are theysound-frequency specific? Acta Psychologica, 113,315–327.

Gabriel, D. N., Munoz, D. P., & Boehnke, S. E. (2010).The eccentricity effect of auditory saccadic reactiontimes is independent of target frequency. HearingResearch, 262(1-2), 19–25.

Gingras, G., Rowland, B. A., & Stein, B. E. (2009). Thediffering impact of multisensory and unisensoryintegration on behavior. The Journal of Neurosci-ence, 29(15), 4897–4902.

Girard, S., Pelland, M., Lepore, F., & Collignon, O.(2013). Impact of the spatial congruence ofredundant targets on within-modal and cross-modal integration. Experimental Brain Research,224(2), 275–285.

Gondan, M., Niederhaus, B., Rosler, F., & Roder, B.(2005). Multisensory processing in the redundant-target effect: A behavioral and event-relatedpotential study. Perception & Psychophysics, 67,713–726.

Hairston, W. D., Laurienti, P. J., Mishra, G., Burdette,J. H., & Wallace, M. T. (2003a). Multisensoryenhancement of localization under conditions ofinduced myopia. Experimental Brain Research, 152,404–408.

Hairston, W. D., Wallace, M. T., Vaughan, J. W.,Stein, B. E., Norris, J. L., & Schirillo, J. A. (2003b).Visual localization ability influences cross-modalbias. Journal of Cognitive Neuroscience, 15(1), 20–29.

Holmes, N. P., & Spence, C. (2005). Multisensoryintegration: Space, time and superadditivity. Cur-rent Biology, 15(18), 62–64.

Holmes, N. P. (2009). The principle of inverseeffectiveness in multisensory integration: Somestatistical considerations. Brain Topography, 21,168–176.

Howard, I. P., & Templeton, W. B. (1966). Humanspatial orientation. New York: Wiley.

Kim, S., & James, T. W. (2010). Enhanced effectivenessin visuo-haptic object-selective brain regions withincreasing stimulus salience. Human Brain Map-ping, 31, 678–693.

Lewald, J. (2002a). Opposing effects of head positionon sound localization in blind and sighted humansubjects. European Journal of Neuroscience, 15,1219–1224.

Lewald, J. (2002b). Rapid adaptation to auditory-visual spatial disparity. Learning and Memory, 9,268–278.

Lewald, J. (2013). Exceptional ability of blind humansto hear sound motion: Implications for theemergence of auditory space. Neuropsychologia, 51,181–186.

Macmillan, N. A., & Creelman, C. D. (1991). Detectiontheory: A user’s guide. New York: CambridgeUniversity Press.

Makous, J. C., & Middlebrooks, J. C. (1990). Two-dimensional sound localization by human listeners.Journal of the Acoustical Society of America, 87,2188–2200.

Middlebrooks, J. C. (1991). Sound localization by


human listener. Annual Review of Psychology, 42,135–159.

Miller, J. (1982). Divided attention evidence forcoactivation with redundant signals. CognitivePsychology, 14, 247–279.

Nguyen, V., Suied, C., Viaud-Delmon, I., & Warusfel,O. (2009). Spatial audition in a static virtualenvironment: The role of auditory-visual interac-tion. Journal of Virtual Reality and Broadcasting,6(5).

Otto, T., & Mamassian, P. (2012). Noise and correla-tions in parallel perceptual decision making.Current Biology, 22(15), 1391–1396.

Pick, H. L., Warren, D. H., & Hay, J. C. (1969).Sensory conflict in judgments of spatial direction.Perception & Psychophysics, 6, 203–205.

Raab, D. H. (1962). Statistical facilitation of simplereaction times. Transactions of the New YorkAcademy of Sciences, 24, 574–590.

Radeau, M., & Bertelson, P. (1974). The after-effects ofventriloquism. Quarterly Journal of ExperimentalPsychology, 26(1), 63–71.

Recanzone, G. H. (1998). Rapidly induced auditoryplasticity: The ventriloquism aftereffect. Proceed-ings of the National Academy of Science, USA, 95,869–875.

Rockland, K. S., & Ojima, H. (2003). Multisensoryconvergence in calcarine visual areas in macaquemonkey. International Journal of Psychophysiology,50(1-2), 19–26.

Roder, B., Kusmierek, A., Spence, C., & Schicke, T.(2007). Developmental vision determines the refer-ence frame for the multisensory control of action.Proceedings of the National Academy of Sciences ofthe U.S.A., 104(11), 4753–4758.

Ross, L. A., Saint-Amour, D., Leavitt, V. M., Javitt, D.C., & Foxe, J. J. (2007). Do you see what I amsaying? Exploring visual enhancement of speechcomprehension in noisy environments. CerebralCortex, 17, 1147–1153.

Ross, L. A., Saint-Amour, D., Leavitt, V. M.,Molholm, S., Javitt, D. C., & Foxe, J. J. (2007).Impaired multisensory processing in schizophrenia:Deficits in the visual enhancement of speechcomprehension under noisy environmental condi-tions. Schizophrenia Research, 97, 173–183.

Rowland, B., Stanford, T., & Stein, B. (2007). ABayesian model unifies multisensory spatial local-ization with the physiological properties of thesuperior colliculus. Experimental Brain Research,180, 153–161.

Spence, C. (2013). Just how important is spatial

coincidence to multisensory integration? Evaluatingthe spatial rule. Annals of the New York Academy ofSciences, 1296(1), 31–49.

Spence, C., Shore, D. I., Gazzaniga, M. S., Soto-Faraco, S., & Kingstone, A. (2001). Failure toremap visuotactile space across the midline in thesplit-brain. Canadian Journal of ExperimentalPsychology, 55(2), 133–140.

Stanford, T. R., Quessy, S., & Stein, B. E. (2005).Evaluating the operations underlying multisensoryintegration in the cat superior colliculus. Journal ofNeurosciences, 25, 6499–6508.

Stein, B. E., & Meredith, M. A. (1993). The merging ofthe senses. Cambridge, MA: MIT.

Stein, B. E., Huneycutt, W. S., & Meredith, M. A.(1988). Neurons and behavior: The same rules ofmultisensory integration apply. Brain Research,448(2), 355–358.

Stein, B. E., Stanford, T. R., Ramachandran, R.,Perrault, T. J., Jr. & Rowland, B. A. (2009).Challenges in quantifying multisensory integration:Alternative criteria, models, and inverse effective-ness. Experimental Brain Research, 198, 113–126.

Stephen, J. M., Aine, C. J., Christner, R. F., Ranken,D., Huang, M., & Best, E. (2002). Central versusperipheral visual field stimulation results in timingdifferences in dorsal stream sources as measuredwith MEG. Vision Research, 42(28), 3059–3074.

Stevenson, R. A., Krueger Fister, J., Barnette, Z. P.,Nidiffer, A. R., & Wallace, M. T. (2012). Interac-tions between the spatial and temporal stimulusfactors that influence multisensory integration inhuman performance. Experimental Brain Research,219, 121–137.

Sumby, W. H., & Pollack, I. (1954). Visual contribu-tion to speech intelligibility in noise. Journal of theAcoustical Society of America, 26(2), 212–215.

Teramoto, W., Hidaka, S., Sugita, Y., Sakamoto, S.,Gyoba, Y., Iwaya, Y., & Suzuki, Y. (2012). Soundscan alter the perceived direction of a moving visualobject. Journal of Vision, 12(3):11, 1–12, http://www.journalofvision.org/content/12/3/11, doi:10.1167/12.3.11. [PubMed] [Article]

Thurlow, W. R., & Runge, P. S. (1967). Effect ofinduced head movements on localization of direc-tion of sounds. Journal of the Acoustical Society ofAmerica, 42, 480–488.

Townsend, J. T., & Ashby, F. G. (1978). Methods ofmodeling capacity in simple processing systems. InN. J. Castellan & F. Restle (Eds.), Cognitive theory(Vol. 3, pp. 199–239). Hillsdale, NJ: Erlbaum.

Ulrich, R., Miller, J., & Schroter, H. (2007). Testing the


http://www.journalofvision.org/content/12/3/11

http://www.journalofvision.org/content/12/3/11

http://www.ncbi.nlm.nih.gov/pubmed/22410584

http://www.journalofvision.org/content/12/3/11.long

race model inequality: An algorithm and computerprograms. Behavior Research Methods, 39(2), 291–302.

Welch, R. B., & Warren, D. H. (1980). Immediateperceptual response to intersensory discrepancy.Psychological Bulletin, 88(3), 638–667.

Yao, L., & Peck, C. K. (1997). Saccadic eye movementsto visual and auditory targets. Experimental BrainResearch, 115, 25–34.

Zambarbieri, D. (2002). The latency of saccades toward

auditory targets in humans. Progress in BrainResearch, 140, 51–59.

Zambarbieri, D., Beltrami, G., & Versino, M. (1995).Saccade latency toward auditory targets dependson the relative position of the sound source withrespect to the eyes. Vision Research, 35, 3305–3312.

Zambarbieri, D., Schmid, R., Magenes, G., & Pra-blanc, C. (1982). Saccadic responses evoked bypresentation of visual and auditory targets. Exper-imental Brain Research, 47, 417–427.


the ventriloquist in periphery: impact of eccentricity …...the ventriloquist in periphery: impact...

Documents