the role of visuomotor regulation processes on perceived ... · graduate department of exercise...

The Role of Visuomotor Regulation Processes onPerceived Audiovisual Events

by

Gerome Aleandro Manson

A thesis submitted in conformity with the requirementsfor the degree of Master of Science

Graduate Department of Exercise SciencesUniversity of Toronto

Copyright © 2013 by Gerome Aleandro Manson

Abstract

The Role of Visuomotor Regulation Processes on Perceived Audiovisual Events

Gerome Aleandro Manson

Master of Science

Graduate Department of Exercise Sciences

University of Toronto

2013

Recent evidence suggests audiovisual perception changes as one engages in action. Specif-

ically, if an audiovisual illusion comprised of 2 flashes and 1 beep is presented during the

high velocity portion of upper- limb movements, the influence of the auditory stimuli is

subdued. The goal of this thesis was to examine if visuomotor regulation processes that

rely on information obtained when the limb is traveling at a high velocity could explain

this perceptual modulation. In the present study, to control for engagement in visuo-

motor regulation processes, vision of the environment was manipulated. In conditions

without vision of the environment, participants did not show the noted modulation of the

audiovisual illusion. Also, analysis of the movement trajectories and endpoint precision

revealed that movements without vision were less controlled than movements performed

with vision. These results suggest that engagement in visuomotor regulation processes

can influence perception of certain audiovisual events during goal-directed action.

ii

Dedication

To Grandma and Nivea

iii

Acknowledgements

I am truly grateful to have such outstanding people around and I am very thankful

for all their support throughout these years. I would like to formally acknowledge my

supervisor, Dr. Luc Tremblay, you continuously challenged me while allowing me to have

the freedom to pursue varied and numerous interests in all aspects of academic life. You

also provided unwavering support in times where I needed it the most. I would also

like to thank members of my committee, including Dr. Timothy Welsh, for providing

me with great advice, both academic and personal. Your expertise in statistics was also

greatly appreciated. To Dr. Jennifer Campos, thank you also for your advice and help

in developing this thesis project, your outlook, suggestions, and stimulating questions

were greatly appreciated. To Dr. Susanne Ferber, thank you for your feedback and

critiques, they were very useful in developing the document. Thank you to all of my lab

mates, both past and present who made conducting research in the AA and PMB labs

very enjoyable. Special thank you to the group here this summer: Taffy, Val, Kim, and

Nat. A very special thank you to Damian Manzone for all of your help during the data

collection process. To my fellow graduate students: John, Rachel, Matt, and Connor,

there is no other group of people I would have rather share this journey with. Thank

you all so much for all the advice, lively discussions, and friendship. I would also like to

extend special thanks to all of my friends who aided with this process - you know who you

are. I would like to extend a special thanks to Danny, Nathaniel, Moe, and Stephanie,

for their hours of research support this summer and throughout the years. Last, I would

like to acknowledge my family for their love and support. To Kwasi Adu-Basowah, thank

you for being there in both the good and hard moments, I appreciate everything you have

done and continue to do. To my brother, Niclas Manson, without you there is no way

I could have accomplished any of this-thank you. To my parents Richard and Dolores,

thank you very much for all of your love and guidance. Your encouragement has made

everything possible.

iv

Contents

1 Multisensory Processing and Visuomotor Regulation 1

1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Multisensory Combination and Integration . . . . . . . . . . . . . . . . . 2

1.3 Multisensory Integration: Neurophysiological Principles . . . . . . . . . . 3

1.3.1 Receptive Fields, Superadditivity, and Spatial and Temporal Prin-

ciples of Sensory Integration . . . . . . . . . . . . . . . . . . . . . 4

1.3.2 Multisensory Information in the Cortex . . . . . . . . . . . . . . . 6

1.4 Multisensory Perception at the Behavioural Level . . . . . . . . . . . . . 7

1.4.1 The Audiovisual Illusion . . . . . . . . . . . . . . . . . . . . . . . 8

1.4.2 The Neural Basis of the Audiovisual Illusions . . . . . . . . . . . 9

1.4.2.1 Neural Basis of the Fission Illusion . . . . . . . . . . . . 9

1.4.2.2 Neural Basis of the Fusion Illusion . . . . . . . . . . . . 10

1.5 Multisensory Integration during Action . . . . . . . . . . . . . . . . . . . 11

1.5.1 Sensory Gating during Goal-Directed Action . . . . . . . . . . . . 12

1.5.2 Audiovisual Perception during Goal-Directed Actions . . . . . . . 13

1.5.3 The Case for Visuomotor Regulation . . . . . . . . . . . . . . . . 16

1.5.4 Experimental Aims and Rationale . . . . . . . . . . . . . . . . . . 18

1.5.5 Hypotheses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

1.5.5.1 Movement Variables . . . . . . . . . . . . . . . . . . . . 19

1.5.5.2 Perceived Flashes Analyses . . . . . . . . . . . . . . . . 19

v

2 Methods and Results 22

2.1 Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.2 Apparatus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.3 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2.4 Data Collection and Analyses . . . . . . . . . . . . . . . . . . . . . . . . 25

2.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

2.5.1 Perceived Flashes in Control . . . . . . . . . . . . . . . . . . . . . 27

2.5.2 Perceived Flashes during Movement . . . . . . . . . . . . . . . . . 27

2.5.2.1 Within Vision Condition . . . . . . . . . . . . . . . . . . 28

2.5.2.2 Between Vision Conditions . . . . . . . . . . . . . . . . 29

2.5.3 Movement Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

2.5.4 Velocity at Stimulus Mid-Point . . . . . . . . . . . . . . . . . . . 29

2.5.5 Endpoint Precision . . . . . . . . . . . . . . . . . . . . . . . . . . 30

2.5.6 Online Corrections . . . . . . . . . . . . . . . . . . . . . . . . . . 30

3 Discussion 37

3.1 Online Control with Vision of the Environment and No-Vision of the En-

vironment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

3.2 Perception of the Audiovisual Illusion and Limb Velocity . . . . . . . . . 39

3.2.1 Illusion Perception in Trials with Vision . . . . . . . . . . . . . . 39

3.2.2 Illusion Perception in Trials without Vision . . . . . . . . . . . . . 40

3.2.3 Relationship with Limb Velocity . . . . . . . . . . . . . . . . . . . 40

3.3 The Influence of Visual Environment on Perception of Fusion and Fission

Illusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

3.4 Explaining the Modulation of the Audiovisual Illusion: The Cautious Case

for Visuomotor Regulation Processes . . . . . . . . . . . . . . . . . . . . 43

3.5 Limitations to the Role of Visuomotor Regulation Processes . . . . . . . 46

3.5.1 Modulation of both Illusions in No-Vision . . . . . . . . . . . . . 46

vi

3.5.2 Ceiling Effects in No-Vision Trials . . . . . . . . . . . . . . . . . . 48

3.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

References 52

Appendices 58

Appendix A: Neurological Questionnaire . . . . . . . . . . . . . . . . . . . . . 60

Appendix B: Handedness Questionnaire . . . . . . . . . . . . . . . . . . . . . 61

Appendix C: Eyedness Assessment . . . . . . . . . . . . . . . . . . . . . . . . 62

Appendix D: Supplementary Analysis of Normalized Perceived Flashes during

Illusory Trials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

vii

List of Tables

2.1 Number of Perceived Flashes . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.1 Susceptibility to the Audiovisual Illusion . . . . . . . . . . . . . . . . . . 51

viii

List of Figures

1.1 Results of Tremblay and Nguyen 2010. The average number of perceived

flashes for the fusion (2 Flash, 1 Beep) illusion, is greater (indicating less

susceptibility to the fusion illusion) at high limb velocities as compared to

low limb velocities. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

1.2 Results of Tremblay, Wong, and Manson (2012). Participants were less

accurate in their perception of the number of beeps when the audiovisual

stimulus was presented during movement, however this alteration was not

related to movement phase or limb velocity. . . . . . . . . . . . . . . . . 21

2.1 Depiction of participant sitting with the target position aligned with their

mid-sagittal plane, reaching from the home position to the target location.

The arrows approximately depict where the stimulus onset occurred during

the reaching trajectory (i.e., 0, 100, and 200 ms relative to movement

onset). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

2.2 Depiction of aiming console and stimuli. A participant’s point of view of

the aiming console, with depiction of the home position (switch) as well

as the target, flash, and beep (piezoelectric buzzer) stimuli locations. . . 33

ix

2.3 Mean number of perceived flashes (and SEM bars) for the fusion (2 Flash,

1 Beep) illusion as a function of presentation time. In the vision condition,

participants perceived significantly more flashes in the 0 ms and 100 ms

conditions. In the no-vision trials, participants perceived fewer flashes

overall, and performance remained stable over the different presentation

times. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

2.4 Mean number of perceived flashes (and SEM bars) for the fission (1 Flash,

2 Beep) illusion as a function of presentation time. In both vision and

no-vision trials, participants were morse susceptible to the illusion in the

100 ms compared to the 0 ms presentation time. . . . . . . . . . . . . . . 35

2.5 R2 values as a function of movement proportion (and SEM bars). This

analysis was used to examine the amount of online corrections occurring

during vision versus no-vision trials. . . . . . . . . . . . . . . . . . . . . . 36

4 Normalized perceived flashes ([flashz] and SEM bars) for the fusion illusion

plotted as a function of presentation time. In both vision conditions par-

ticipants exhibit a higher relative flashz at the 0 ms and 100 ms conditions

compared to control. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

x

Chapter 1

Multisensory Processing and

Visuomotor Regulation

1.1 Introduction

In our daily interactions, our sensory receptors are flooded with tremendous amounts of

information emanating from the numerous stimuli present in our surroundings. One of the

most widely investigated lines of research in modern neuroscience and psychology relates

to the mechanisms and processes involved in transforming this vast amount of sensory

information into a coherent perceptual representation. More relevant to this thesis is the

fact that humans are seldom stagnant in their environment. Given the possibility that

our perceptual systems evolved to facilitate movement, it is logical to study how, or if, the

function of these systems changes as we engage in purposeful actions. The main objective

of this thesis is to examine multisensory integration during goal-directed movement by

testing whether perception of multisensory events is altered when an individual is engaged

in a voluntary sensorimotor behaviour.

1

Chapter 1. Multisensory Processing and Visuomotor Regulation 2

1.2 Multisensory Combination and Integration

To perform an action, one must gather information about the position of the body from

various sensory modalities, including somatosensory, vestibular, and visual systems. In-

formation from these senses must then be used in concert with visual and auditory

sensory signals relaying information about the external environment (Ernst & Bulthoff,

2004; Sabes, 2011). Seemingly simple movements, such as reaches towards a target,

require complex computations of both the target location and limb position before an

accurate reach can be initiated. Furthermore, these initial computations can be adjusted

based on the sensory feedback received as the movement unfolds (i.e., online control:

see Elliott et al., 2010). Thus, one’s ability to perform an accurate reaching movement

is therefore related to how multiple sources of sensory information about the body and

target locations are combined and integrated during planning and online control.

Multisensory combination refers to the act of associating complementary information

from multiple sensory afferences. Although multisensory combination is not a process

culminating in the formation of a useable percept, this process is believed to occur either

prior to or simultaneously with the processes responsible for the assembly of a perceptual

representation (Ernst & Bulthoff, 2004). When calculating the relative position of a

reachable object for example, multisensory combination processes use information about

the eye (or gaze) orientation relative to the head, in combination with the head position

relative to the trunk to evaluate the relative position between the eyes, body, and hand.

In contrast, Multisensory integration processes refer to the assimilation of information

emerging from different sensory afferences. For instance, when identifying the location of

the target object, visual and auditory signals from the object along with proprioceptive

cues about positions of the body segments are coded to provide a representation of the

body’s orientation and the external world. These sources of information are encoded and

transformed into a common reference frame (e.g., Medendorp, 2011). It is hypothesized

that the establishment of a common reference frame involves defining a space common to


all inputs, and transforming the locations of sensory signals into a common coordinate

system (e.g., Crawford, Medendorp, & Marotta, 2004). Once a common reference frame

has been established, signals from the different senses can be computed and weighted.

The mechanisms behind the formation of this common reference frame is an area of

interest for many contemporary researchers; however, the main focus of this thesis will be

on the factors affecting the integration. Specifically, the work presented below is focused

on the detection of multisensory stimuli is affected by visuomotor processes occurring

during voluntary movements.

Historically, the combination and integration of the senses was thought to be a late

phenomenon, occurring only after each modality has been processed independently and

investigations conducted under this framework have greatly enhanced our knowledge of

the individual sensory processing systems (Alais, Newell, & Mamassian, 2010). How-

ever, the discovery of multisensory neurons in subcortical and cortical areas (Stein &

Stanford, 2008; Alais et al., 2010; Stein & Meredith, 1993), the realization of the early

onset of multisensory processing (Murray & Wallace, 2012; Stein & Meredith, 1993), and

behavioural studies conducted on the perception of multisensory events (Stein & Mered-

ith, 1990; Todd, 1912) have shifted the traditional linear progression views of sensory

processing and have formed the basis for the development of a more multisensory model

of perception.

1.3 Multisensory Integration: Neurophysiological Prin-

ciples

Discussion of the neurophysiological basis of multisensory integration often begins with

a summary of the principles discovered through in depth study of the superior colliculus

in felines and non-human primates (Stein & Meredith, 1993). Located in the midbrain,

the superior colliculus is functionally associated with head orientation, and orienting


behaviours in response to visual, auditory, and tactile stimuli. Within the superior

colliculus researchers discovered the presence of multisensory neurons that were capable

of responding to bi-modal (visual and auditory, or visual and tactile), and trimodal

(visual-auditory-tactile) stimuli (Stein & Meredith, 1993; Alais et al., 2010). A myriad

of studies using single cell recordings have since revealed specific properties of these

multimodal neurons. The discovery of these properties has greatly shaped the principle

features for the experimental assessment of multisensory responses.

1.3.1 Receptive Fields, Superadditivity, and Spatial and Tem-

poral Principles of Sensory Integration

One of the first findings to emerge from electrophysiological investigations of single multi-

sensory neurons was the unique properties of their receptive fields. Multisensory neurons

in the superior colliculus are unique in that each neuron possesses multiple excitatory

receptive fields for each modality the neuron responds to (Alais et al., 2010). One partic-

ularly interesting property of these receptive fields is their broad spatial overlap. Specif-

ically, multisensory neurons in the superior colliculus tend to respond to stimuli in the

same region of space irrespective of the input modality (for an in-depth description of

receptive field properties see Murray & Wallace 2011).

Another noteworthy property of these multisensory neurons is the observed multi-

sensory enhancement that occurs when these neurons respond to stimuli comprised of

multiple modalities. One of the most cited examples of such multisensory enhancements

is the “superadditivity” observed in audiovisual multisensory neurons. When an audiovi-

sual stimulus is presented, the amount of activation recorded from a multisensory neuron

is greater than the sum of the same neuron’s response to either modality alone (Alais

et al., 2010). The magnitude of superadditive responses is also inversely related to the

strength of the stimulus that is presented. In general, the early studies demonstrate that

the combination of weak unimodal stimuli produces a greater response than a combi-


nation of stronger unimodal stimuli (e.g., Stein & Meredith, 1993). The persistence of

this observation has lead to the development of the principle of inverse effectiveness for

multisensory integration.

Both properties of multisensory neurons, overlapping receptive fields and superad-

ditivity, have interesting consequences with regard to multisensory integration in the

superior colliculus. One phenomena directly linked to both of the above-mentioned con-

cepts is the spatial principle of multisensory integration. This principle states that the

neural enhancement produced by multisensory stimuli is in part influenced by the spatial

alignment of their individual modalities. As expected, it is a common observation that

visual and auditory stimuli originating from the same spatial position are more likely

to be bound together, and produce a superadditive neural responses (Alais et al., 2010;

Stein & Meredith, 1993).

A second principle developed through investigation of multisensory interactions con-

cerns the relative timing of the two sensory events. As stated previously, the neural

enhancement of multisensory neurons in the superior colliculus is maximized when the

receptive fields of the auditory and visual inputs overlap. Typically, this phenomenon

is observed when the stimuli are presented simultaneously. Interestingly, at the level

of the superior colliculus, multisensory responses (i.e., enhancement or suppression) to

audiovisual stimuli may emerge even though the triggering stimuli are not temporally

coincident. The temporal principle of multisensory processing predicts that integration

effects will be greatest when neuronal responses triggered by the presentation of each

stimulus modality are within a temporal window (Alais et al., 2010; Stein & Meredith,

1993). The exact timing of the temporal window has been described to as both brief and

broad. Furthermore properties of the stimuli including the intensity of the individual

modalities and their order of presentation appear to influence the nature of temporal

requirement (Alais et al., 2010).

The above-mentioned principles have guided research into multisensory integration in


humans. These findings have also served as the basis for detecting multisensory responses

in areas associated with higher level functions.

1.3.2 Multisensory Information in the Cortex

The presence of multisensory interactions is not limited to the superior colliculus and

subcortical areas. Recent advances in neuroimaging and the development of novel ex-

perimental approaches have allowed researchers to probe and being to understand how

multisensory interactions are manifested at the level of the cortex (for a review see Kle-

men and Chambers, 2012). Investigations of multisensory interactions in the cortex

have enriched our understanding of the neurophysiological basis of multisensory integra-

tion. Many early investigations into the cortical contributions to multisensory integration

sought to identify responses and mechanisms similar to the those present in the supe-

rior colliculus. Although instances of superadditivity, inverse effectiveness, and spatial

and temporal congruence have been noted in cortical structures (e.g., posterior parietal

cortex, superior temporal gyrus, superior temporal sulcus), these interactions were much

more uncommon at the cortical level. Arguably, the more important discoveries emerg-

ing from investigations in the cortex have been the findings revealing early instances

of activations associated with multisensory responses and the possible mechanisms and

interactions associated with these activations.

In sum, investigations into multisensory interactions in the cortex have revealed that

primary sensory processing areas are associated with the encoding of multiple sensory

modalities. These interactions are attributed to projections from other primary sensory

areas or as a result of neuroplasticity (Alais et al., 2010; Klemen & Chambers, 2012).

Another important finding highlighted by cortical multisensory research is that these mul-

timodal interactions may occur earlier than previously thought (Ghazanfar & Schroeder,

2006). One study that clearly outlines the above-mentioned finding was completed by

Wang, Celebrini, Trotter, & Barone, 2008, who investigated primary visual cortex (V1)


activity in response to an audiovisual stimulus. Recordings were taken from two mon-

keys who were trained to complete a saccadic eye movement task signalled by either a

visual stimulus or visual-auditory stimulus. Compared to the visual stimulus alone, the

audiovisual stimulus elicited faster saccadic reaction times, and also reduced the latency

associated with V1 neural activity. The authors hypothesized that, because no V1 ac-

tivity was noted when auditory tone was presented alone (i.e., in a control condition),

this modulation could be a result of early multisensory processing by cells in V1. These

findings are also supported by TMS studies in humans (Ramos-Estebanez et al., 2007).

The study of mechanisms responsible for early multisensory processing in cortical

areas is still in its infancy relative to other lines of research on multisensory integra-

tion. However, investigations such as the one’s mentioned above have helped illuminate

some of the underlying mechanisms and allow for more accurate interpretations of early

behavioural findings.

1.4 Multisensory Perception at the Behavioural Level

Numerous studies have documented behavioural responses to mulitsensory stimuli. In

fact, it was early behavioural studies that inferred the neurophysiological principle of su-

peradditivity by demonstrating that individuals produce faster reaction times in response

to multisensory stimuli compared to unimodal stimuli (Todd, 1912). Furthermore, re-

search on perceptual responses to multisensory stimuli has revealed how the presence of

two or more modalities affects the perception of each individual modalities (e.g., Howard

& Templeton, 1966; McGurk & MacDonald, 1976). For example, in the well documented

visual capture or “ventriloquist illusion”, the location of speech sounds is biased by the

presence of moving parts in the mouth of a dummy. In many instances, these interactions

between modalities give rise to illusions. Illusions apparent at the behavioural level have

become very useful tools in multisensory research, both to examine the time course of


neurophysiological activations associated with processing different types of sensory in-

formation, and as tools to inform models of multisensory integration related to human

behaviour.

1.4.1 The Audiovisual Illusion

The main stimuli employed in this thesis was an audiovisual illusion (Shams, Kamitani,

& Shimojo, 2000). Shams et al. (2000) first described the appearance of an auditory-

induced visual illusion occurring when participants were asked to report the number of

visual stimuli when presented with an audiovisual stimulus comprised of brief flashes and

briefs sounds. In their study, they flashed a uniform white disk (diameter subtending

2 degrees of visual eccentricity) located at 5 degrees below a fixation point on a dark

screen. Either 1 or 2 brief flashes, with a stimulus onset asynchrony (SOA) of 50 ms,

were presented on any given trial. Accompanying the presentation of the flashes were

1 to 4 brief auditory tones (57 ms SOA between tones). In the condition wherein 1

flash was presented with 2 or more tones, participants tended to perceive an additional

illusory flash (Shams et al., 2000). This perception of an illusory flash, when 1 flash is

accompanied by 2 tones is known as the fission illusion (see also Andersen, Tiippana, &

Sams, 2004).

Andersen et al. (2004) further investigated factors contributing to this illusion when

they sought to characterize the effect of different stimulus parameters and task instruc-

tions on the number of perceived flashes. Using a very similar stimulus and protocol to

that described in Shams et al. (2000), these authors noted the appearance of two types

of audiovisual illusions. Not only did they find the appearance of the illusory perceptual

flash, the fission illusion, but they also discovered that when 2 flashes were paired with

1 tone, there was a tendency for participants to report the occurrence of only 1 flash.

The perception of 1 flash when 2 flashes are presented with 1 tone is known as the fusion

illusion (Andersen et al., 2004).


Both illusions have since been employed to help understand concepts of multisensory

integration using different tasks and populations (e.g., Innes-Brown et al., 2011; Marco,

Hinkley, Hill, & Nagarajan, 2011). Often, both fusion and fission are used in experiments

and discussed with reference to a common mechanism. However, recent neurophysiolog-

ical evidence on the time course of neural activations associated with these illusions

suggests the mechanisms underlying the perception of each may be different (Mishra,

Martinez, Sejnowski, & Hillyard, 2007).

1.4.2 The Neural Basis of the Audiovisual Illusions

The neural basis of the fusion and fission illusions has been explored through the use of

neurophysiological recordings (Shams, Kamitani, Thompson, & Shimojo, 2001; Watkins,

Shams, Josephs, & Rees, 2007; Mishra et al., 2007; Mishra, Martinez, & Hillyard, 2008;

Zhang & Chen, 2006). Shams et al. (2001) measured event related potentials (ERPs)

during trials where the fission illusion was presented. The researchers observed a relation-

ship between the illusory flash and activity in the occipital cortex. These results provided

evidence for the role of visual cortex in the perception of the illusory flash. These data

were supported by a functional magnetic resonance imaging (fMRI) study where they

observed increased activation in multisensory areas (e.g., right superior temporal sulcus)

and V1 during trials wherein the illusion was perceived compared to trials where the the

illusion was not experienced (Watkins et al., 2007).

1.4.2.1 Neural Basis of the Fission Illusion

Using a high resolution electroencephalography (EEG) system, Mishra et al. (2007)

sought to better characterize the precise timing of neural activity related to the fission

illusion. A broad range of ERPs were recorded from participants while they observed

different combinations of unimodal auditory (brief sounds alone), unimodal visual (brief

flashes alone), and audiovisual stimuli (a combination of flashes and sounds). On trials


where unimodal visual and audiovisual stimuli were presented, participants were asked to

report the number of flashes they perceived. Overall, participants reported experiencing

the fission illusion on 37 % of the audiovisual trials (similar rates were noted in Watkins

et al., 2007). To appropriately assess ERPs associated with the fission illusion, the

researchers conducted two main analyses. First, the group of participants was split

based on their susceptibility to the illusion. Participants who saw the illusion more often

were analyzed as the SEE group and those who were less susceptible to the illusion were

analyzed as the NO SEE group. Second, the researchers also analyzed ERPs within

each participant for the illusion trials (classified as SEE trials) and compared them to

no illusion trials (NO SEE trials). Participants in the SEE group exhibited a pattern

of activity characterized by a significantly larger positive deflection at 120 ms (PD 120)

compared to the NO SEE group. The timing of this deflection was approximately 30-60

ms after the onset of the second tone. Localization of the PD 120, suggests this early

activity related to the illusion occurs in the extrastriate visual cortex. Trial-by-trial

analyses revealed that the illusion could be a function of auditory processing activity. In

trials where the illusion was perceived, researchers found noted a negative deflection 110

ms after stimulus onset (20-40 ms after the second tone) localized in superior temporal

gyrus and auditory cortex. Taken together these results suggest the appearance of the

fission illusion is due to rapid bursts of activity in auditory, visual, and multisensory

areas.

1.4.2.2 Neural Basis of the Fusion Illusion

Mishra et al. (2008) investigated the occurrence of the fusion illusion. When participants

were presented with a multisensory stimulus comprised of 2 brief flashes and 1 brief sound,

participants perceived fusion of the two visual stimuli (Mishra et al., 2008). Analysis of

this illusion revealed a pattern of neural activity different than the pattern observed in

fission (Mishra et al., 2007). Specifically, auditory induced fusion was associated with


a positive deflection at 180 ms (PD 180) and a large persistent negativity at 240 ms

(ND 240). Similar to the analyses used in Mishra et al. (2007), Mishra et al. (2008)

separated participants into groups based on the frequency of the perceived illusion: a

SEE1 group that saw the illusion on more than 50 % of trials and a SEE2 group that

was less susceptible to the illusion. The researchers also looked at the differences in

neural activity between trials by separating trials into illusion trials (SEE trials) and

non-illusion trials (NO SEE trials). The main finding of the between-subjects analyses

was the sizeable reduction in the SEE1 group’s PD 180. Trial-by-trial analyses of PD 180,

similar to between-subjects analyses, also revealed a reduction in PD 180 in illusion trials

across all participants. Activity of the PD 180 was localized to the superior temporal

sulcus, an area implicated in multisensory processing. The trial-by-trial analyses also

revealed a difference in ND 240 wave localized in the occipital cortex occurring about 60

ms after the PD 180. The researchers suggested this localization could be the result of

interactions between visual cortex and the superior temporal sulcus.

The conclusion of these ERP studies suggests that the illusions are associated with

both communication between primary processing areas and activations in multisensory

areas. Also these investigations suggests that even though both illusions are comprised of

different combinations of the same unimodal stimuli, the neural mechanisms associated

with the experience of each illusion are distinct.

1.5 Multisensory Integration during Action

Most of the research looking at multisensory integration, especially audiovisual integra-

tion, has examined the perception of multisensory events while the actor is little more

than a passive observer. Parallel to the progression of research investigating sensory inter-

actions, the earlier work on perception during action was focused on unimodal paradigms

(Alais et al., 2010). Furthermore, many studies on the relationship between action and


perception at the end of the 20th century have been devoted to understanding the dis-

sociation between perception and action (e.g., Goodale & Milner, 1992). With the fairly

recent propositions that the processes of perception and action share common mecha-

nisms (Prinz, 1997), there is a growing need for investigations examining how engagement

in action influences perceptual systems.

1.5.1 Sensory Gating during Goal-Directed Action

One possible reason for the little work to date describing multisensory integration dur-

ing movement is the observation that conscious perception of some sources of sensory

information is suppressed or “gated” during action. One of the most well studied forms

of sensory gating occurs in eye movements (i.e., saccades) where conscious perception

of visual information is suppressed as the eye moves rapidly. During eye movements,

humans do not perceive the blurred images of the world on retina nor can they perceive

the real-time displacement of their eyes (Bridgeman, Hendry, & Stark, 1975).

More relevant to this thesis is the early research on tactile gating in limb movements.

One of the first descriptions of tactile gating was described in a study by Chapman,

Bushnell, Miron, Duncan, and Lund (1987), who noted a reduction in sensitivity to tactile

stimuli at the onset of an upper arm movement. In their study, participants performed

three tasks: a detection task, a forced-choice judgment of stimulus intensity differences,

and a subjective magnitude estimation task. During these tasks, participants were either:

moved passively, asked to move actively, or remained at rest while electrical stimulation

was applied to the forearm. Overall, the authors found that the participants’ ability to

detect the presence of tactile stimulation was significantly reduced in both the active and

passive movement conditions compared to the resting condition. These findings have

been replicated numerous times, and are in accordance with neurophysiological studies

demonstrating that the transmission of cutaneous signals to the primary sensory cortex is

reduced during movement (Chapman et al., 1987; Ghez & Lenzi, 1971; Seki, Perlmutter,


& Fetz, 2003).

Although the observation of sensory gating and sensory suppression during goal-

directed actions are fairly stable in the literature, recent evidence suggests the process

of perception may change as function of movement phase. For instance, in a examining

the time course of tactile sensitivity during goal-directed action, it was found that tactile

detection thresholds changed as a function of when the stimulus is presented (Juravle,

Deubel, Tan, & Spence, 2010). The researchers performed three experiments where

participants had to perform goal-directed movements between two computer mice, spaced

25 cm apart. In the first experiment, tactile stimulation was applied to the hand at 4

points during the movement: during preparation (prior to a “go” signal), during initiation

(0 ms after the “go” signal), during execution (100 ms after the movement onset), or after

movement completion (100 ms after grasp of the goal mouse). As the authors predicted,

their was a decrease in tactile sensitivity during the movement (e.g., 100 ms after the

first mouse was released). In addition to the reduced sensitivity during movement, the

authors observed comparable levels of sensitivity when tactile stimulation was applied

during movement preparation and after movement execution. These data would suggest

that tactile sensitivity changes depending on movement phase. Based on these findings,

the authors concluded that sensory suppression may be greatest in the execution phase

of the movement, but tactile sensitivity returns to levels comparable to preparation after

contact.

1.5.2 Audiovisual Perception during Goal-Directed Actions

Two main findings emerge from the early studies on tactile detection during goal-directed

actions. As mentioned above, the first and most common observation is that perception

of certain sensory information can be gated during action. The second observation is that

this gating of sensory information could be dependent on movement phase. The idea that

perception of certain sensory events may be dependent on movement phase could have


consequences for the perception of multisensory stimuli. For example, if engagement in a

goal-directed task alters the perception of tactile cues, and this sensitivity is dependent

on the movement phase, then it is possible that other modalities may show a similar

movement-dependent alteration.

One study to further test this hypothesis of movement-dependent sensory processing

was conducted by Tremblay and Nguyen (2010). The authors reasoned that if tactile sen-

sitivity is decreased during different movement phases, perhaps the processing for more

task-relevant information (e.g., vision) may be prioritized. The importance of vision for

the planning and control of goal-directed action has been thoroughly documented (see

Elliott et al., 2010). Thus, the authors hypothesized that perception of visual events

could be altered depending on the phase of the movement. The authors employed the

above-mentioned audiovisual illusion (Shams et al., 2001) as a way of monitoring visual

perception during voluntary action. Participants performed rapid goal-directed aiming

movements to a visual target (5 mm diameter) located 30 cm away from the start po-

sition. Below the target position, an audiovisual stimulus (1 or 2 beeps accompanied

by 1 or 2 flashes) was presented at 0, 50, 100, 150, or 200 ms relative to movement on-

set. Participants were asked to reach the target location as accurately as possible, while

trying to complete movements within a 290 to 350 ms movement time bandwidth. Par-

ticipants were also asked to report the number of flashes they perceived after each trial,

although they were told that this was a secondary task. Overall, the results of the study

revealed that participants were less susceptible to the fusion illusion (i.e., the erroneous

perception of 1 flash when 2 flashes are presented with 1 beep) when the stimulus was

presented at 50 ms and 100 ms compared to when it was presented at 0 ms and 200 ms

relative to movement onset (see Figure: 1.1). The authors also noted the limb velocity

was highest when the stimulus was presented at these time points, and these time points

corresponded to 20-50 % of overall movement time (Tremblay & Nguyen, 2010). This led

the authors to conclude that perception of audiovisual events is modulated as a function


of the real-time limb velocity.

Recent follow-up work by Tremblay, Wong, and Manson (2012) explored if the find-

ings reported by Tremblay and Nguyen (2010) could be explained by sensory gating.

Tremblay et al. (2012) employed a similar experimental task and protocol to Andersen

et al. (2004). Also, prior to the experimental trials, the participants’ auditory detection

thresholds were measured using an auditory stimulus discrimination procedure (i.e., ad-

justing the SOA until participants experienced an illusion on 50% of the illusory-inducing

trials). After the detection threshold was estimated, participants were asked to perform

aiming movements to a visual target (30 cm amplitude). Audiovisual stimuli (1 or 2

tones accompanied by 0, 1, or 2 flashes) were presented below the 30 cm target at either

at 0, 100, or 200 ms relative to movement onset. Participants reported the number of

perceived beeps after each trial. Overall, the perception of auditory events was decreased

during all portions of goal-directed actions (see Figure: 1.2). These results suggest there

is gating of auditory information throughout the movement; however, because the au-

thors found no relationship between auditory gating and real-time limb velocity, they

concluded that the gating of auditory information could not solely explain the results of

Tremblay and Nguyen (2010).

Based on the results of Tremblay et al. (2012), auditory suppression alone may not be

a sufficient mechanistic explanation for the decreased susceptibility to the fusion illusion

noted in Tremblay and Nguyen (2010). Another possible explanation, and the focus of

this thesis, is rooted in the idea that visual information is processed to a greater extent at

high velocity portions of a reaching trajectory in order to obtain information necessary for

online visuomotor regulation. Support for this hypothesis is noted in the contemporary

work examining the use of vision during upper-limb reaches.


1.5.3 The Case for Visuomotor Regulation

Discussions of the visual control of movement often begin with the seminal experiment

conducted by Woodworth (1899), who asked individuals to perform reciprocal tracing

movements using a pencil on a roll of paper that was rotating at a constant speed. The

target lines were a fixed distance apart and participants were required to alter their pace

to the sound of a metronome. Importantly, Woodworth (1899) also had participants

complete these movements with vision (eyes open) and without vision (eyes closed).

There were two main findings that emerged from Woodworth’s work. The first is that

limb movements are comprised of two components; 1) a ballistic or initial impulse phase

that propels the limb in the direction of the target, and 2) a homing-in or current-control

phase wherein adjustments to the initial trajectory bring the limb to the target accurately.

The second main finding of Woodworth’s work was that time with vision is important

for the completion of limb movements. Woodworth (1899) noted that participants were

more accurate in the eyes open condition, but also as movements increased in speed, the

differences in accuracy between the eyes closed and eyes open conditions became smaller,

yielding no accuracy differences when reciprocal movements took less than 400 ms.

Since Woodworth (1899), how vision is used in the planning and control of movements

has been thoroughly investigated (see Elliott, Helsen, & Chua, 2001; Elliott et al., 2010,

for reviews). Most relevant to this thesis, is the branch of research examining how vision

is used to make trajectory amendments. Traditionally, vision was thought to be most

useful to individuals at later stages of the aiming movements. Beaubaton and Hay (1986)

asked participants to aim to targets in five vision conditions: visual feedback throughout

the trajectory, no visual feedback, feedback at the end of the movement, feedback during

the initial half of the trajectory, and feedback at the terminal half of the trajectory. The

authors noted that movements completed with terminal feedback were just as accurate

as movements completed with vision throughout the trajectory. Furthermore, providing

vision for only the initial half of the trajectory yielded precision values similar to the no


visual feedback condition. However, these conditions were presented in a blocked fashion,

which is known to lead participants to spend more time of the reaching trajectory in the

portion of the movement where vision is available (e.g., Carlton, 1981). Overall, the

conclusion of the early study on vision utilization suggests providing visual feedback in

later portions of movement is more beneficial for limb control.

Since these initial experiments, it has been hypothesized that the improvements ob-

served when providing visual feedback later in movements is actually based on informa-

tion obtained when vision first becomes available. Khan and Franks (2003) replicated

the original results of Beaubaton and Hay (1986) and showed that providing vision during

the first 50% of a trajectory does not yield better endpoint performance than no-vision.

However, they also found that when participants were given vision during the first 75% of

the movement, individuals greatly improved their performance. Khan and Franks (2003)

hypothesized that visual information acquired near the peak velocity of movements is

important for online visuomotor regulation.

A recent study by Tremblay, Hansen, Kennedy, and Cheng (2013) examined the link

between velocity, vision, and limb control. The results of this experiment also provide

a possible explanation for the decreased susceptibility to the fusion illusion noted in

Tremblay and Nguyen (2010). In Tremblay et al. (2013), the experimenters manipulated

vision at different limb velocities and measured aiming performance of rapid goal-directed

reaches. Consistent with the findings of Tremblay and Nguyen (2010), the researchers

observed that trials where participants had vision only at high limb velocities (>0.8 m/s)

had similar endpoint variability to trials where participants had vision throughout the

movement. The authors concluded that visual information obtained from high velocity

portions of the limb movement is particularly important for online visuomotor regulation

processes (Tremblay et al., 2013).


1.5.4 Experimental Aims and Rationale

The main objective of this thesis was to test a possible mechanism for the modulation of

audiovisual processing at high limb velocities, as noted in Tremblay and Nguyen (2010).

One possible explanation, not yet is explored is the role of visuomotor regulation pro-

cesses on the perception. The literature presented above suggests that visual information

obtained at high limb velocities is important for visuomotor regulation. It is therefore

possible that engagement in visuomotor regulation may facilitate the uptake of visual

information during action. To investigate this hypothesis, a similar protocol as the one

used in Tremblay and Nguyen (2010) was employed, but in half of the experiment, par-

ticipants performed reaches without vision of the environment.

Manipulating vision of the environment was employed because it has been demon-

strated that movements without vision of the limb are characterized by less visuomotor

regulation and a more offline mode of control (Heath, 2005). Heath (2005) manipulated

vision of the limb because both visual information of the target and limb may be critical

for visuomotor regulation processes (i.e., online corrections). Trials without vision of the

limb were characterized by a more pre-planned mode of control, and exhibited a higher

endpoint errors. These results indicated that vision of the limb was important for visuo-

motor regulation, and movements performed without vision of the limb may rely less on

online sensory feedback.

Therefore, if participants do not show the same velocity-dependent modifications in

their perception of flashes without vision of the environment, then the engagement in

online visuomotor regulation processes could be a viable explanation for the results of

Tremblay and Nguyen (2010). In contrast, if vision of the environment does not alter the

modulation of the fusion illusion across the different stimuli presentation times, then the

other mechanisms will need to be explored.


1.5.5 Hypotheses

1.5.5.1 Movement Variables

In the present study, it was expected that, when performing aiming movements in the no-

vision condition (i.e., no-vision the environment including the limb), participants should

exhibit higher endpoint variability, and a more pre-planned mode of control. Further-

more, it was expected that participants would remain within the movement time band-

width (290-350 ms).

1.5.5.2 Perceived Flashes Analyses

In the present study, it was expected that participants would be susceptible to both the

fusion and fission illusion in the control condition. It was also hypothesized that, when

aiming with vision, participants would experience the fusion illusion less often when the

associated stimuli are presented at 100 ms compared to when the stimuli are presented

during control, 0 ms and 200 ms (similar to Tremblay and Nguyen 2010). Also, the

limb velocity at the 100 ms presentation time should be higher than the limb velocities

in the 0 ms and 200 ms presentation times. No modulation of the fusion illusion was

expected in no-vision because participants were expected to adopt a more pre-planned

aiming strategy. Lastly, no modulated of the fission illusion was expected.


Figure 1.1: Results of Tremblay and Nguyen 2010. The average number of perceived

flashes for the fusion (2 Flash, 1 Beep) illusion, is greater (indicating less susceptibility

to the fusion illusion) at high limb velocities as compared to low limb velocities. Adapted

from “Real-Time Decreased Sensitivity to an Audio-Visual Illusion during Goal-Directed

Reaching,” by L.T. and T.N. PLoS ONE, 5, p.3. Copyright 2010 by Tremblay, L. and

Nguyen, T. Adapted with permission.


Figure 1.2: Results of Tremblay, Wong, and Manson (2012). Participants were less

accurate in their perception of the number of beeps when the audiovisual stimulus was

presented during movement, however this alteration was not related to movement phase

or limb velocity.

Chapter 2

Methods and Results

2.1 Participants

Fifteen right-handed participants (6 female, 9 male, age range: 20 - 44 years) with self-

reported normal or corrected-to-normal vision and hearing were recruited from the Uni-

versity of Toronto community. Participants were naıve to the purpose of the experiment

and had no self-reported history of neurological impairment (see Appendix A). Hand

dominance was assessed using a handedness inventory questionnaire (Oldfield, 1971; see

appendix B) and eye dominance was assessed with a simple eye-target alignment test

(Miles, 1930: see Appendix C). Written informed consent was obtained prior to the ex-

periment and the protocol was approved by the University of Toronto Research Ethics

Board. Participants received $15 for their time.

2.2 Apparatus

The experiment took place in a dark room located in the Perceptual-Motor Behaviour

Laboratory at the University of Toronto. Participants were seated on an adjustable

kneeling chair at a desk (73.5 cm in height) with a custom built aiming console (50 cm x

27.5 cm x 8.5 cm) positioned on a desk (see Figure 2.1). The aiming console was equipped

22

Chapter 2. Methods and Results 23

with a home position (i.e., small microswitch), 2 light emitting diodes (LED) of 0.3 cm in

diameter and a piezoelectric buzzer (Model SC628: Mallory Sonalert Products Inc. 2900

Hz) (see Figure 2.2). The microswitch was used to detect movement onset and trigger the

presentation times of the audiovisual stimuli (see below for details). The target LED was

located 30 cm to the left of the home position, and the position of the aiming console was

adjusted such that the target position was aligned with the participant’s midline. A flash

LED was located 6 cm below the target (i.e., proximal to the participant). The centre of

the piezoelectric buzzer was located within 1 cm of the flash LED. An infrared emitting

diode (IRED) was placed on the participant’s right index finger and the location of the

IRED was tracked by an Optotrak Certus (Northern Digital Inc.) sampling at 500 Hz

sampling for 2 s. A custom Matlab (The MathWorks Inc.) program was used to collect

limb position data and send outputs to the aiming console. An analog-to-digital board

(PCI-6024E: National Instruments Inc.) was used to deliver the digital signals to the

LEDs and the piezoelectric buzzer. Information from the microswitch was also gathered

using Matlab and a custom built breakout box connected to the computer’s parallel port.

Throughout the experiment, the experimenter was seated outside of the room, and

monitored the participant through a infrared webcam (Sabrent Infrared NightVision

WCM-6LNV: Sabrent USA). The webcam allowed the experimenter to monitor fatigue

and posture of the participants and enabled the participant to communicate to the exper-

imenter through the built-in microphone. Communication to the participant was enabled

by 2 way radios (Motorola Talkabout MR350R: Motorola Solutions Inc.).

2.3 Procedure

The overall experiment consisted of 2 sets of 3 experimental phases. Participants com-

pleted one set of phases with vision of the environment (vision condition) and completed

the other set of phases without vision of the environment (i.e., with the room lights


extinguished) which includes withdrawing vision of the limb (no-vision). The order of

the sets was counterbalanced across participants. In the first two phases participants

were asked to perform fast and accurate reaching movements from the home position to

the 30 cm target. In the familiarization phase, participants performed 10 reaches and

received feedback about their movement accuracy and duration to the nearest 1 mm and

10 ms increment (e.g., 3 mm short and 320 ms). Experimentally, the goal of this phase

was to have participants reach the target within a movement time bandwidth of 290 to

350 ms. Participants were reminded of the movement time bandwidth, but they were

instructed to prioritize accuracy. In the second phase of the experiment, 1 or 2 flashes

(24 ms in duration) accompanied by 1 or 2 beeps (24 ms in duration) were presented at

1 of 3 presentation times during the reaching movement: 0, 100, or 200 ms relative to

movement onset. In the 1 Flash,1 Beep condition (1F1B), both modalities were triggered

simultaneously (e.g., SOA <1 ms). In illusion-inducing conditions (e.g., 2 Flash, 2 Beep

[2F1B] and 1 Flash, 2 Beep [1F2B]) the SOA between the onset of visual and auditory

stimuli was 36 ms. In the 2 Flash, 2 Beep (2F2B) conditions, both stimulus modalities

were triggered simultaneously, the SOA between the sets of stimuli was 72 ms. For all

phases, participants were asked to fixate on the green target.

The four possible combinations of stimuli were presented 12 times for each presenta-

tion time, yielding 144 trials. In the no-vision condition, in order to avoid dark adapta-

tion, the room lights were turned on for 2 minutes every 5 minutes in the no-vision phase

(every 40 trials). Feedback about accuracy was not provided in the experimental phases

but the participant was informed if their movement was too fast or too slow for two con-

secutive trials. The third experimental phase was a control phase in which participants

placed their hand on the home position and did not perform reaching movements. Partic-

ipants were exposed to 1 or 2 flashes accompanied by 1 or 2 beeps, which were presented

at 100 ms relative to target onset. After each trial, in all phases, participants verbally

reported the number of perceived flashes. Participants were periodically reminded that


the main goal was to aim as accurately as possible to the target. In total, the experi-

ment consisted of 404 trials (10 familiarization, 144 experimental and 48 control for each

phase).

2.4 Data Collection and Analyses

Movement time was calculated as the difference between movement onset and movement

offset. The sample where the microswitch was released was labelled as movement onset

and movement offset was labelled when the limb velocity in the primary axis fell below

30 mm/s for 2 consecutive samples. Other dependent measures calculated were end-

point precision (the standard deviation of IRED position at movement end), velocity at

stimulus mid-point (velocity of the limb at half of the audiovisual stimulus presentation

duration), and IRED position in the primary axis at every 5% of total movement time.

The main dependent variable was the number of perceived flashes. To ascertain if

the illusion was present at rest, a 2 Vision (vision, no-vision) by 2 Flash (1, 2) by 2 Beep

(1, 2) repeated measures ANOVA was used to analyze responses for each experiment in

the control trials. For the main analysis a 2 Vision by 4 Presentation Time (Control,

0ms, 100 ms, 200ms) by 2 Flash by 2 Beep repeated measures ANOVA was conducted.

As the main purpose of the experiment was to determine if the illusion was modulated

during action, post- hoc comparisons were made for the fusion (2F1B) and fission (1F

2B) illusions comparing across presentation times and between vision conditions. A 2

Vision by 3 Presentation Time (0, 100, 200 ms) by 2 Flash by 2 Beep repeated-measures

ANOVA was conducted to analyze measures associated with the limb trajectories. These

included movement time (ms), limb velocity at stimulus mid-point (m/s), and endpoint

precision (mm).

Inferences about the extent of visuomotor control processes were obtained through

the use of R2 analyses adapted from Heath (2005). In this type of correlation analysis


the position of the limb at various proportions of the trajectory (e.g., 25% 50% and

75% of movement time) are correlated to the position at movement end. One strength

of this analysis is that it does not consider the movement endpoint distribution from

trial-to-trial but rather assesses the trajectory scaling. When high correlations between

the limb position at points during the trajectory and movement end are observed, this

is an indication that the aiming trajectories are more stereotyped. Stereotypical limb

trajectory profiles are associated with greater pre-planning and less online control. When

low correlation coefficients are observed, this is thought to indicate more online control.

Heath et al. (2004) noted that upper-limb reaching trajectories performed with vision

exhibited significantly lower within trial correlation coefficients compared to trials where

there was no visual feedback. This is not surprising, because individuals likely utilize

online visual feedback to make amendments to the limb trajectory. In the present study,

R2 values were calculated based on the limb position at 15%, 45% and 75% of the

movement trajectory. These proportions are slightly different than those used by Heath

(2005), but were more appropriate as they correspond to the average movement time

proportion at which the stimulus mid-point occurred for each of the stimulus presentation

times of the present study. These values were submitted to a 2 Vision by 3 Proportion

(15%, 45% and 75%) repeated measures ANOVA. For the above-mentioned analyeses

alpha was set at .05 and Tukey’s Honestly Significant Difference (HSD) post-hoc test was

used to decompose any significant interactions involving more than two means. Where

sphericity was violated, the Hyunh-Feldt correction was applied (corrected degrees of

freedom are reported to one decimal place). The data reported below are the mean

values and the associated standard deviation.


2.5 Results

2.5.1 Perceived Flashes in Control

Analysis of the control trials revealed both illusions were apparent in both vision con-

ditions at rest (i.e., control). There were significant main effects of Flash, F(1,14) =

12.57, p <.05, and of Beep, F(1,14) = 95.31, p <.001. Participants generally perceived

fewer flashes when 1 beep was presented (1.13 ±0.25) compared to when two beeps were

presented (1.75 ±0.31). Also, as expected, participants perceived more flashes when 2

flashes were presented (1.53 ±0.41) compared to when only one flash was presented (1.34

±0.41). This analysis of the control phases also revealed a significant main effect of

Vision, F(1,14) = 6.35, p <.05 and a significant Vision by Flash interaction, F(1,14) =

19.34, p <.001, HSD = 0.15. Overall, participants reported seeing fewer flashes in the

no-vision condition (1.38 ±0.40) compared to the vision condition (1.49 ±0.44). Breaking

down the interaction revealed that when 2 flashes were presented, participants reported

fewer flashes in the no-vision condition (1.40 ±0.39) compared to the vision condition

(1.67 ±0.39). The number of flashes did not differ between vision (1.32 ±0.40) and

no-vision (1.36 ±0.37) when only one flash was presented (see Table: 2.5.6).

2.5.2 Perceived Flashes during Movement

The main analysis revealed several significant main effects and interactions. Only the sig-

nificant main effects and the highest order interactions were decomposed and presented.

The analysis revealed a significant main effect of Vision, F(1,14) = 31.41, p <.001. Sim-

ilar to what was noted in the control conditions, participants overall perceived a greater

number of flashes with vision (1.55 ±0.42) compared to no-vision (1.41 ±0.37). There

was also a significant main effect of Presentation Time, F(3,42) = 6.92, p <.01, where

participants perceived significantly more flashes at 100 ms (1.55 ±0.40) compared to any

other presentation time (control: 1.44 ±0.42; 0 ms: 1.46 ±0.36; 200 ms: 1.48 ±0.41).


Both a significant main effect of Flash, F(1,14) = 29.1 p <.001, and a significant effect

of Beep, F(1,14) = 84.59, p <.001, were also observed. As expected, throughout the

different conditions, participants’ perception of the number of flashes was influenced by

the number of beeps and flashes.

There were significant 2-way interactions between Vision and Flash, F(1,14) = 84.59,

p <.001, Presentation Time and Flash, F(3,42) =4.25, p <.05, Presentation Time and

Beep, F(3,42) = 11.75, p <.001, and Flash and Beep, F(1,14) = 5.003, p <.01. There

were also significant 3-way interactions between Vision and Presentation Time and Flash,

F(3,42) = 3.17, p <.05, and between Vision and Flash and Beep interaction, F(1,14) =

12.40, p <.01. Critically, there was also a significant 4-way interaction between Vision

and Presentation Time and Flash and Beep, F(3,42) = 2.84, p <.05, HSD = 0.17. The 4

way interaction was decomposed with a focus on comparisons both within and between

the two types of vision trials (vision and no-vision) for both types of illusion trials (Fusion

and Fission).

2.5.2.1 Within Vision Condition

Within the trials with vision of the environment, post-hoc tests on the fusion inducing

conditions (2 flashes 1 beep) revealed that participants perceived a greater number of

flashes during the 0 ms (1.66 ±0.34) and 100 ms (1.62 ±0.32) presentation times compared

to the control (1.40 ±0.37) and 200 ms (1.46 ±0.34) presentation times. For the fission

inducing trials (1 Flash, 2 Beep), participants perceived more flashes in the 100 ms

(1.85 ±0.25) presentation time compared to control (1.63 ±0.39) and 0 ms (1.51 ±0.32).

Also, participants reported more flashes in the 200 ms (1.75 ±0.31) compared to the

0 ms presentation time (see Figure 2.3). For trials performed without vision of the

environment, post-hoc analyses revealed no differences between presentation times for

fusion inducing trials. For fission inducing trials, when the stimulus was presented at 100

ms, participants perceived significantly more flashes (1.75 ±0.23) compared to when the


illusion was presented at 0 ms (1.56 ±0.26), see Figure 2.4.

2.5.2.2 Between Vision Conditions

Post-hoc analyses on fusion trials for both vision conditions revealed significant differences

for every presentation time. When performing reaches with vision, participants perceived

a greater number of flashes in control (1.40 ±0.36), 0 ms (1.66 ±0.35), 100 ms (1.62

±0.33) and 200 ms (1.46 ±0.34) compared to the analogous presentation times in no-

vision: control: 1.07 ±0.10; 0 ms: 1.17 ±0.20; 100 ms: 1.14 ±0.21; 200 ms: 1.11 ±0.16.

There were no significant differences between vision conditions for any presentation times

for the fission illusion.

2.5.3 Movement Time

Data analyses with regard to movement time revealed no significant main effects or

interactions. Also, the average movement times for both vision (325 ms ±21) and no-

vision trials (327 ms ±22) were within the established bandwidth.

2.5.4 Velocity at Stimulus Mid-Point

Analysis of the velocity at stimulus mid-point yielded a main effect of Presentation Time

F(1.3,17.8) = 58.36, p <.001, HSD = 0.58, and a significant Vision by Presentation

Time by Flash by Beep interaction, F(2,28) = 9.68, p <.001, HSD = 0.13. Overall,

the velocities at stimulus mid-point were significantly higher for the 100 ms (2.91 m/s

±0.36) and 0 ms (2.29 m/s ±0.56) presentation times compared to the 200 ms (1.0 m/s

±0.32) presentation time. Furthermore, limb velocity at stimulus mid-point in the 100

ms presentation time was significantly higher than the 0 ms presentation time. Breaking

down the 4-way interaction revealed no meaningful differences. That is, velocities between

vision conditions for the same presentation times and velocities were not different. There


was also no differences between Flash or Beep conditions within a given presentation

time.

2.5.5 Endpoint Precision

Analysis of variable error in the primary movement axis revealed a significant main effect

of Vision, F(1,14) = 49.34, p <.001. As expected, participants were significantly more

precise in the vision trials (4.46 ±1.69), compared to the no-vision trials (7.5 ±6.06).

2.5.6 Online Corrections

To infer the amount of online corrections, and make inference about the contribution of

visuomotor control processes, an R2 analysis was conducted. The position of the limb at

15%, 45%, and 75% of movement time was contrasted because these times corresponded

average stimulus mid-points for each of the presentation times (i.e., 0 ms: 15%; 100 ms:

45%; 200 ms: 75%). The 75% proportion also corresponds to the last comparison used in

Heath (2005) to assess the differences between trials with vision and trials with no-vision

of the limb. The analysis revealed a significant main effect of Vision, F(1,14) = 10.05, p

<.01, indicating that the no-vision trials (0.22 ±0.25) had significantly higher R2 values

than the vision trials (0.12 ±0.19). There was also a main effect of Proportion, F(2,28)

= 59.69, p <.001, HSD = 0.12, demonstrating that R2 values at 75% (0.40 ±0.24) were

significantly higher than the R2 for 15% (0.03 ±0.10) and 45% (0.08 ±0.13) of MT. Lastly,

there was a significant Vision by Proportion interaction, F(2,28) = 32.80, p <.001, HSD

= 0.08. Decomposing the interaction revealed that at 75% of the movement, participants

had significantly higher R2 values in the no-vision trials (0.52 ±0.17) compared to the

vision trials(0.27 ±0.23) (see Figure 2.5).


Table 2.1: Number of Perceived Flashes

Stimuli Presented Control 0 ms 100 ms 200 ms

vision no-vision vision no-vision vision no-vision vision no-vision

1F 1B 1.01(0.02) 1.03(0.09) 1.09(0.14) 1.11(0.19) 1.06(0.09) 1.16(0.21) 1.03(0.07) 1.09(0.13)

1F 2B 1.63(0.39) 1.64(0.34) 1.50(0.32) 1.55(0.27) 1.84(0.25) 1.72(0.23) 1.75(0.31) 1.66(0.28)

2F 1B 1.47(0.37) 1.07(0.11) 1.66(0.35) 1.17(0.20) 1.63(0.33) 1.15(0.18) 1.46(0.34) 1.12(0.16)

2F 2B 1.94(0.11) 1.11(0.19) 1.95(0.09) 1.60(0.21) 1.99(0.02) 1.76(0.20) 1.98(0.03) 1.70(0.26)

Number of perceived flashes (and standard deviation) for each presentation time and

experimental phase.


Figure 2.1: Depiction of participant sitting with the target position aligned with their

mid-sagittal plane, reaching from the home position to the target location. The arrows

approximately depict where the stimulus onset occurred during the reaching trajectory

(i.e., 0, 100, and 200 ms relative to movement onset).


Figure 2.2: Depiction of aiming console and stimuli. A participant’s point of view of the

aiming console, with depiction of the home position (switch) as well as the target (LED),

flash (LED), and beep (piezoelectric buzzer) stimuli locations.


Figure 2.3: Mean number of perceived flashes (and SEM bars) for the fusion (2 Flash,

1 Beep) illusion as a function of presentation time. In the vision condition, participants

perceived significantly more flashes in the 0 ms and 100 ms conditions. In the no-vision

trials, participants perceived fewer flashes overall, and performance remained stable over

the different presentation times.


Figure 2.4: Mean number of perceived flashes (and SEM bars) for the fission (1 Flash,

2 Beep) illusion as a function of presentation time. In both vision and no-vision trials,

participants were morse susceptible to the illusion in the 100 ms compared to the 0 ms

presentation time.


Figure 2.5: R2 values as a function of movement proportion (with SEM bars). This

analysis was used to examine the amount of online corrections occurring during vision

versus no-vision trials. Compared to trials with vision trials, no-vision trials exhibited

higher R2 values at 75% of movement time indicating that participants utilized a more

pre-planned control strategy.

Chapter 3

Discussion

The present study was designed to examine if engagement in visuomotor regulation pro-

cesses affects the perception of multisensory events during goal-directed actions. Specif-

ically, the purpose of the study was to test if the observed modulation in the perception

of the fusion audiovisual illusion also occurs in situations where participants employ a

pre-planned mode of control. To answer this question, participants were presented with

an audiovisual illusion as they performed upper-limb reaches in two vision conditions:

under normal lighting (vision), and in complete darkness without of their limb or the en-

vironment (no-vision). In the trials with vision, we expected to replicate the findings of

Tremblay and Nguyen (2010). That is, we expected to observe a decreased susceptibility

to the fusion illusion at high limb velocities (corresponding to the 100 ms presentation

time) compared to when the illusion was presented at rest and at low limb velocities (con-

trol, 0 ms, and 200 ms). Aiming movements performed with vision were also expected

to exhibit low endpoint errors and more online control as assessed by endpoint preci-

sion and R2 analyses. Conversely, trials performed without vision of the environment,

where participants cannot see their limb, should exhibit higher endpoint errors and a

more pre-planned mode of control. Importantly, if engagement in visuomotor regulation

is responsible for the decrease in susceptibility to the fusion illusion, we expected that

37

Chapter 3. Discussion 38

presentation time would exert no influence in the no-vision condition.

Manipulating vision of the reaching environment was chosen because previous work

has demonstrated that both limb and target vision are important for precision and con-

trol of rapid upper-limb reaches (Heath, 2005). Without vision of the limb, participants

exhibited more variable endpoints compared to movements where visual feedback about

the limb and the target was present. Furthermore, correlational analyses of the trajec-

tories obtained from movements without vision of the limb revealed more stereotyped

movement trajectories which is indicative of less online control. These results suggests

that vision of the limb is important for both engagement in visuomotor regulation and

endpoint precision. More importantly, these findings provide strong evidence that, when

vision of limb is withdrawn, participants adopted an aiming strategy that relies less on

visuomotor regulation.

3.1 Online Control with Vision of the Environment

and No-Vision of the Environment

Our initial hypotheses with regard to visuomotor regulation in the no-vision trials were

supported by the results obtained from the movement time, endpoint error, and move-

ment trajectory analyses. No differences in movement time were found between vision

and no-vision trials. Average movement times for both trial types were also within the

prescribed bandwidth. More importantly, no movement time differences between vision

conditions provides some additional evidence that presentation of the audiovisual stimuli

were comparable in terms of the proportion of movement time where they were presented

and thus provides some additional validity to the comparison made between proportions

in the R2 analyses.

Both the endpoint precision results and movement trajectory data also fit well with

what was predicted and with the extant literature. Heath (2005) examined the impact


of vision of the limb and target on aiming performance. The author hypothesized, in line

with previously mentioned models of limb control (e.g., Elliott et al., 2001), that vision of

the limb is used throughout the trajectory to provide the necessary input for trajectory

amendments. In trials where the limb was occluded the author noted that, in addition

to higher endpoint errors, movement trajectories appeared to be more stereotyped as

assessed by the R2 analyses. Based on this evidence, it was concluded that vision of the

limb was important for engagement in visuomotor regulation processes (Heath, 2005).

In the present study, both the key results of Heath (2005) were replicated. First, our

results indicated that trials without vision were less precise than trials performed with

vision. Also, compared to trials with vision, trials without vision were significantly more

stereotyped in their trajectories.

3.2 Perception of the Audiovisual Illusion and Limb

Velocity

One of the main findings of Tremblay and Nguyen (2010) was the link between limb

velocity, presentation time, and the modulation of the fusion illusion. Recall, the authors

reported that participants experienced the fusion illusion to a lesser extent when it was

presented at 50 ms and 100 ms compared to when the stimulus was presented at 0

ms and 200 ms relative to movement onset. The main purpose of the present study

was to determine if engagement in visuomotor regulation processes could explain this

relationship.

3.2.1 Illusion Perception in Trials with Vision

As expected, the present study replicated Tremblay and Nguyen (2010) in the vision

condition. Compared to the control and the 200 ms presentation time, participants were

less susceptible to the fusion when the associated stimuli was presented 0 ms and 100


ms relative to movement onset. A novel result, not previously observed in Tremblay

and Nguyen (2010), was the modulation of the fission illusion. Participants perceived a

greater number of illusory flashes during both the 100 ms and 200 ms presentation times

compared to the 0 ms presentation time. These results indicate that the relationship

between limb velocity and perception of the fission illusion is not comparable to the

relationship between limb velocity and the fusion illusion. Considering the differences in

the time course of activations associated with each of the illusions (Mishra et al., 2007;

2008), this result was not completely surprising. The possible reasons for these findings

are discussed more thoroughly in section 3.4.

3.2.2 Illusion Perception in Trials without Vision

In the no-vision condition, and in accordance with our hypotheses, there was no mod-

ulation of the fusion illusion at any presentation time. Similar to the vision condition,

a slight increase in susceptibility to the fission illusion was noted for the 100 ms pre-

sentation time compared to the 0 ms presentation time. Although not significant, there

was also a trend towards greater susceptibility for the fission illusion at 200 ms. The

perception of the fusion illusion was also not modulated in no-vision trials even though

the velocities were not different between vision conditions.

3.2.3 Relationship with Limb Velocity

In the present study, participants were less susceptible to the fusion illusion when it was

presented at 0 ms and 100 ms relative to of movement onset. Analysis of the limb velocity

at stimulus midpoint revealed a similar velocity-dependent modulation of perceptual

fusion as the phenomenon noted in Tremblay and Nguyen (2010). The limb velocities

at both the 0 ms and 100 ms presentation times were significantly higher than the 200

ms condition. At first, results at the 0 ms presentation time appear to be surprising as

a modulation at this time was not found in the previous study (Tremblay & Nguyen,


2010); however, upon further analysis it is evident that differences in the methodology

employed to detect movement onset may be the likely reason for this result. In the

present study, the microswitch required a movement of at least 1.5 mm of displacement

to detect movement onset while the motion capture system sampling at 500 Hz used in

Tremblay and Nguyen (2010) would identify movement start after the limb had travelled

more than 0.06 mm over 4 ms, for a minimum displacement of 0.12 mm. It was thus not

surprising that high limb velocities were observed at stimulus mid-point in both the 0 ms

and 100 ms presentation times. It is also not surprising that the perceptual experiences

of the fusion illusion did not differ between these presentation times in the vision trials.

Even though the average movement times are not reported in Tremblay and Nguyen

(2010), the limb velocities at 15% (the average proportion of the movement attained in

the present study) appear to be lower than what was found in the present study. Thus,

the discrepancy in the velocities observed at the 0 ms presentation time between the

present study and Tremblay and Nguyen (2010) is likely due to a combination of a later

trigger for movement start and overall faster average movement times.

3.3 The Influence of Visual Environment on Percep-

tion of Fusion and Fission Illusions

The presence of both the fusion and fission illusion were observed in the control conditions

(Shams et al., 2000; Andersen et al., 2004; Mishra et al., 2007, 2008). In control trials,

irrespective of visual feedback availability, participants’ perception of the number of

flashes was altered by the number of co-occurring beeps. According to the present data,

and in line with our hypotheses, participants experienced both the fission and fusion

illusions while at rest (see Table 3.1). Also, in accordance with previous findings, there

was some variability in the degree to which the illusion was experienced, with some

individuals being more susceptible than others (Mishra et al., 2007, 2008).


One peculiarity present in the control data is the difference in susceptibility to the

fusion illusion when it was presented in no-vision. Our results indicate, in the control

conditions, that participants were more susceptible to the fusion illusion in the no-vision

condition compared to the vision condition. While this result was not expected, the find-

ing is in agreement with other observations in the extant literature. In the investigation

done by Mishra et al. (2007, 2008), participants were exposed to the audiovisual illusion

in a low-light environment (with lighting measured at 2cd/m2). Recall, Mishra and col-

leagues conducted analyses where they separated groups based on their perception of the

fusion illusion. The SEE1 group was comprised of individuals who were more susceptible

to the illusion than participants in the SEE2 group. Analogous to the no-vision and vision

data in the present study, participants in SEE1 and SEE2 groups, despite differences in

fusion perception, were not different in their perceptions of the fission illusion. A poten-

tial explanation for this observation could be extrapolated by examining the behavioural

responses to the unimodal visual stimuli reported in Mishra et al. (2008). Specifically

examining the responses to the unimodal stimulus comprising of 2 visual flashes reveals

that participants in the SEE1 group were more likely to report 1 flash compared to those

in the SEE2 group. In simpler terms, it appears these participants were more likely to

“fuse” flashes even if the flashes were not presented with an accompanying beep. Such

findings are not completely unexpected, as early research on the topic of visual percep-

tion in response to dark adaptation provides convergent evidence for both our results

and the results noted in Mishra et al. (2008). An early study done by Federov and

Mkrticheva (1938) examined how critical fusion frequency (i.e., the frequency that visual

flashes appear as one continuous light) changes as a function of light and dark adaptation.

As the authors predicted, adaptation to darkness decreased the critical fusion frequency

(indicating individuals were more likely to fuse flickering lights at a lower frequency in

darkness). The researchers also went one step further by injecting strychnine to prevent

the pupil from dilating in response to a dark environment. Indeed, without the pupil-


lary response, the critical fusion frequency remained constant in both light and darkness.

Dark adaptation has also been shown to affect double-flash discrimination. The double-

flash discrimination paradigm is a task wherein the lowest threshold for the detection of

two flashes is determined; in essence, the stimuli presented in double flash discrimination

is the same as the 2 flash unimodal visual stimuli presented to participants by Mishra

et al. (2008). When examining how darkness affects double discrimination thresholds,

Skrandies (1985) noted that thresholds increased as the ambient light decreased. This

finding suggests that, in darkness individuals have a harder time detecting two visual

events (Keller, 1967; Boynton, 1972; Mahneke, 1958). In sum, the perceptual threshold

for perception of two flashes appears to be higher in low-light environments, even at

the early stages of adaptation. In the present study, we attempted to control for dark

adaptation by turning on the lights every 5 minutes (roughly 40 trials); however, it is

possible, considering the results of Fedorov & Mkrticheva, 1938, (p750, Figure 1) that

this time period might have been too long to fully negate the effects of dark adaptation

on perception. At first, this was not considered a major issue as similar rates of fusion

susceptibility have been reported in other studies (Mishra et al., 2007, 2008; Tremblay

& Nguyen, 2010).

3.4 Explaining the Modulation of the Audiovisual Il-

lusion: The Cautious Case for Visuomotor Reg-

ulation Processes

In Tremblay and Nguyen (2010), the authors hypothesized that the observed decrease

in susceptibility to the fusion illusion could be related to either the sensory gating of

auditory information, or a better signal-to-noise ratio due to the contrast of the limb on

the retina. The first hypothesis proposed by Tremblay and Nguyen (2010) was based on


the previously discussed sensory gating phenomenon (Chapman et al., 1987). According

to the authors, the noted modulation in audiovisual perception could have occurred as

a result of auditory suppression during goal-directed action. This assertion was tested

in a recent experiment (Tremblay, Wong, & Manson, 2012). The main results of this

study suggested that gating of auditory information does take place during goal-directed

action; however, the effect is not linked with limb velocity (see Figure 1.2). Thus, auditory

gating alone does not seem to be a sufficient explanation for the alterations in audiovisual

perception during action.

The second, and more speculative, hypothesis forwarded by Tremblay and Nguyen

(2010) was the reduction in signal-to-noise ratio due to the retinal contrast of limb po-

sitions as the limb is moving at high velocities. The results of the present study do not

appear to lend support to this hypothesis. In the present experiment, the no-vision trials

represent a condition wherein there is no representation of limb position on the retina.

In these trials, the perception of the secondary audiovisual stimuli was altered as a func-

tion of movement phase, at least in the fission condition. Also, further analyses indicate

that modulation of the fusion illusion may occur to some extent in no-vision trials (see

appendix D).

As stated previously, visual information about both limb and target positions has been

deemed important for the planning and online control of reaching movements (Elliott et

al., 2010, 2001; Heath, 2005; Carlton, 1981). The use of online visual information has

also been linked to attention and motor planning. When participants are made aware

of the vision condition of a reaching movement prior to execution, participants adopt a

strategy reflective of the sensory conditions of the upcoming trial (Hansen, Glazebrook,

Anson, Weeks, & Elliott, 2006). Thus, when participants know vision will be available,

the evidence suggests that participants plan to use visual information to engage in visuo-

motor regulation. Traditionally, the visual information most important for visuomotor

regulation was thought to be available late in aiming trajectories (Beaubaton & Hay,


1986); however, this view began to change as corrections occurring late in trajectories

were found to be based on visual information obtained earlier (Khan & Franks, 2003).

Furthermore, it was proposed that this early visual information may be linked to limb

velocity (Tremblay et al., 2013). Investigations into this link have demonstrated that

providing vision at high limb velocities, even if these windows are small and occur rela-

tively early in the trajectory (i.e., before peak velocity), facilitates both accurate aiming

movements and engagement in visuomotor regulation processes (Hansen, 2010; Tremblay

et al., 2013).

The research presented above indicates that individuals plan to use visual information

when they know that visual information will be available; also, visual feedback obtained

from high velocity portions of limb movements is proven to be important for visuomotor

regulation processes. Thus, it is possible that, in the vision trials, participants up-

regulated their visual information processing to facilitate engagement in the visuomotor

regulation processes necessary to complete their movements as accurately as possible.

With reference to multisensory processing, this presumed shift toward greater visuomotor

regulation could mean enhanced reliance on visuomotor networks. The idea that this

shift is velocity based also has some merit as limb velocity is a parameter encoded by

visuomotor neurons in humans and non-human primates (Jerbi et al., 2007; Ashe &

Georgopoulos, 1994). Furthermore, while the most common neural circuits associated

with visuomotor regulation have been localized in the posterior parietal cortex, recent

evidence is starting to suggest a greater role for primary visual processing areas as well.

In a recent study examining neural activation during reaches with and without vision,

functional magnetic resonance imaging revealed greater activations in primary visual

processing areas in reaches with visual feedback of the hand as compared to reaches with

no-visual feedback of the hand position (Filimon, Nelson, Huang, & Sereno, 2009). Taking

into consideration that the fusion illusion is characterized by a reduction in processing

in primary visual areas and a quick shift to more multisensory processing areas (Mishra


et al., 2008), the possibility that the visuomotor regulation requirements of the primary

task may damper this change could be an alternative mechanistic explanation to the

retinal contrast hypothesis hypothesis presented in Tremblay and Nguyen (2010).

To delve into the nature of this up-regulation of vision and how exactly this pro-

cess occurs would be very speculative, and investigation into the possible mechanisms

is beyond the scope of this discussion. Furthermore, according to additional analyses

(see appendix D), the role of visuomotor regulation processes may not be a sufficient

explanation for the modulation of the audiovisual perception.

3.5 Limitations to the Role of Visuomotor Regula-

tion Processes

Although theoretically sound, the claim that visuomotor regulation processes are respon-

sible for the alterations in audiovisual perception is not fully supported by the present

dataset. As mentioned above, the fission illusion was modulated during goal-directed

action regardless of vision condition. Also, because the illusion was stronger in dark

environments, perhaps the comparisons made between conditions was not reflective of

modulation that could be occurring in no-vision. Both of the aforesaid limitations are

discussed in detail below.

3.5.1 Modulation of both Illusions in No-Vision

Contrary to our predictions, susceptibility to the fission illusion was modulated by move-

ment phase and similar patterns of the observed modulation occurred in both vision

conditions. In the present study, susceptibility to the fission illusion increased in the 100

ms compared to the 0 ms presentation time. This result is not easily accounted for by the

visuomotor regulation hypothesis as it was presented. As stated previously, the visuo-

motor regulation hypothesis suggests that participants prepare to use visual information


at high velocities for online corrections (Hansen et al., 2006; Tremblay & Nguyen, 2010;

Elliott et al., 2010). Furthermore, to engage in these visuomotor regulation processes,

participants likely employ visuomotor processing networks (Jerbi et al., 2007; Filimon et

al., 2009). Unlike the fusion illusion, the mechanisms for the fission illusion are defined

by an up-regulation in visual processing and greater audiovisual interactions manifested

in both multisensory and auditory processing areas. The timing associated with fission,

and also the lack of differences between vision conditions, makes it unlikely this modula-

tion is due to engagement in visuomotor regulation. Recall, Mishra et al. (2007) noted

the perception of the extra flash illusion is associated with later activity and possibly

interactions between auditory and visual cortex occurring after the presentation of the

second auditory stimulus.

One hypothesis to explain this result could be taken from the data in Tremblay et

al. (2012). Because auditory information is gated during action, the recovery of audi-

tory processing as the movement ends could contribute to increased susceptibility of the

illusion after the second stimulus is presented. However, there is very little behavioural

or neurophysiological evidence to justify this claim. Another possible hypothesis to ex-

plain this finding could be drawn from Filimon et al. (2009). Recall, these authors

were interested in observing brain activity (as assessed by fMRI) in response to reaches

performed in different sensory conditions. Though the authors observed increased acti-

vation in known visual and visuomotor areas (e.g., parietal occipital sulcus); the authors

also noted that other visuomotor areas were as active in both tasks (e.g., anterior pre-

cuneus and superior parietal cortex when participants were reaching for a visual target

with feedback compared to no feedback). Both anterior precuneus and superior parietal

cortex have been previously associated with visual and proprioceptive movement control

(Filimon et al., 2009; Wenderoth, Debaere, Sunaert, & Swinnen, 2005; Buneo & An-

dersen, 2006; Desmurget et al., 1999). Thus, it is evident that both early visual and

proprioceptive processing contributes to visuomotor regulation processes, at least at the


neural level. Based on these data, we can hypothesize that some visuomotor regulation

also occurred in the no-vision trials in the present study. Perhaps, a likely explanation

for the modulation in the fission illusion could be a revised version of the visuomotor

regulation hypothesis wherein there is combination of an early shift toward more visuo-

proprioceptive sensory processing. Thus, if the illusion is presented early, as it was in the

0 ms condition in the present study, the combination of shifts toward visuo-proprioceptive

processing networks and the gating of auditory stimuli may decrease the initial influence

of the auditory component of the multisensory stimuli (Filimon et al., 2009; Tremblay

et al., 2012). However, if the illusion is presented later, as in the 100 ms or 200 ms

presentation time, there may be less influence of these visuo-proprioceptive networks and

also the recovery of auditory processing as the movement ends. If this is true, one would

predict participants would experience the illusory flash to a greater extent at this point.

The latter portion of this hypothesis requires further investigation into the time course

associated with the recovery of auditory perception to baseline levels after a goal-directed

task.

3.5.2 Ceiling Effects in No-Vision Trials

As stated above, even in cases where there was no-vision of the environment, evidence

suggests that there are still significant visuomotor regulation processes taking place,

especially in cases where a visually-guided movement is being performed (Filimon et al.,

2009). If visuomotor regulation processes can explain the modulation observed in the the

present study, then we would have expected to observe a modulation of the fusion illusion

in the no-vision trials. The results of our analyses did not allude to the presence of such

an alteration. However, there may be reasons to investigate this question further. Recall

that in the present study there was a discrepancy in the initial perception of the fusion

illusion between vision and no-vision trials. In trials where vision of the environment

was withdrawn, participants were more susceptible to the audiovisual fusion illusion. As


stated above, this discrepancy may have been a result of rapid perceptual alterations

due to pupillary dilation. To better evaluate if perception of the illusion was altered

in no-vision trials, and to make the vision conditions more comparable, an additional

z-score analysis was conducted to normalize the data (see Appendix D for a complete

description and results).

Results from this additional analysis indicated that there could have been a modula-

tion of the fusion illusion in the no-vision trials. If we convert the values to standardized

scores, the pattern of results is the same for both vision conditions. Therefore it could be

the case that these early shifts toward visuomotor processing and away from multisensory

areas could explain the changes in susceptibility to both illusions. For example, in the

no-vision trials, because it is still a visually-guided goal-directed movement, participants

may be trying to utilize visual information from location of the target and propriocep-

tive information about the limb position for visuomotor regulation instead of adopting

a pre-planned mode of control. However, it is unclear how sensitive the R2 analysis is

to distinguishing between strategies that rely on early adjustments and a pre-planned

mode of control. It is because of this reason, a shift toward an altered control strategy

cannot be fully ruled out as an explanation for results obtained from the present study.

However, if the z-score analysis were to deemed more reliable, discussion of the results

would centre around visual up-regulation to facilitate early visual feedback use in the case

of a vision trial, and visuo-proprioceptive control in both vision conditions(Tremblay &

Nguyen, 2010; Hansen, 2010; Filimon et al., 2009).

3.6 Conclusions

The purpose of this thesis was to test if engagement in visuomotor regulation processes

could explain the previously observed modulation in audiovisual perception during goal-

directed action. During rapid upper-limb reaching movements, participants were pre-


sented with one of four audiovisual stimuli and asked to report the number of visual

stimuli they perceived. To encourage participants to adopt a more pre-planned mode of

control, vision of the environment was also manipulated.

As expected, in conditions where visual information was available, analyses of the

movement trajectories and endpoint precision revealed that participants were indeed

engaging in normal visuomotor regulation. In darkness, participants exhibited more

stereotyped trajectories and lower endpoint precision, both of which are associated with

decreased visuomotor regulation.

With regard to audiovisual perception, when vision of the environment was available,

perception of both fusion and fission illusions was influenced by action. The fusion results

replicated previous findings while the fission illusion observations were novel. Conversely,

in the no-vision condition, initial analyses indicated that perception of the fusion illusion

was not modulated by action; however, for the fission illusion, the pattern of results were

similar to those observed when the illusion was presented with vision. These findings were

in accordance with the hypotheses that engagement in visuomotor regulation processes

does alter the influence of the audiovisual illusion. These results, however, were slightly

confounded. It was also observed that susceptibility to the fusion illusion at rest was

exacerbated when vision of the environment was unavailable, a phenomenon that could

be attributed to pupillary changes as a result of dark adaptation. This led to a subsequent

normalization procedure to better evaluate the performance in the no-vision condition.

The analyses revealed that indeed, when equated, susceptibility to the fusion illusion was

perhaps modulated by action in both vision and no-vision. Altogether, engagement in

visuomotor regulation processes may not fully be responsible for the modulation of the

fusion illusion during action. Rather, the modulation may emerge as a result of early

visual up-regulation processes that are a result of engaging in a visually guided task.


Table 3.1: Susceptibility to the Audiovisual Illusion

Stimuli Presented Control 0 ms 100 ms 200 ms

vision no-vision vision no-vision vision no-vision vision no-vision

1F 1B 1(2) 3(9) 9(14) 11(18) 7(9) 14(21) 3(7) 7(13)

1F 2B 63(39) 69(34) 50(31) 56(27) 84(25) 75(23) 75(31) 68(27)

2F 1B 61(37) 93(11) 34(35) 83(20) 38(32) 86(18) 54(34) 89(16)

2F 2B 6(11) 27(27) 4(9) 37(21) 1(2) 20(19) 2(3) 24(25)

Susceptibility is expressed as a percentage (and standard deviation) of trials where an illusion

was perceived.

References

Alais, D., Newell, F. N., & Mamassian, P. (2010). Multisensory processing in review:

From physiology to behaviour. Seeing and Perceiving , 23 (1), 3–38.

Andersen, T. S., Tiippana, K., & Sams, M. (2004). Factors influencing audiovisual fission

and fusion illusions. Cognitive Brain Research, 21 (3), 301–308.

Ashe, J., & Georgopoulos, A. P. (1994). Movement parameters and neural activity in

motor cortex and area 5. Cerebral Cortex , 4 (6), 590–600.

Beaubaton, D., & Hay, L. (1986). Contribution of visual information to feedforward and

feedback processes in rapid pointing movements. Human Movement Science, 5 (1),

19–34.

Boynton, R. M. (1972). Discrimination of homogeneous double pulses of light. In

D. Jameson & L. M. Hurvich (Eds.), Visual psychophysics (Vol. 4, pp. 202–232).

Berlin: Springer-Verlag.

Bridgeman, B., Hendry, D., & Stark, L. (1975). Failure to detect displacement of the

visual world during saccadic eye movements. Vision research, 15 (6), 719–722.

Buneo, C. A., & Andersen, R. A. (2006). The posterior parietal cortex: Sensorimotor

interface for the planning and online control of visually guided movements. Neu-

ropsychologia, 44 (13), 2594–2606.

Carlton, L. G. (1981). Visual information: The control of aiming movements. The

Quarterly Journal of Experimental Psychology , 33 (1), 87–93.

Chapman, C., Bushnell, M., Miron, D., Duncan, G., & Lund, J. (1987). Sensory percep-

52

References 53

tion during movement in man. Experimental Brain Research, 68 (3), 516–524.

Crawford, J. D., Medendorp, W. P., & Marotta, J. J. (2004). Spatial transformations

for eye–hand coordination. Journal of Neurophysiology , 92 (1), 10–19.

Desmurget, M., Epstein, C., Turner, R., Prablanc, C., Alexander, G., & Grafton, S.

(1999). Role of the posterior parietal cortex in updating reaching movements to a

visual target. Nature Neuroscience, 2 (6), 563–567.

Elliott, D., Hansen, S., Grierson, L. E., Lyons, J., Bennett, S. J., & Hayes, S. J. (2010).

Goal-directed aiming: Two components but multiple processes. Psychological bul-

letin, 136 (6), 1023.

Elliott, D., Helsen, W. F., & Chua, R. (2001). A century later: Woodworth’s (1899)

two-component model of goal-directed aiming. Psychological Bulletin, 127 (3), 342–

357.

Ernst, M. O., & Bulthoff, H. H. (2004). Merging the senses into a robust percept. Trends

in Cognitive Sciences , 8 (4), 162–169.

Fedorov, N., & Mkrticheva, L. (1938). Mechanism of light flicker fusion during the course

of dark and light adaptation. Nature, 142 , 750–751.

Filimon, F., Nelson, J. D., Huang, R.-S., & Sereno, M. I. (2009). Multiple parietal reach

regions in humans: cortical representations for visual and proprioceptive feedback

during on-line reaching. The Journal of Neuroscience, 29 (9), 2961–2971.

Ghazanfar, A. A., & Schroeder, C. E. (2006). Is neocortex essentially multisensory?

Trends in Cognitive Sciences , 10 (6), 278-285.

Ghez, C., & Lenzi, G. (1971). Modulation of sensory transmission in cat lemniscal system

during voluntary movement. European Journal of Physiology , 323 (3), 273–278.

Goodale, M. A., & Milner, A. D. (1992). Separate visual pathways for perception and

action. Trends in Neurosciences , 15 (1), 20–25.

Hansen, S. (2010). Determining the temporal limits of a visual sample for visual regula-

tion. Journal of Motor Behavior , 42 (2), 107–110.

References 54

Hansen, S., Glazebrook, C. M., Anson, J. G., Weeks, D. J., & Elliott, D. (2006). The influ-

ence of advance information about target location and visual feedback on movement

planning and execution. Canadian Journal of Experimental Psychology , 60 (3), 200-

208.

Heath, M. (2005). Role of limb and target vision in the online control of memory-guided

reaches. Motor Control , 9 (3), 281-311.

Heath, M., Westwood, D. A., & Binsted, G. (2004). The control of memory-guided

reaching movements in peripersonal space. Motor Control , 8 (1), 76-106.

Howard, I. P., & Templeton, W. B. (1966). Human spatial orientation. Oxford, England:

Wiley and Sons.

Innes-Brown, H., Barutchu, A., Shivdasani, M. N., Crewther, D. P., Grayden, D. B.,

& Paolini, A. G. (2011). Susceptibility to the flash-beep illusion is increased in

children compared to adults. Developmental Science, 14 (5), 1089–1099.

Jerbi, K., Lachaux, J.-P., Karim, N., Pantazis, D., Leahy, R. M., Garnero, L., et al.

(2007). Coherent neural representation of hand speed in humans revealed by meg

imaging. Proceedings of the National Academy of Sciences , 104 (18), 7676–7681.

Juravle, G., Deubel, H., Tan, H. Z., & Spence, C. (2010). Changes in tactile sensitivity

over the time-course of a goal-directed movement. Behavioural Brain Research,

208 (2), 391–401.

Keller, M. (1967). Two-flash thresholds as a function of luminance in the dark-adapted

eye. Journal of Experimental Psychology .

Khan, M. A., & Franks, I. M. (2003). Online versus offline processing of visual feedback

in the production of component submovements. Journal of Motor Behavior , 35 (3),

285–295.

Klemen, J., & Chambers, C. D. (2012). Current perspectives and methods in study-

ing neural mechanisms of multisensory interactions. Neuroscience & Biobehavioral

Reviews , 36 (1), 111–133.

References 55

Mahneke, A. (1958). Foveal discrimination measured with two successive light flashes.

Acta Ophthalmologica, 36 (1), 3–11.

Marco, E. J., Hinkley, L. B., Hill, S. S., & Nagarajan, S. S. (2011). Sensory processing in

autism: a review of neurophysiologic findings. Pediatric Research, 69 , 48R–54R.

McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264 ,

746–748.

Medendorp, W. P. (2011). Spatial constancy mechanisms in motor control. Philosophical

Transactions of the Royal Society: Biological Sciences , 366 (1564), 476-491.

Miles, W. R. (1930). Ocular dominance in human adults. The Journal of General

Psychology , 3 (3), 412–430.

Mishra, J., Martinez, A., & Hillyard, S. A. (2008). Cortical processes underlying sound-

induced flash fusion. Brain Research, 1242 , 102–115.

Mishra, J., Martinez, A., Sejnowski, T. J., & Hillyard, S. A. (2007). Early cross-modal

interactions in auditory and visual cortex underlie a sound-induced visual illusion.

The Journal of Neuroscience, 27 (15), 4120–4131.

Murray, M. M., & Wallace, M. T. (Eds.). (2012). The neural bases of multisensory

processes. Boca Raton, FL: CRC Press.

Oldfield, R. C. (1971). The assessment and analysis of handedness: the edinburgh

inventory. Neuropsychologia, 9 (1), 97–113.

Prinz, W. (1997). Perception and action planning. European Journal of Cognitive

Psychology , 9 (2), 129–154.

Ramos-Estebanez, C., Merabet, L. B., Machii, K., Fregni, F., Thut, G., Wagner, T. A.,

et al. (2007). Visual phosphene perception modulated by subthreshold crossmodal

sensory stimulation. The Journal of Neuroscience, 27 (15), 4178–4181.

Sabes, P. N. (2011). Sensory integration for reaching: models of optimality in the context

of behavior and the underlying neural circuits. Progress in Brain Research, 191 ,

195-209.

References 56

Seki, K., Perlmutter, S. I., & Fetz, E. E. (2003). Sensory input to primate spinal cord is

presynaptically inhibited during voluntary movement. Nature Neuroscience, 6 (12),

1309–1316.

Shams, L., Kamitani, Y., & Shimojo, S. (2000). What you see is what you hear. Nature,

408 , 708.

Shams, L., Kamitani, Y., Thompson, S., & Shimojo, S. (2001). Sound alters visual

evoked potentials in humans. Neuroreport , 12 (17), 3849–3852.

Skrandies, W. (1985). Critical flicker fusion and double flash discrimination in different

parts of the visual field. International Journal of Neuroscience, 25 (3-4), 225–231.

Stein, B. E., & Meredith, M. A. (1990). Multisensory integration. Annals of the New

York Academy of Sciences , 608 (1), 51–70.

Stein, B. E., & Meredith, M. A. (1993). The merging of the senses. Cambridge, MA:

The MIT Press.

Stein, B. E., & Stanford, T. R. (2008). Multisensory integration: current issues from the

perspective of the single neuron. Nature Reviews Neuroscience, 9 (4), 255–266.

Todd, J. W. (1912). Reenforcement and inhibition (J. W. Todd, Ed.) (No. 25). The

Science Press.

Tremblay, L., Hansen, S., Kennedy, A., & Cheng, D. T. (2013). The utility of vision

during action: Multiple visuomotor processes? Journal of Motor Behavior , 45 (2),

91–99.

Tremblay, L., & Nguyen, T. (2010). Real-time decreased sensitivity to an audio-visual

illusion during goal-directed reaching. PloS One, 5 (1), e8592.

Tremblay, L., Wong, J., & Manson, G. (2012). Auditory gating during visually-guided

action? Seeing and Perceiving , 25 (s1), 106.

Wang, Y., Celebrini, S., Trotter, Y., & Barone, P. (2008). Visuo-auditory interactions in

the primary visual cortex of the behaving monkey: electrophysiological evidence.

BMC Neuroscience, 9 (1), 79.

References 57

Watkins, S., Shams, L., Josephs, O., & Rees, G. (2007). Activity in human v1 follows

multisensory perception. Neuroimage, 37 (2), 572–578.

Wenderoth, N., Debaere, F., Sunaert, S., & Swinnen, S. P. (2005). The role of anterior

cingulate cortex and precuneus in the coordination of motor behaviour. European

Journal of Neuroscience, 22 (1), 235–246.

Woodworth, R. S. (1899). The accuracy of voluntary movement. Psychological Review ,

3 , 1-119.

Zhang, N., & Chen, W. (2006). A dynamic fmri study of illusory double-flash effect on

human visual cortex. Experimental Brain Research, 172 (1), 57–66.

Appendices

58

Brief Neurological Questionnaire How often do you experience the following? Headaches Never Seldom Often Light-headed or dizziness Never Seldom Often Numbness or tingling Never Seldom Often Tremor Never Seldom Often Paralysis Never Seldom Often Convulsions or seizures Never Seldom Often Stroke Never Seldom Often Sensory impairment Never Seldom Often [To be considered neurologically intact, participants cannot select more than one “often” box in the first four categories and must tick “never” in the last four categories.]

Appendix A: Neurological Questionnaire

Handedness Test

Hand dominance test (adapted from Oldfield, 1971) Please indicate which hand you would use for the following activities: Writing right left Throwing right left Scissors right left Toothbrush right left Drawing right left [Participants answering right to 4 items or more are deemed to be right hand dominant.]

Oldfield, R.C. 1971. The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia, 9, 97-113.

Appendix B: Handedness Questionnaire

Eye Dominance Test To perform the Miles test (1930), participants will be asked to extend both arms in front of themselves. They are then asked to bring both hands together to create a small opening and then view a distant object through the opening. The experimenter will then ask the participant to close right eye. If the viewed the object is no longer visible, the participant will be deemed to be right-eye dominant.

Miles, W.R. (1930). Ocular dominance in human adults. The Journal of General Psychology, 3, 412-430.

Appendix C: Eyedness Assessment

62

Appendix D: Supplementary Analysis of Normalized

Perceived Flashes in Illusory Trials

The following analysis was conducted to better compare data between the two types of

vision trials in the present study. The analysis focused on comparing the differences

between the 2 types of illusion conditions (fission: 1F, 2B, fusion: 2F, 1B) across the

4 presentation times (control, 0 ms, 100 ms, 200 ms). Within each vision condition,

and within each type of illusion, a population Z-score for the number of perceived flashes

(flashz) was computed. These scores were then submitted to a 2 Vision (vision, no-vision)

by 4 Presentation Time (control, 0 ms, 100 ms, 200 ms) by 2 Illusion (fusion, fission)

repeated-measures ANOVA.

Results

Analysis revealed a significant effect of Presentation Time, F(3,42) = 10.1, p <.001 HSD

= 0.37. Post-hoc comparisons revealed that participants had a higher flashz in the 0

ms (0.39 ±0.99) presentation time compared to all other presentation times (rest: -

0.16 ±1.02, 100 ms: -0.11 ±0.92, 200 ms: -0.13 ±0.93). The analysis also yielded a

significant interaction between Presentation Time and Illusion, F(3, 42) = 4.85, p <.01,

HSD = 0.50. Post-hoc tests revealed that, similar to the main ANOVA results, for the

fission illusion, there were significantly lower flashz in the 100 ms presentation time(-0.38

±0.77 ) compared to the 0 ms (0.46 ±0.92) and the 200 ms (0.12 ±0.94) presentation

times. Within the fusion condition, a modulation of the illusions was noted as post-hoc

decomposing the interaction for fission trials revealed a significantly higher flashz for the

0 ms (0.33 ±1.1) and 100 ms (0.168 ±0.99) presentation times compared to the control

(-0.35 ±0.84) condition.

63

Modulation of the Fusion Illusion in No-Vision

The results of the Z-score analysis are plotted in Figure: 4. Overall, the results of the

Z-score analysis indicates that there appears to be the same perceptual modulation of

audiovisual illusion in both vision and no-vision trials. This provides some opposing

evidence to the hypothesis that engagement in visuomotor regulation processes is re-

sponsible for the observed modulation in the perception of the fusion illusion. These

data also make a convincing case for methodological changes to the present experiment

as these alterations could have been hidden by the effects of dark adaptation which likely

cause differences in the control condition.

64

Figure 4: Normalized perceived flashes ([flashz] and SEM bars) for the fusion illusion

plotted as a function of presentation time. In both vision conditions participants exhibit

a higher relative flashz at the 0 ms and 100 ms conditions compared to control.

the role of visuomotor regulation processes on perceived ... · graduate department of exercise...

Documents