classicthesis.pdf

8/13/2019 ClassicThesis.pdf

1/116

fiammetta ghedini

T H E I L L U S I O N O F A M B I G U I T Y: F R O MB I S TA B L E P E R C E P T I O N TO

A N T H R O P O M O R P H I S M


2/116


3/116

T H E I L L U S I O N O F A M B I G U I T Y: F R O M B I S TA B L EP E R C E P TI O N T O A N T H R O P OM O R P H I S M

fiammetta ghedini

A dissertation submitted in partial fullment of the requirements forthe degree of Doctor in Philosophy in Innovative Technologies of

Information and Communication Engineering and Robotics

30 May 2011


4/116

Fiammetta Ghedini: The Illusion of Ambiguity: from Bistable Per-ception to Anthropomorphism, A dissertation submitted in par-tial fullment of the requirements for the degree of Doctorin Philosophy in Innovative Technologies of Information andCommunication Engineering and Robotics, 30 May 2011


5/116

No lesson of psychology is perhaps more important for thehistorian to absorb than this multiplicity of layers, the peaceful

coexistence in man of incompatible attitudes.

Sir Ernest Gombrich

There is an universal tendency among mankind to conceive all beings like themselves, and to transfer to every object, thosequalities, with which they are familiarly acquainted, and of

which they are intimately conscious.

David Hume


6/116


7/116

A B S T R A C T

This thesis is a multidisciplinary work at the merging of neuro-science, art and human interaction with technologies. Its objec-tives are:

1. Review the literature and further investigate, by means of a brain imaging study, which are the mechanisms allow-ing the illusion of ambiguity in the brain, by proposing aframework of analysis based on different levels of ambigu-ity;

2. Discuss the concept of the illusion of life, dened as a per-ceptual phenomenon included into the wider category of ambiguity and caused by intentionality and animacy beinghard-wired in the brain (part of that previous knowledgenecessary to successfully process sensory inputs);

3. Explore features in behaviour and form which are mostlikely to trigger anthropomorphism, drawing insights fromart, technological applications and cognitive sciences, andfocusing on human interaction with articial creatures.

vii


8/116


9/116


10/116


11/116

A C K N O W L E D G M E N T S

This dissertation would not have been possible without the sup-port and encouragement of Professor Massimo Bergamasco, myPh.D. supervisor at Perceptual Robotics Laboratory of ScuolaSuperiore SantAnna. Throughout my thesis he provided guid-ance, advices and inspiration. I would like to thanks all thecolleagues with whom I had the pleasure of collaborating: CarloAlberto Avizzano, Franco Tecchia, Antonio Frisoli, Chiara Evan-gelista, Elisabetta Sani, Federico Vanni, Walter Aprile, AndreaBizdideanu, Vittorio Spina, Davide Vercelli, Rosario Leonardi,Paolo Tripicchio, Paolo Gasparello, Emanuele Giorgi; VittorioLippi and Emanuele Ruffaldi for their precious help with LateXand those other colleagues whom I have neglected to mention.A special thanks to Haakon Faste for his inspiring enthusiasmand eclectic skills, Marcello Carrozzino for providing usefulfeedback and to Francesca Farinelli and Alessandra Scucces for being not only helpful colleagues but also very dear friends,and for supporting me during my thesis-writing period. Mygratefulness goes also to all the people working in the adminis-tration of the Scuola SantAnna and especially Laura Bevacqua.

For enlightening conversations, encouragement, feedback andinspiration I would like to thank Israel Roseneld. I owe a lot tohim and to his perspectives on neuroscience. I also would liketo thank Simon Penny and Bill Vorn for visiting our Laboratoryand inspiring me with their artworks on articial life. My deep-est gratefulness goes to Professor Semir Zeki who made possiblethe fMRI experiment described in this thesis. Thank you for yourguidance and for working with me for over one year at the Well-come Lab of Neurobiology at UCL, London. I am indebted to allthe people working in Professor Zekis lab: Barbara Nordhjem

for her continuous support, John Romaya for his experience, JonStutters for his skills, Shelley Tootell for her helpfulness andTomohiro Ishizu and Sam Cheadle for being always supportive,through difcult times too. Thanks for you friendship and kind-ness. Thanks to all the people from the academic world who inconferences, seminars and workshops provided new inspiringideas, feedback on my research and good company: Mel Slater,Mavi Sanchez, Doron Friedman, Peggy Weil, Nonny de la Pena,Marcelo Wanderlay, Elena Pasquinelli, Benoit Bardy and many

xi


12/116

others. A very special thanks goes to Tommaso Andreussi: youwill print your next book on Luoyang paper! Finally, "merci"to Franois Pachet for his encouragement, for being a modelof what a researcher should be, and most importantly for com-ing into my life. And "grazie" to my parents, Anna Casanovaand Fabio Ghedini, because they taught me the importance of knowledge: to them I dedicate my thesis.

xii


13/116

C O N T E N T S

i introduction 111 in troduct ion 13

1.1 Perception: Making Sense of the Senses . . . . . . 141.1.1 Watching without seeing . . . . . . . . . . . 151.1.2 The Brain: an Abstraction Machine . . . . . 171.1.3 The Case of Colour Vision . . . . . . . . . . 19

1.2 Concepts and Categories in the Brain . . . . . . . 221.3 Seeing through Illusions . . . . . . . . . . . . . . . 24

1.3.1 Two Different Categories of Illusions . . . . 241.3.2 Ambiguity and Ambiguities . . . . . . . . . 27

ii f irs t and second level ambiguity 312 percept ion of b is table ambigui ty 33

2.1 What is multistable ambiguity . . . . . . . . . . . . 332.1.1 Neural processes underlying multistable

phenomena . . . . . . . . . . . . . . . . . . 362.1.2 Attention and perception of bistable gures 412.1.3 Levels of ambiguity . . . . . . . . . . . . . . 42

2.2 An Experiment on Two-Levels Bistable Ambiguity 422.2.1 General fMRI Analysis . . . . . . . . . . . . 46

2.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . 482.3.1 Behavioural results . . . . . . . . . . . . . . 48

2.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . 502.4.1 Conclusion . . . . . . . . . . . . . . . . . . . 53

iii third level ambiguity 613 th ird level ambigui ty : the i l lusion of l ife 63

3.1 Perceptual knowledge of life . . . . . . . . . . . . . 633.2 The illusion of intentionality . . . . . . . . . . . . . 663.3 The illusion of life in the brain . . . . . . . . . . . 69

3.3.1 Emotional clues in the illusion of life . . . 72

iv forth level ambiguity 754 aesthetics of anthropomorphism 77

4.0.2 Anthropomorphism as a forth level ambi-guity . . . . . . . . . . . . . . . . . . . . . . 77

4.0.3 Variability in anthropomorphisation . . . . 774.1 Life evocation in the arts . . . . . . . . . . . . . . . 78

4.1.1 Dening life away . . . . . . . . . . . . . . . 78

xiii


14/116

xiv contents

4.1.2 Art and the illusion of life . . . . . . . . . . 794.1.3 From art to technology . . . . . . . . . . . . 834.1.4 Uncanniness as a result of life evocation . . 84

4.2 The illusion of life in articial creatures: featuresof believabil i ty . . . . . . . . . . . . . . . . . . . . . 854.2.1 Form: is realism a necessity? . . . . . . . . 86

4.3 Conclusions . . . . . . . . . . . . . . . . . . . . . . 90

bibliography 95


15/116

L I S T O F F I G U R E S

Figure 1.1 The response of an orientation selective cell 18Figure 1.2 The response of an orientation selective

cell in V5 . . . . . . . . . . . . . . . . . . . . 19Figure 1.3 A rotation of a mask: the forth image shows

the inside of the mask, appearing convexeven if it is hollow . . . . . . . . . . . . . . 25

Figure 1.4 Model of perception, based on (Gregory2009 ) . . . . . . . . . . . . . . . . . . . . . . 27

Figure 1.5 Different kinds of ambiguities . . . . . . . . 29Figure 2.1 a) Necker Cube - b) Rubin Vase . . . . . . . 34Figure 2.2 Binocular rivalry . . . . . . . . . . . . . . . 35Figure 2.3 Auditory streaming is a case of multistable

perception in the auditory modality . . . . 36Figure 2.4 Experiment stimuli: bistable intra-categorical

images + stabilized versions . . . . . . . . . 54Figure 2.5 Experiment stimuli: bistable inter-categorical

images + stabilized versions . . . . . . . . . 55Figure 2.6 Bistable Figures > Stable Figures . . . . . . 56Figure 2.7 Global 3D view of activations for the con-

trast internal change > external change fora random effects analysis with 16 subjects:selected activations superimposed on toaveraged anatomical sections . . . . . . . . 57

Figure 2.8 T statistic for Bistable > Stable in conjunc-tion with Intra-categorical: Bistable > Sta- ble switches (left) and Inter-categorical:Bistable > Stable switches (right) . . . . . . 57

Figure 2.9 Bistable activations conjoined with Inter-categorical and intra-categorical bistable

activations . . . . . . . . . . . . . . . . . . . 58Figure 2.10 Bistable faces > All. FFA activation pro-

jected onto averaged structural scans (left)and main HRF and TD plotted for FFA [ 38-58 -14] (right) . . . . . . . . . . . . . . . . 59

xv


16/116

xvi List of Figures

Figure 4.1 The Uncanny Valley graph following Mori 85


17/116


18/116

Part II N T R O D U C T I O N


19/116


20/116

1I N T R O D U C T I O N

From bistable perception toanthropomorphism

This thesis is a multidisciplinary work, whose goal is to mergeinsights and research from elds as different as art, neuroscienceand technology. My original question was: how do we attributefeatures of animacy (such as emotions and intentions) to objects,in spite of the certain knowledge of their non-animacy? Thisquestion brought me to consider, in the rst place, some basicphenomena of perception, and in particular, illusions. Illusionsmake evident that perception is not a passive mapping of in-puts coming from the environment, but an active interpretation,involving a mechanism of inference. As illustrated in Chapter I,studies on optical illusions may be very useful in order to revealthe active role of the brain in the organisation of perceptualprocesses. Among all different kinds of illusion, ambiguity isone of the most interesting, since it claries how our brain dis-ambiguates the information received. Ambiguity is traditionallydened as an alternation in time of two mutually exclusive inter-pretations of the same stimulus [ Zeki , 2004]. As I propose in theframework of this thesis, this denition ts to a specic kind of

ambiguity, namely the "intra-categorical" one, taking place whenthe two possible interpretations of the same stimulus belongto the same semantic category, for example the two recessionalplanes of the Necker cube. This kind of ambiguity is the basicmodel of a mechanism that probably happens endlessly in our brain, since our everyday environment contains ambiguitiesand conicts which we do not usually notice because our brainsuccessfully - and continuously - disambiguates them. It can behypothesised that such an evaluation process is occurring allthe time, but becomes evident when ambiguities are maximised,

as in the case of ambiguous stimuli. Indeed, also multistableambiguity involves a continuous and frequent evaluation of sensory inputs, but it results in puzzling ips (or reversals) inperception, since ambiguous percepts do not provide enoughclues for "deciding" which is the "good" interpretation. But, inpartial contrast with the above quoted traditional denition, Ido not consider all possible interpretations of an ambiguousimage as mutually excluding. In Chapter II, on the basis of brain imaging data, I will propose the notion of different levels

13


21/116

14 i nt ro du ct io n

of ambiguity, including in such a framework the concepts of theillusion of life and of life evocation as "higher level" ambiguities.The illusion of life is a phenomenon that can be consideredas the perceptual basis of anthropomorphising, or attributinganimacy, emotions, intentionality, personality and other traitscommon to living agents to non-animated stimuli. The illusionof life takes place when geometrical stimuli showing certainfeatures of motion and behavior automatically elicit attributionof animacy and intention, as investigated in the seminal studiesof [Michotte , 1946 ] and [ Heider and Simmel , 1944 ]. While theillusion of life is automatically and universally perceived, withthe only exception of persons with very specic brain damages,anthropomorphising is a psychological tendency implying anindividual variability and depending on factors such as social

isolation and need of mastering the environment. In Chapter IIII will focus on the features and neural events underlying theillusion of life, outlining why it can be considered as an illusionof ambiguity. As illustrated in Chapter IV, perception of animacyof non-animated objects has always been exploited and explored by artists, who have been featuring in their works the aestheticsof life evocation. Finally, life evocation triggering anthropomor-phising has a great potential of application in todays society,where humans interact more and more with technological objects such as robots and avatars, designed to embody believable

creatures. In Chapter IV I will discuss some features of animacyperception drawing inspiration from arts and insights from thecognitive sciences, with the objective of outlining believabilityissues for the design of technological applications.

1.1 percep tion : making sense of the senses

Perception as anactive process Philosophy and science have traditionally separated intelligence

from perception, vision being interpreted as a passive windowson the world. It was German fellow Hemann von Helmholtz(1821 -1894 ) who rst proposed the principle that visual percep-tions are unconscious inferences [ von Helmholtz , 1962 ]. VonHelmholtz thought that human perception is only indirectlyrelated to objects, being inferred from fragmentary data. Therehave been researchers who maintained a "direct" theory of per-ception, notable American psychologist J. Gibson [ Gibson , 1950]who outlined the theory of affordances, proposing a model inwhich our senses "pick up" information form the environmentgiving signicance to pattern of stimulation without recurring


22/116

1.1 perception : m ak in g s en se o f t he s en se s 15

to further processing. But the majority of studies today agree onthe hypothesis that perception is not a passive reception of theinformation coming from the surrounding environment, ratherafrming that the brain is a story-teller who actively buildsthe reality surrounding us, as illustrated by [ Roseneld , 1988 ].Researchers point out that our senses are faced by a chaotic, per-sistently changing world without labels. What our brain activelydoes is organising external reality, allowing us to generaliseand thus use the information we need from the environment,imbuing it with meaning. Since meaning is not in things butis in the brain, information, if not interpreted by the brain, isempty of meaning.

1.1.1 Watching without seeing

The born-blind issueResearchers supporting the thesis of "active perception" often re-fer to the argument of born-blind people regaining sight duringadulthood. Indeed, born-blid individuals who recuperated theirsight after many years, even if visually perceiving objects, donot "see" them (they do not understand what the objects are), orlearn to do it after a long and difcult training. In this brain con-dition, generally labelled as agnosia, sensations of light, colour,movement and shape can reach the brain but are meaningless;therefore for individuals suffering from this condition, objects

are seen as "meaningless" items. The rst debate about this topiccan be traced back to a correspondence between English philoso-pher John Locke and William Molyneaux [ Gregory , 1987 ]. In1688 the Irish scientist and politician William Molyneux ( 16561698 ) sent a letter to John Locke in which he asked whethera man who has been born blind and, during the course of hislife, has learnt to distinguish and name a cube and a sphere by touch, would be able to distinguish and name these objectssimply by sight, once he had been enabled to see. The so-called"Molyneaux problem" has been solved forty years later, whenthe English surgeon William Chelsden operated a 13 years oldfrom a cataract, allowing the boy to see for the rst time in hislife. The rst impression of the boy was objects were "dispropor-tioned". Moreover he did not have any sense of distance, and of the relation between size and distance: a little object very closeto his eyes was equivalent to a big house seen from far away. Hewas not able to distinguish a cat from a dog, unless he couldtouch them.


23/116


a case of s ight recovery When he rst saw, Cheseldenwrites, he was so far from making any judgement of dis-tances, that he thought all object whatever touched hiseyes (as he expressed it) as what he felt did his skin, andthought no object so agreeable as those which were smoothand regular, though he could form no judgement of theirshape, or guess what it was in any object that was pleasingto him: he knew not the shape of anything, nor any onething from another, however different in shape or magni-tude; but upon being told what things were, whose formhe knew before from feeling, he would carefully observe,that he might know them again; and (as he said) at rstlearned to know, and again forgot a thousand things in aday. One particular only, though it might appear triing,

I will relate: Having often forgot which was the cat, andwhich the dog, he was ashamed to ask; but catching thecat, which he knew by feeling, he was observed to lookat her steadfastly, and then, setting her down, said, So,puss, I shall know you another time. He was very muchsurprised, that those things which he had liked best, didnot appear most agreeable to his eyes, expecting thosepersons would appear most beautiful that he loved most,and such things to be most agreeable to his sight, that wereso to his taste. We thought he soon knew what pictures

represented, which were shewed to him, but we foundafterwards we were mistaken; for about two months afterhe was couched, he discovered at once they representedsolid bodies, when to that time he considered them only asparty-coloured planes, or surfaces diversied with varietyof paint; but even then he was no less surprised, expectingthe pictures would feel like the things they represented,and was amazed when he found those parts, which bytheir light and shadow appeared now round and uneven,felt only at like the rest, and asked which was the lying

sense, feeling or seeing? [ Cheselden , 1683-1775 ]The boy thus had a troublesome learning path, in which noth-

ing came natural or spontaneous; he was conscious of the effortand he was trying to learn and record things everyday. For in-stance, he knew that he had a cat and a dog, and that he hadto distinguish them by seeing them; this task was probably asdifcult as for a normal-sighted person learn to distinguish at aglance two very similar specimen of the same breed. But, tridi-mensionality appeared to Cheseldens patient as some kind of


24/116


illusion. Even after he acquired the capacity to correctly interpret bidimensional images, he was still "amazed" to perceive the body represented by paintings as at. The phenomenon thanksto which he could see a bidimensional object as tridimensionalwas so obscure to him that he was convinced that one of hissensory channels must have being lying. Several similar oper-ations have been conducted afterwards, and always with thesame results. In particular, bidimensional representations are al-ways troubling for born-blind individuals recovering their sight, because they see such representations in three dimensions, andthey cannot understand how this can be, when they touch theimage nding it at. "Normal" perception of size, distance andtridimensionality is a phenomenon that we take for granted - but is actually the result of an operation of abstraction produced

by the brain.

1.1.2 The Brain: an Abstraction Machine

Distance, size and tridimensionality are abstractions produced by the brain among all the mechanisms allowing us to recog- Abstraction at cell

levelnise objects. Acquiring the capacity to create the abstraction of "representation on two dimensions" is probably something thatwe learn to do very early in time, and blind born individualsdo not have the possibility to develop at "due time". Studies

afrm that the common capacity of the cerebral cortex - whethervisual, auditory, somato-sensory ot otherwise- is namely theone of abstraction: "the capacity to abstract seems to accompany,and to be a corollary of, every specicity" [ Zeki , 2009 ]. What itis meant for abstraction, in this specic context, is the capacityto grasp a general property instead of the particular one. Anexample of this process at a single cell level is orientation se-lectivity [ Hubel and Wiesel , 1977]. Orientation selectivity is theproperty of some cells in the visual brain, which respond toobjects (lines) of a specic orientation and do not respond tolines oriented orthogonally at their "preferred" orientation: thereare cells which will respond only to vertical lines, and otherto horizontal ones. For instance, a vertical-selective cell willrespond when the stimulus is vertical, without being concernedwith what is vertical, whether a tree or a vertical line or a tower:this means that the cell has abstracted the property of verticalityfrom different stimuli discarding individual specicities.

In Figure 1.1, we can see the reponse of an orientation selectivecell. Such a response can be studied by inserting an electrode


25/116


Figure 1.1: The response of an orientation selective cell

into the visual cortex (a) and showing bars of light of differentorientation into the Receptive Field (the small rectangle on theleft, RF), that is to say that part of the global visual eld (the bigger rectangle) which will trigger an electrical discharge in thestudied cell. (b) shows the representation of the cells selectivityto orientation: the cell responds positively to the oblique linemoved in two opposite directions (rst three records) while itis unresponsive to the orthogonal orientation. In the same way,cells of the brain area specialised for processing motion, V 5, aredirectionally selective. This means that they respond to motionin a specic direction and not in the opposite one, as illustratedin Figure 1.2. This kind of abstraction does not concern exclu-sively the visual brain, but also other sensory modalities. Cells"designed" for perceiving pressure in the somatosensory cortexwill respond to pressure independently from what causes it; thesame thing is true for a cell responding to temperature, pain,and for the auditory system.


26/116


Figure 1.2: The response of an orientation selective cell in V 5

1.1.3 The Case of Colour Vision

Abstraction is the strategy used by the brain in order to create acoherent environment out of a chaotic and ever-changing world.

As in the largely discussed example of colour vision, we cannot What are colours?help but perceive and organise constant features; we cannotavoid seeing a leaf as green at dawn and dusk, or a rose as red both in a sunny and cloudy day, taking for granted that "colours"are features intrinsic to objects. Since the world we see is alwayschanging and the retina receives a constant ow of differentkinds of visual information, the brain must be able to selectvisual properties of objects and surfaces in order to give themmeaning. In acquiring this ability, the brain has developed spe-cialized functions for the analysis of different properties, such

as colour, shape, and movement. For example, contrary to ourvisual experience, there are no colours in the world, only elec-tromagnetic waves of many frequencies. Our retinal receptorsfor colour are divided in three categories, responding optimallyto long (red), middle (green), and short (blue) wavelengths. The brain compares the amount of light reected in the wavelengths,and from these comparisons creates the colours we see. Theamount of light reected by a particular surface - a table, forexample - depends on the frequency and the intensity of the


27/116


light hitting the surface; some surfaces reect more short-wavefrequencies, others more long-wave frequencies; but the par-ticular sensation produced by a specic colour is created byour brain. Indeed, if our perception of an object as red were tochange with every change in the wavelength of light reectedfrom it, the object would constantly change in appearance. If wewere aware of our real visual worlds we would see constantlychanging images of dirty grey - constantly changing images thatwould be very confusing, often making it impossible for us tosee forms. While, thanks to this mechanism, the brain is ableto attribute a constant colour making itself largely independentof the amount of light of the waveband reected from objectssurfaces. This mechanism is a way of our brain of "inventing" anattribute allowing us to identify objects by means of a constant

and meaningful feature. How can this happen in the brain? VonHelmholtz [ von Helmholtz , 1962 ] based this phenomenon onlearning: he thought that since we know that a leaf is green,we operate a mechanism, which he names "unconscious infer-ence", allowing to "attribute" a colour to the leaf even if theilluminant (the wavelength composition of the light reected indifferent lighting conditions) always changes. Another Germanpsychophysicist, Ewald Hering, thought that memory playeda central role [ Hering , 1964 (originally pub. 1877 ]. Higher cog-nitive functions have therefore always been addressed when

trying to explain mechanisms underlying colour vision, untilEdwin Land (the inventor of the Polaroid) nally proposedanother point of view [ Land , 1974 ]. He hypothesised that theassignment of constant colours to objects is the result of a sim-ple computational process of the brain, due to some kind of innate capacity of organisation of visual signals. Following Land,colours are the result of the comparison between the amount of light of different wavebands reected from a surface and fromits surrounds. This comparison has as a result a ratio, which isconstant (differently from the amount of reected light).The Land hypothesis

the inherited concept of rat io -taking : In the Landexperiment, a green surface is surrounded by yellow, redand blue surfaces. If the surface is lighten by a projectorprojecting blue light, the surface will reect lets say 60units (milliwatts) of green light. The surrounds will bereecting less green light. The same surface is seen undera red light: the green surface will now reect 30 units of green light. But we will still see it as green! This is be-cause the surrounding surfaces will reect even less units


28/116


of green light than before. Indeed, the ratio (relationship between the amount of green light reected by the greensurface and the one reected by non-green surfaces) isalways identical.

Colour perception is the most powerful example of how ourexperience of the world is based on the brains capacity toperform operations related to inherited concepts rather thenmere physical reality. The crucial feature of the colour system inthe brain is the ability to ascribe a constant colour to an objectdespite of wide-ranging changes in the wavelengths-energy com-position of the light reected by such an object. If the colour of the surface would change along with every change in its lightingenvironment, objects would have ever-changing surfaces (e.g.,leafs would be green in the morning and red in the evening).This mean that colours would not mean much and, rather thanprovide us with information about things, would actually con-fuse us, being a non-constant overow of useless information.The importance of understanding colour as a construction of the brain is that colour is an example of mechanism that the brain has developed through evolution, and that concurs toinstil meaning and thus gain additional knowledge from theenvironment: our visual worlds are stabilized because the brain,through colour perception, simplies the environment by com-

paring the amounts of lightness and darkness in the differentfrequencies from one moment to the other. In the same way,an object maintains its identity independently from the pointof view and distance we look at it. The capability to organizeconstantly changing stimuli in a stable and meaningful way isan impressive feature of perception and is the way our brainallows us to cope with reality. And just as colours are differentin kind from the images projected on our retinas, space anddepth are different in kind from the individual images projectedthe retinas of our eyes. In creating three-dimensionality and

depth the brain in actually invents subjective experience. Thethree-dimensionality we normally see is not part of the "real"world; all of our perceptions are from a particular point of viewand that individual point of view is our subjectivity; it is cre-ated by the brain as it makes sense of the physical world thatsurrounds us.


29/116


1.2 concepts and categor ies in the bra in

We saw how the brain can be independent from the continualchange of the surrounding information, allowing us to recognizeessential and non-changing characteristics of objects and situa-tions, optimising knowledge acquisition by abstraction. Colourperception is an example of categorisation intrinsic to the brainsdesign. The capacity to see colours can be lost, as happenedto the most famous case of cerebral achromatopsia, the one of the colour-blind painter Johnathan I. [ Sacks , 1995 ]. But, evenif the capacity to see colours stays intact, the brain can losethe capacity to abstract; and in this case, even if colours are"viewed", they are not "understood", becoming useless. Gelband Goldstein [ Gelb and Goldstein , 1925] have reported the case

of a patient who was incapable to dene the colours of objects,even if, apparently, he could perceive them. The patient couldnot indicate the name of the colour of an object mentioned tohim, neither he was able to point out a colour correspondingto a colour-name. Presented with colour samples, he utteredsome names, giving the impression that words had no meaning.When he was asked to choose a red object among a sample of different objects, he would choose by chance. But, if he was re-quested to chose a colour sample tting to a coloured object, healways succeed: he never chose a wrong colour. On the contrary,

he was never satised: if the colour specimen did not matchperfectly with the colour of the object, he continued looking fora more tting specimen. This proved that he could "perceive"colours. But, when the patient had to assort different specimenof colours, the patient was incapable to nd a unifying categoryin order to group them: the factor of hue, brightness, or otherfactors may prevail. Gelb and Goldstein explained the patientsCategorising

behaviour behaviour (as well as other patients symptoms) as effects of thereduction from the level of "categorical" behaviour to the level of "concrete" behaviour. Confronted with the same task (grouping

colour specimina according to hue) the normal person imme-diately assorts the same specimina, e.g. a very light and a verydark red, by categorizing them as "red". In doing so, she is notunaware of the difference between them; she is performing anoperation of abstraction (abstracting the "redness") and categori-sation (by discarding "individual" differences among speciminaand unifying under the category "red"). Gelb and Goldsteinspatient, even seeing colours, was unable to establish abstractrelations between the category (red, blue, green, etc.) and the


30/116

1.2 concep ts and categor ies in the brain 23

concrete colour. When he was asked to dene the colours, healways had to make a comparison (e. g. "like an orange, like acherry") and he was meticulously grouping colours but without backing up on a categorical sense. The patient was conned tothe content of perception as it is actually experienced (he wasseeing the "real" world, in a way); in other words he could notgo through his perceptual experience under the perspective of a principle external to the perceptual experience itself, beingunable to refer his actual experience to any conceptual order.In the framework of this regression to a "concrete" behaviour,there is no hierarchy or differentiation between the experientialfeatures and their signicance. All features are equally impor-tant; each features is of paramount importance because all of them are encountered "at the same level" in actual perception.

The patient was overwhelmed by the actual experience, and hecould not emancipate from it by imposing a structure able tocreate meaning in the world.

What Goldstein refers to as "categorising" behaviour is par-allel to Zekis theory of inherited and acquired concepts: "The Inherited and

acquired conceptsinherited concepts organize the signals coming into the brain soas to instil meaning into them and thus make sense of them",[Zeki , 2009 ] while acquired concepts are generated through-out our existence, with the goal of simplifying perceiving andrecognizing things and situations. Inherited brain concepts are

identiable by three features, in Zekis view: absence of freewill, immutability and autonomy. Therefore, in this perspectivecolour perception may be classied as an inherited concept, sincewe can not choose what colours we perceive, nor we can chooseof seeing the "real" colours of things instead that the colours"invented" by our brain; secondly, our system of perception forcolours is immutable, does not change over years - in normalconditions - and thirdly, the generation of colours is dependenton a specialised cortical system that differs from other systemsdesigned for processing other kinds of visual features. While

acquired concepts are a way to organise knowledge throughoutour life (for instance, we can learn to dene a specic set of paintings as "still life" and we will be able to insert in such a cat-egory all still life that we will see afterwards, even if they will becomposed of different objects, lighting conditions, colours, etc.).Still, what acquired and inherited concepts have in common isthe capacity of the brain to abstract and to generalize.


31/116


1.3 seeing through i l lus ions

The way in which our brain organises knowledge can be more ef-ciently investigated in exceptional conditions: those situationsin which things do not work as usual. This means analysingthose circumstances in which the brains capacity of stabilisationand abstraction is invalidated, temporary or permanently dis-carded, as in the cases of brain damages and illusions [ Gregory ,2009 ]. It is hard to give a satisfactory denition of the term "il-lusion". The OED (Oxford English Dictionary) denes it as "aninstance of a wrong or misinterpreted perception of a sensoryexperience"; here I will refer more precisely, to those phenom-ena implying a discrepancy measurements and inner perception.Such a discrepancy can happen when the knowledge (inherited

or acquired) that we "superimpose" to perception is inappro-priate or misapplied; in other words, where the "assumptions"we are doing are wrong. Indeed, perception is - following thisperspective - based on inferences that we deduce from sensorysignals; being an inference, it requires a previous knowledge.This is why illusions can clarify many mechanisms behind per-ception, indicating where and how this "previous knowledge"(abstraction, categories and things we learn throughout our life)play a role. As Chelsdens blind born boy asked himself whetherwas the sight or the touch which was lying: he thought that

perceived tridimensionality was an illusion. And in a way itis, because behind our capacity to view tridimensional imagesin pictures there is a level of abstraction: illusions can provideevidence of working rules of perception.

1.3.1 Two Different Categories of Illusions

Physical illusionsGregory identies two different kinds of illusion, the cognitiveand the physical ones. While the latter ones have a physicalcause, cognitive illusions are due following Gregory to "misap-plication of knowledge" employed by the brain in interpretingsensory signals [ Gregory , 2009]. Illusions due to the disturbanceof light, between objects and the eyes, are different from illu-sions due to the disturbance of sensory signals of eye, though both might be classied as physical. Among these, we can listwith Gregory mist (loss of information increasing uncertainty),mirage (refraction of light between the object and the eyes dis-placing objects or parts of objects, rainbow. Discussing this


32/116

1.3 see ing through i llus ions 25

taxonomy is nevertheless outside the scope of this thesis, andfor further reading on this subject we remand to [ Gregory , 2009]. Cognitive illusions

Cognitive illusions are further devided by Gregory into "spe-cic knowledge" of objects from "general knowledge". An exam-ple of specic knowledge cognitive illusion is the "hollow face"illusion.

Figure 1.3: A rotation of a mask: the forth image shows the inside of the mask, appearing convex even if it is hollow

As illustrated in Figure 1.3, when seeing a hollow mask weare strongly biased in seeing it as convex. This bias of seeingmasks as convex is so strong it competes with monocular depthcues, such as shading and shadows, and also very considerableunambiguous information from the two eyes signalling stereo-scopically that the perceived object is hollow. Also, a textureimitating wood could look like wood even if it is plastic, or if it ispainted; this is because we apply our specic knowledge to thetexture and make an inference about the material on the basisof a specic and repeated experience (wood is grainy, brownish,etc.). An example of general knowledge leading to cognitiveillusions is misleading rules of Gestalt (Wertheimer, 1923 / 1938 )applied to tricky objects (designed to trigger the illusion) e. g.Kanitsa triangle, due to postulating a nearer occluding surface to


33/116


34/116


35/116


36/116

1.3 see ing through i llus ions 29

Figure 1.5: Different kinds of ambiguities


37/116


38/116

Part II

F I R ST A N D S E C O N D L E V E LA M B I G U I T Y


39/116


40/116

2P E R C E P TI O N O F B I S TA B L E A M B I G U I T Y

This chapter will discuss literature about how and where ambiguous perception takes place in the brain, and why ambiguityis a crucial issue for understanding how the mind works andhow do we gather information from the surrounding environ-ment. It will also discuss the results of an fMRI experimentcarried out in the general framework of ambiguity with a focuson visual bistability, proposing a framework of analysis basedon multiple levels of ambiguity, which takes into account thatdifferent interpretations of the stimulus content may not only ex-clude each other, but also coexist, depending on different levelsof ambiguity.

2.1 what i s mult istable ambigui ty

Bistable ambiguous images are puzzling because they can bespontaneously experienced as two equally valid percepts. Fol-lowing the model of perception illustrated in Chapter I, whenthe brain is confronted with an ambiguous stimulus perceptual

knowledge is oscillating between two different interpretation,while conceptual knowledge is aware that the percept is notactually changing. Following the literature, during the so-called bistability illusion, the brain is confronted with one ambiguousstimulus that can be stably interpreted in only one way at anygiven moment, but will present two possible interpretationsover time; in other words, a stimulus is ambiguous when it isconsistent with two or more mutually exclusive interpretations[Zeki , 2004].

Repeated viewing of ambiguous stimuli lead to spontaneousperceptual switches, or "ips", where the brain alternates be-tween two or more stable perceptual interpretations every fewseconds. Multistable ambiguity is an especially interesting il-lusion because it can help us understand the mechanisms gen-erating a meaningful and coherent experience of the world,even though the the information we have is fragmentary andambiguous. Visual illusions of ambiguity are actually a sortor quintessential and simplied model of the choices that our brain is confronted with everyday, and in every modality. Some

33


41/116

34 percept ion of b istab le ambigui ty

Figure 2.1: a) Necker Cube - b) Rubin Vase

of the most well-know ambiguous gures are the Necker cubeand the Rubin vase (Figure 2.1). The perceptual interpretationof the Necker cube oscillates between two different recessionalplanes, while the interpretation of the Rubin vase alternates between one vase and two proles. Ambiguity is present alsoin other perceptual phenomena, namely bistable apparent mo-tion and binocular rivalry (Figure 2.2). Binocular rivalry resultsBinocular rivalryfrom presentation of different images to each eye, thanks to adispositive separating the view of one eye to the other one. The

result is bistable alternation between the two images: the subjectis conscious of one stimulus at the time, and the perception isincessantly uctuating between the two stimuli. For instance,the subject represented in Figure 2.2, looking at the vertical lineswith his left eye and looking at the horizontal lines with hisright eye will cyclically perceive only vertical lines alternatingwith only horizontal lines.

Multistable perception (when the possible interpretations are Auditorymultistability more than two) can characterise the auditory modality as well,

in the form of auditory stream segregation [ Bregman , 1990] andverbal transformation effect [ Warren and Gregory , 1958]. Audi-tory stream segregation happens when two tones of a differentfrequency are presented alternately in a repeating temporal pat-tern, as illustrated in Figure 2.3. Subjects perception alternates between interpreting the sequence either as one stream withuctuating tones or as two segregated streams (in the picture,indicated by the grey thick line).Verbal

transformation effect Verbal transformation effect occurs when a word is cycledin continuous repetition [ Warren and Gregory , 1958 ] like for


42/116

2.1 what i s mult is table amb igui ty 35

Figure 2.2: Binocular rivalry

instance the word "life". Initially, a percept corresponding to theoriginal form prevails, but after a certain period another percepttakes over and it stably alternates with the original one: in thecase of the repeated word "life", at a certain point the subjectwill perceive the word "y" alternating with "life". Ambiguity in

languageAmbiguity is also a feature of many sentences, which canmean two very different things, being in a way "bistable" and in-

terpretable only by means of the context in which they are used.There is a distinction between vagueness and real ambiguity:truly "bistable" sentences are the ones whose "meaning" can ipand can be stabilized only by context, for instance. "People likeus" [Gregory , 2000] or "She cant bear children". Tactile ambiguity

Bistability exist also in the modality of touch, even if it hasto be triggered articially. A tactile illusion has been devel-oped by Carter et al [ Carter et al. , 2008 ]. By means of a devicedesigned to provide the subject with vibrotactile stimuli, theresearchers could lead participants to report switches between

the perception of motion directed either up and down or leftand right across their ngertip, while the sensory input due tothe vibrotactile device stayed unvaried.

Nevertheless, visual ambiguity is undoubtedly the most stud-ied phenomenon among all these. Indeed, ambiguous gureshave become an experimental tool to study interpretive cognitiveprocesses related to conscious awareness because the perceptualexperience alternates over time without any external changes of the stimuli [ Leopold and Logothetis , 1999]; [Sterzer et al. , 2009].


43/116


Figure 2.3: Auditory streaming is a case of multistable perception inthe auditory modality

In particular, interest in this eld has been growing thanks to theadvent of non-invasive brain imaging techniques such as fMRI

(functional magnetic resonance imaging), which is the techniqueused for the experiment described further on.

2.1.1 Neural processes underlying multistable phenomena

Even tough perception of ambiguous gures has been exten-sively studied, there are many contradicting ndings and contro-Top-down vs

bottom-up versies. Traditionally, two main theories explain why ambiguousgures reverse; the main distinction is between supporters of thecentral role of bottom-up versus top-down processes [ Leopoldand Logothetis , 1999]. The bottom-up theory proposes that neu-rons maintaining one percept fatigue over time and give rise toneurons supporting the other percept; the switch should thus be independent from high-level (cognitive) control. Accordingto the top-down theory, ambiguous gures do not switch spon-taneously: the preconditions for seeing reversals are that theviewer knows the gure is bistable, knows the possible inter-pretations of the gure, and switches are initiated by intention.There are experimental ndings supporting both theories, thus


44/116


more recent studies have adopted an hybrid model explicat-ing bistability as the result of a continuous dialogue between bottom-up and top-down processes [ Toppino and Long , 2005 ][Mitroff et al. , 2006]

Concerning the localisation in the brain where bistable ambiguity takes place, several studies show that changes in earlyvisual activity precedes conscious changes of perception. Thanks Subcortical and

early cortical visual processing

to fMRI studies, it has been consistently demonstrated that binoc-ular rivarly strongly effects V 1, the earliest visual processingarea [ Polonsky et al. , 2000], [Tong and Engel , 2001] [Lee, 2005]. AMEG study [ Parkkonen et al. , 2008] shows that early visual brainareas (V 1 and V2) reect how an ambiguous gure is perceived, both for binocular rivalry and for Rubins vase. Also activity inprimary visual cortex recorded with EEG could predict what

perspective is perceived even before a geometrical ambiguousgure was presented [ Kornmeier and Bach , 2005 ]. This studydemonstrated that perceived reversal of perspective was pre-ceded by 160 ms with negativity in primary visual cortex. Asimilar negative component was found 50 ms earlier doing theexperiment with a stabilized version of the gure. fMRI studieshave also shown that activity not only in V 1 but also in anotherpost-retinal processing area, the LGN (lateral geniculate nucleus)reects the perceptual outcome of binocular rivalry [ Tong et al. ,2006 ]. These experiments alone could support the bottom-up

thesis, that is to say the hypothesis that perception of ambigu-ous gures is resolved in a feed-forward manner by primarysensory areas without the involvement of higher cognitive areas; but, since there is evidence also for top-down processes, thisdata can be interpreted as a proof that early visual processingstages including V 1 and LGN are the prerequisite for consciousinterpretation of percepts. In any case, the role of this earlyvisual areas is that of processing information but also inuenceinterpretation ("am I going to see a vase or a face?") either vialocal interactions or through modulation by feedback signals

from higher cognitive areas [ Tong , 2003]. Evidence for the latterhypothesis comes from fMRI experiments on bistable apparentmotion. Indeed, whenever the perception of apparent motion isinconsistent with added clues, early visual activity is suppressed[Sterzer and Kleinschmidt , 2005].

Most studies agree on the role of the extrastriate cortex in Extrastriate visualcortexperceiving multistable ambiguities (extrastriate visual cortex

includes those areas lying beyond V 1). Many fMRI studies re-vealed correlations between subjective perception and activity


45/116


in the functionally specialised extrastriate cortex, that is to say,areas specialised to process a certain type of information, re-ecting the "content" of the stimulus itself. fMRI studies (see[Sterzer et al. , 2009 ] for a review) demonstrated that during binocular rivalry, uctuations in signals in extrastriate cortexare similar to those during actual alternation of two differentstimuli, suggesting that the interpretation of ambiguous stimuliare fully resolved at the stage of processing, without maintain-ing a representation of the temporally suppressed stimulus.Actually, perception of binocular rivalry is inuenced by infor-mation linked to the suppressed stimulus as well [ Andrewsand Blakemore , 1999 ] indicating that the suppressed stimulusis somehow "present" and processed in the brain. For instance,the emotional content of a suppressed stimulus (e. g. a fearful

face) is still processed in the amygdala [ Jiang and He , 2006

],[Williams , 2004 ], where activity is expected when the stimulusis consciously perceived. A more recent and high resolutionfMRI study has also found activities corresponding to responsesto different object category in "houses versus faces" binocularrivalry, namely activations in the fusiform face area (FFA) andin the parahippocampal place area (PPA) even during binocularsuppression of one of the two categories of stimuli [ Sterzer et al. ,2008 ]; in the same way, face-specic responses are reduced butstill present in EEG during a face-suppression period [ Sterzer

et al., 2009]. In parallel with these ndings on binocular rivalry,studies on ambiguous motion and bistable images conrm thatactivity in the extrastriate visual areas correlates with consciousperception, and also that those areas are involved in the cycli-cal resolution of ambiguities. Following [ Andrews et al. , 2002 ]and [ Hasson et al. , 2001 ], signals from FFA are greater duringthe perception of two faces in the Rubin vase-face illusion; andfollowing the same principle, a bistable illusion whose elementscan be perceived either as a coherent shape or as a randomstructure, will correlate with lateral occipital complex (LOC,

which preferably activates processing objects) activations whenthe stimulus is perceived as coherent. In the case of ambiguousapparent motion versus icker, the conscious perception of mo-tion will correlates with activations in V 5 (area which processesmotion) while the conscious perception of icker will not.

Another line of research has been focusing not on the per-ceptual states between one conscious perception and the otherNeural correlates of

ips one, but on the ips themselves, in other words neural eventscorrelating with perceptual reversals. Flips-related activity is


46/116


generally observed in extrastriate visual areas and it is associ-ated with activations tuned to the features of the percept that isperceived. For instance, perceptual reversals from faces to objectcorrelates with activations in the ventral stream [ Leopold andLogothetis , 1999 ] while apparent changes in motion directioncorrelates with activation in V 5, motion sensitive area. [ Sterzerand Kleinschmidt , 2007 ] suggested that prefrontal areas set off changes in perception of ambiguous motion. In an fMRI studythey used both ambiguous and unambiguous moving dots; acti-vation in right inferior frontal cortex appeared earlier than V 5activation in both conditions. Moreover, the frontal activationhappened earlier for spontaneous ips when looking at ambiguous stimuli, compared to when a ip was stimulus driven.By using EEG, [ Britz et al. , 2009 ] were able to predict stimulus

perception in a similar manner to Kornmeier and Bach (2004

),they found signicant increase of activity in right inferior pari-etal cortex 50 ms before a ip in perception occurred. It hasalso been demonstrated that activity in the FFA can be used topredict whether faces or a vase is perceived during presentationof the Rubin gure [ Hesselmann et al. , 2008 ]. Indeed, activityin the FFA is higher when subjects subsequently report perceiv-ing two faces instead of a vase, suggesting that pre-stimulusneural activity precede subsequent perceptual inference. Suchactivations suggest that ongoing brain activity inuences the

resolution of ambiguity before stimulus driven processes, anda key role of functionally specialised areas in processing andinterpreting conscious visual perception. Parietal, frontal and

prefrontal cortexfMRI activations correlated with perceptual reverals or ipsare also assessed in the parietal and frontal areas [ Lumer et al. ,1998 ]. While extrastriate areas are equally activated both bynon-ambiguous and by ambiguous stimuli, parietal and pre-frontal regions show higher levels of activity during ambiguityillusions [ Lumer et al. , 1998]. Such special activations in frontaland prefrontal cortex could suggest top-down mechanisms trig-

gering a re-organisation of activity in the primary sensory areasduring ips as in [ Leopold and Logothetis , 1999 ]. Or, from the bottom-up perspective, they could reect the feed-forward com-munication of events from the earlier visual cortex to highercognitive areas, a sort of initiating gateway for further pro-cessing. For example, changes in apparent motion perceptionshow that activations of the prefrontal cortex precede that of V5 for bistable motion perception [ Sterzer and Kleinschmidt ,2007 ]. This "initiating" role is corroborated by other ndings,


47/116


drawn from a study suggesting that ips during observation of a Necker cube are preceded by activations in the right parietalcortex [ Britz et al. , 2009 ]. In such a framework, it is noteworththat many people are able to control ips of bistable gures,which suggest that perception is also inuenced by top-downfactors. For instance, participants were able to voluntarily in-crease or suppress the frequency of switches for the Neckercube [Tracy et al. , 2005]. It is demonstrated that higher cognitiveareas have an effect on voluntary switches. Indeed, the abilityto voluntarily increase switches is diminished in frontal cortex-damaged patients; but, the same patients did not experience asignicantly different rate of reversals when ips where happen-ing spontaneously [ Windmann et al. , 2006 ], while research of effects of meditation consistently demonstrate that meditators

can alter the normal uctuations in conscious state induced by binocular rivalry [ Carter et al. , 2005 ]. The role of frontal cortexin active switches suggests that top down inuence may not benecessary, but higher cognitive areas can play a role voluntarilyinitiation changes, while alternation during passive viewing isless dependent on prefrontal cortex. Activities in frontal andparietal cortex is not only associated with ips, but also inpercept stabilisation. Indeed, the tendency of an individual tostabilize a percept during, for instance, periods in which thestimulus has been removed, is correlated with activations in the

frontal and parietals areas [ Raemaekers , 2009].Bottom up versustop down:conclusions

In conclusion, experimental ndings neither support a puretop-down or bottom-up account. There is both support for re-versals driven by primary visual areas, processing specic areassuch as the FFA and parietal and frontal regions. Even thoughactivations in typical higher cognitive areas appear to be relatedto perceptual switches [ Kleinschmidt et al. , 1998]; [Lumer et al. ,1998 ]; [Sterzer et al. , 2002 ] their involvement is still debatable.[Zeki , 2004 ] argues that the frontoparietal network is involvedwhen there is a change in perception, without being involved

in the actual visual percept. One solution to the contrastingexperimental ndings is that there might be interaction betweenhigher cognitive and primary sensory regions. Indeed, recentstudies do not point to either a purely top-down or bottom-upmodel. For instance, [ Britz et al. , 2009 ] found that activationin right inferior parietal cortex precedes the ips, while [ Ko-rnmeier and Bach , 2005 ] were able to predict perception fromactivation in primary visual cortex. Also behavioural ndingsseem to contrast a purely cognitive top-down account. It is true


48/116


that subjects who are not informed that a picture is ambiguousdo not always experience reversals, but evidence supports thatperceptual bistability be spontaneously experienced by childrenas young as 5 years [Mitroff et al. , 2006].

2.1.2 Attention and perception of bistable gures

The overlap between areas involved in spatial attention andperception of bistable gures have lead to speculation about therole of attention processes, suggesting that frontoparietal regions The attention issuecould activate when re-directing attention to sensory input andre-initiate an evaluation of the current interpretation, leadingeither to maintain stable the current interpretation or change it.[Slotnick et al. , 2003 ] found common neural activation for both

voluntary shifts of attention and voluntary perceptual reversals.Since perceptual reversals usually occur spontaneously there has been some debate about the exact role of the frontoparietal net-work [ Sterzer et al. , 2009]. Areas involved in voluntary attentionmay serve several functions, such as being engaged in feedbackto the sensory areas and perform an ongoing re-evaluation orthe visual experience [ Leopold and Logothetis , 1999 ]. Anothersuggestion is that ambiguous information is detected in earlysensory areas activates the frontoparietal network, which shiftsattention between the possible versions over time. According

to the so-called focal-feature hypothesis, local areas within anambiguous gure favour different global interpretations. Neckerhimself (quoted in [ Toppino , 2003 ] proposed that reversals aredriven by eye movements. Perception of the Necker cube can be biased by moving the point of xation during viewing [ Petersonand Gibson , 1991 ]; [Toppino , 2003 ], and similar effect has beenfound other bistable gures as well [ Tsal and Kolbet , 1985]. Alsofree viewing conditions support that eye gaze and perception of the Necker cube is closely linked [ Einhauser et al. , 2004]. After aswitch, the eye position also shifts. The authors suggest that thechanges of eye position serves as a negative feedback signal tosuppress the previous percept. [ Leopold and Logothetis , 1999 ]propose that the same motor processes underlie both selectiveattention and bistable perception, and that there in most cases isa close coupling between saccades and percept switches. Chang-ing eye gaze and perceptual reversals could both reect the waywe actively explore and constantly reinterpret the stimuli. It mayhowever be possible to alternate bistable gures without eyemovements.


49/116


To sum up, while previous studies opposed top-down [ Leopoldand Logothetis , 1999 ] to bottom-up [ Attneave , 1971 ], [Blake ,1989 ] models, now multistable illusion is considered as a con-tinuous and frequent dialogue between low-level (sensorial)and high-level (parietal and frontal) areas, aiming to verify intime interpretation of stimuli (and initiating changes in percep-tion). There may be a relation between selective attention andperception of bistable gures, but this still needs to be claried.

2.1.3 Levels of ambiguity

In his essay "The Neurology of Ambiguity" [ Zeki , 2004 ] dis-tinguishes between different levels of ambiguity, introducinga new component in a discussion traditionally limited to the

bottom-up/top-down approach. Zeki proposes that ambiguityhas different layers, or levels, which correlate with the neuralactivity in one or more processing regions with a consciouscorrelate. Following Zeki, gures like the Necker cube belongto the most simple form of ambiguity because brain activityremains within the same area over time, with the same neuralcorrelates for both states it can be experienced. At a higherlevel of ambiguity such as the Rubin gure, several processingareas are involved and the current perceptual state becomesconscious by uctuation of neural activity between these areas.

The Necker cube is always seen as a cube, while some imagesuctuate between image categories such as the Rubin gure.On a much more sophisticated level, artwork such as Vermeerspaintings are ambiguous in terms of narrative interpretation.Zekis differentiation between levels of ambiguities is rootedin his theory of microconsciousness, where consciousness isseen as distributed over functionally specialized processing sitesin the brain, which give rise to consciousness without furtherhigher interpretation. The level of ambiguity can be denedat uctuations within one or between several microconsciousstates.

2.2 an experiment on two -levels bistable ambiguity

In the work reported here, to investigate if there are generalpatterns of neural activations when looking at bistable gures,we compared perception of bistable images, where the physicalstimulus remains the same but perception alternates betweentwo interpretations, with perception of two externally alternat-


50/116

2.2 an experiment on two -l evels b is table amb igui ty 43

ing stable images. Subjects were instructed to repetitively reporttheir conscious experience of the visual stimuli by key-presses,reporting the occurrence of perceptual endogenous reversalsand the occurrence of actual reversals of stable percepts during a"replay condition". Based on previous research we hypothesizedincreased activation in the ventral occipital cortex, in parietal ar-eas, as well as in frontal areas during bistable stimuli comparedwith stable stimuli [ Ilg et al. , 2008 ]; [Kleinschmidt et al. , 1998 ];[Lumer et al. , 1998]; [Sterzer et al. , 2002].

Further we wanted to test the theory of levels of ambigu-ity, verifying if perception of gures with different levels of ambiguity engage different brain areas. To compare levels of Levels of ambiguityambiguity we used two types of bistable images. We denethose images where perceptual ip takes place within the same

category as "intra-categorical". This is the case for the Neckercube (Figure 2.1 a). Here, what appears to be in the front canoccupy a different recessional plane with prolonged viewing, but nevertheless it conceptually remains the same gure. Thiscontrasts with what we refer to as "inter-categorical", namelythe two images created by a single picture belong to differentcategories, as in the face- vase bistable image (Figure 2.1 b). Wedene as "intracategorical" those images in which the percep-tual ip takes place within the same category. In the presentstudy, inter-categorical stimuli alternates between bodies and

faces, since studies have demonstrated the selectivity of different brain regions to the visual representation of faces [ Kanwisherand Yovel , 2006 ] and bodies [ Peelen and Downing , 2007 ]. Wecompared how neural activity during perception of bistableintra-categorical images and bistable inter-categorical imagescontributed to the overall activations during perception of all bistable gures. To our knowledge, this is the rst imagingstudy comparing two levels of ambiguity. In line with [ Zeki ,2004 ] above quoted theory we expected more involvement of higher cognitive areas for the ambiguous inter-categorical im-

ages. As a third approach, we also addressed how brain activityuctuates over time, and if transient patterns of activation arerelated to the type of images perceived. We investigated if sepa- Content of the

perceptrate neural mechanisms are involved in the transition from onepercept to another. The goal was to investigate if two percepts belonging to the same attribute or category, for example the tworecessional planes of the Necker cube, evoke different areas of activation compared to when the transition is from one categoryto another, as in the Rubin face-vase bistable image. We looked


51/116


at neural activation during alternating percepts for both intra-and inter-categorical gures. We hypothesized that same brainarea would be active when the translation would take placewithin images belonging to the same category, while areas of activation would change when the two percepts belong to twodifferent categories. Based on previous ndings, we further hy-pothesized that during reported face perception the FFA wouldrespond stronger [ Andrews et al. , 2002 ]; [Hasson et al. , 2001 ];[Tong et al. , 1998 ], while the same areas would be active whenthe translation would take place within images belonging to thesame category.

Design : We used bistable images to separate, and thereforeExperimental designcompare, perceptual from stimulus-driven changes. In our study,subjects were requested to repetitively report by key-presses

their conscious experience of ips during the observation of bistable gures. Their responses were recorded and the subjective occurrence of perceptual reversals was replayed by al-ternating the two stabilized versions of the same percepts. Weused two sets of bistable images: intra-categorical and inter-categorical. Subjects were instructed, when looking at "geomet-rical gures" (intracategorical stimuli) to alternate key presseswhen spontaneously perceiving a ip. For the intercategoricalambiguous gures, subjects were requested to press a specickey indicating body perception and another key indicating face

perception. During the replay condition, subjects were presentedwith stabilised versions of the ambiguous gures. The onsets of the alternating stabilised pictures were the same as when subjects indicated seeing a ip in the ambiguous condition. Subjectswere also requested to perform key presses during the replaycondition, in order to control for motor responses.

Subjects : 16 healthy subjects (with normal or corrected tonormal vision; 8 females) were recruited through advertisementsrequesting volunteers for a study about optical illusions. Theirage varied from 21 to 40 years (mean 29 ,8 years). Two subjects

were left handed. Informed written consent was obtained fromall participants and the study was covered by the MinimumRisk Ethics (Minimum risk magnetic resonance imaging studiesof healthy human cognition, UCL Ethics Project ID number:1825 / 003 / Data protection ref: Z 6364106 / 2010 / 03/ 04). Duringa rst visit to the laboratory, prior to scanning, each subjectwas requested to do a pre-test in order to qualify for the fMRIexperiment, by performing the same task to be carried out inthe scanner. X participants were excluded because they were not


52/116


able to perform the task correctly. During each scanning sessionsubjects heart-rate and respiration were continuously recorded,providing physiological measurements to be subsequently usedas regressors-of-no-interest in the rst-level SPM analysis foreach subject.

Stimuli : Stimuli were generated using Cogent 2000 and Co-gent Graphics (http://www.vislab.ucl.ac.uk/cogent.php). Fourintra-categorical images were chosen from the images in ThePsychophysics of Form: Reversible-Perspective Drawings of Spa-tial Objects, Hochberg and Brooks, The American Journal of Psychology, Vol. 73, No. 3 (Sep., 1960), pp. 337-354 , University of Illinois Press) and 1 was created manually. Images were selectedfollowing a pre-test in which four subjects viewed 8 images andchose the ones that most easily ipped between two states for

all subjects. All ve intra-categorical ambiguous images weregenerated using Adobe InDesign CS 3. Five intercategorical im-ages were chosen from existing ambiguous gures. The chosenintercategorical images were those displaying two mutuallyexclusive interpretation, a body OR a face. Subsequently, eachambiguous image was modied to create two stable versions,which could be shown successively to the subjects using Photo-shop CS 3, and two stable versions of each image were created(see Figures 2.4 and 2.5 ).

Each subject was exposed to two runs displaying the same

images in the same order. Each run began with a neutral back-ground, lasting 26 s, during which the rst six brain volumeswere discarded to allow T 1 equilibration effects to subside. Thestimulus sequence then began. During each session the 10 ambiguous images ( 5 intracategorical + 5 intercategorical) weredisplayed; subjects were instructed to alternate key presseswhen perceiving a ip from one percept to the other. Withintercategorical ambiguous images, subjects were instructedto press a button to indicate whether it was a body or a facethat they perceived. During each session, ambiguous images

were mixed with a following replay condition of the subjectsperception, displaying the two alternating stabilised versions of each ambiguous image. The replay condition was implementedusing the recorded button presses relative to each ambiguousimage so that the time sequences remained the same. Conse-quently, the onsets of the alternating stabilised pictures werethe same as the alternating perceptual ips indicated by thesubjects. In order to control for motion correction, subjects werealso required to press buttons during the replay condition, each


53/116


time that the image changed. Each epoch lasted 16 s with aninterstimulus interval varying in duration between 3 and 5s between stimuli where a blank grey screen was presented.Stimuli were presented in a pseudo-random sequence, ensuringthat each ambiguous image was presented before its stabilizedversions.

Scanning details : Scans were acquired using a 1.5T SiemensMagneton Sonata MRI scanner tted with a head volume coil(Siemens, Erlangen, Germany) to which an angled mirror wasattached, allowing subjects to view a screen onto which stimuliwere projected using an LCD projector. An echo-planar imag-ing (EPI) sequence was applied for functional scans, measuringBOLD signals (echo time TE = 50 ms, repeat time TR= 90 ms,volume time 4.32 s). Each brain image was acquired in a de-

scending sequence comprising 48

axial slices covering the whole brain. The experiment consisted of 2 runs; 100 volumes were ac-quired per run. After functional scanning had been completed,a T1* weighted anatomical scan was acquired in the sagittalplane to obtain a high resolution structural image ( 176 slices pervolume).

2.2.1 General fMRI Analysis

Analysis: Data were analysed using SPM 8 (http://www.l.ion.ucl.ac.uk/SPM).

The time series of functional brain volume images for each subject was realigned and normalized into MNI space (voxel size3 x 3 x 3 mm) and then smoothed using a Gaussian smooth-ing kernel of 9 mm. The stimulus for each subject was mod-elled as a set of regressors in the SPM 8 general linear model(GLM) (rstlevel) analysis. The stimulus was a block designmerged with an event-related design; boxcar functions wereused to dene regressors which modelled the onsets and dura-tions of each stimulus, as indicated by each subject by meansof keypresses. Consequently, the regressors were: faces, bod-ies, geometrical state 1 and geometrical state 2. Regressors werefurther subdivided in stables versus unstable; stable onsets corre-sponded to actual changes, while unstable onsets correspondedto perceptual changes, as indicated by key-presses. Key-presseswere modelled as delta functions in an additional regressor.Head-movement parameters calculated from the realignmentpre-processing step and physiological data acquired during thescan (heart-rate and respiration) were included as regressors of no interest. Regressors were convolved with the default SPM 8


54/116


canonical hemodynamic response function (HRF), its temporalderivative (TD) and dispersion derivatives (DD). The resultantparameter estimates for each regressor (at each voxel) were com-pared using ttests to establish the signicance of differences inactivation between conditions. We have investigated ve maineffects: Unstable Figures vs Stable Figures; in respect to intercategorical stimuli, Unstable Faces vs baseline and UnstableBodies vs. Baseline; in respect to inter-categorical stimuli, Geo-metrical State 1 vs. Baseline and Geometrical State 2 vs Baseline.Contrast images for these effects for each subject were enteredinto randomeffect analyses at the second level. Conjunction

analysisA conjunction analysis [ Friston et al. , 1999 ] was performedto asses how the two types of bistable images, intra-categoricaland inter-categorical, each contributed to the areas of activation

found when contrasting all bistable with all stabilised gures.Separate contrasts were made for each for both types of g-ures: Bistable intra-categorical vs. Stable intra-categorical, andBistable inter-categorical vs. Stable inter-categorical. To make theconjunctions, the contrast Bistable Figures vs. Stable Figures waspaired separately with the contrasts Bistable intra-categorical vs.Stable intra-categorical, and Bistable inter-categorical vs. Stableinter-categorical. Event-related

analysisFor the even-related part of the analysis we made contrasts forinter-categorical stimuli: Bistable Faces vs. All and Bistable Bod-

ies vs. All, and in respect to inter-categorical stimuli: Bistablestate 1 vs. All and Bistable state 2 vs. All. Regressors wereconvolved with the default SPM 8 canonical hemodynamic re-sponse function (HRF) and a rst-order Taylor approximationin terms of the temporal derivative (TD) was added [ Fristonet al., 1998]. Whole brain t-maps for main effects of interest andfor temporal and dispersion derivatives were created for eachsubject. Contrast images for the main HRF and its TD were com-puted separately for each effect investigated. Then, second levelrandom-effects models were created for each contrast, using the

t-maps from the rst-level xed effects analysis. The onsets of internally and externally driven changes were modelled basedon the recorded key presses and set to a xed duration of onesecond. Perception of faces and bodies as well as the two statesof the intra-categorical gures were also modelled separately based on the recorded key presses. A more complex model foranalysing the event-related results has been chosen; adding theTD to the canonical HRF gave us the possibility to model BOLDsignals with deviations in onsets. The Henson et al. ( 2002 ) have


55/116


demonstrated that it can be useful to model differential latenciesof the HRF as a to investigate if BOLD signal may occur earlieror later then what the canonical parameter estimates.

2.3 r es ult s

2.3.1 Behavioural results

All subjects reported that they were able to see the stimuliduring scanning and alternate their key-presses according to theinstructions. Overall, subjects perceived each type of state for asimilar period of time during presentation of bistable images; bistable faces mean 2.5, 2.35 s., bistable bodies mean 2.1,1.47 s., bistable geometrical 1 mean 2.3, 1.64 s., and bistable

geometrical 2 mean 2.51 , 2.29 s. Subjects indicated perceptdurations ranging from . 02 to 15.12 s. The periods betweenips found here are shorter than what subjects indicated inKleinschmidt et al. ( 1998 ) who had inter-reversal times of 9.0 2.6 s and 8.1 1.9 s, but in line with studies of ambiguousmotion where similar durations of alternating percepts wereobserved ([ Ilg et al., 2008]).Blocked fMRI

analysis: Neuralspecicity of bistable

images

We compared spontaneous and stimulus-driven perceptualswitches. We rst contrasted perception of Bistable Figures withperception of Stable Figures with blocked design analysis. Spon-

taneous perceptual changes were correlated with increased ac-tivations in right inferior and superior parietal lobules, and in bilateral inferior frontal, middle frontal, and insular cortex. In-creased activity was also observed in regions of the anteriorcingulate cortex, supplementary motor area, and left primarymotor and somatosensory cortex. Selective activation duringperceptual transitions were also found in the right extrastriatevisual cortex and the cerebellum, putamen and thalamus (seeFigures 2.6 and 2.7).Conjunction

analysis: comparing

levels of ambiguityA conjunction analysis was performed to provide more in-

sight into how each type of gures contributed to the overallactivation found during perception of bistable stimuli. Severalsimilar areas of activation were identied for overall bistableperception in conjunction with both inter- and intra-categorical bistable perceptions respectively (Figures 2.8 and 2.9 ). Therewere clearly also differences between the two conjunctions: theintra-categorical gures evoked bilateral activation of superiorparietal lobule while inter-categorical gures only showed sig-nicant right activation of this region. The signicant regions


56/116

2.3 results 49

were also noticeably larger for the intra-categorical conjunction, both in frontal and parietal areas. Event-related

results: contrastinginterpretations

We assessed if face perception was correlated with increasedBOLD activation in the FFA, known as a region specicallyinvolved in face processing. We predicted that we would ndpercept specic activation during face perception, which hasalso been demonstrated, in previous studies [ Andrews et al. ,2002 ]; [Hasson et al. , 2001]. As hypothesized, our results showedclear activation in the fusiform gyrus when subjects indicatedperceived faces during inter-categorical stimulus presentation,an activation pattern we did not nd in any of the other contrastswith bistable stimuli. To ensure we did not miss any activationfor the contrast Bistable bodies > All, we did a small volumecorrection for the contrast Bistable bodies > All with a 16 mm

sphere at [38

-58

-14

] which was identied as the extrstriate bodyarea (EBA) [ Downing et al. , 2001 ]; no signicant voxels wererevealed. For the intra-categorical stimuli we did expect to haveactivations in the same areas for both Bistable state 1 > All andBistable state 2 > All. Our results showed signicant activationonly for Bistable state 2 in right middle occipital gyrus. We did asmall volume correction to test if there was possible activationin the same area for State 1 with a 16 mm sphere at [ 36 -85 7].The small volume search revealed a highly signicant cluster(x, y, z = 36 , -82 , 7, Z = 4.79). The central interest of the event-

related part of this study was to investigate if activation wouldchange between the two perceptual states for both inter- andintra-categorical bistable images. For the inter-categorical imagesthere was a correlation between face perception and activation inthe fusiform face area, which was not found during perceptionof bodies. For both states of the intra-categorical gures weidentied signicant activation in the same area, the middleoccipital gyrus. The middle occipital gyrus has previously beenassociated with spatial attention [ Noesselt et al. , 2002]. All event-related contrasts showed deactivations in primary visual regions

in line with previous ndings by [ Kleinschmidt et al. , 1998 ],where deactivations in occipital areas also were found during bistable perception.

The contrast estimates showed that the main canonical HRFaccounted for very little of the activation (see gure 2.10 for Latencya representative example), while the TD is much larger andpositive which shows that activation actually takes place earlierthan the events [ Friston et al. , 1998]. For the deactivations in theoccipital regions the TD was negative, indicating that suppres-


57/116


sion might take place just before upcoming perceptual switches.Also for the deactivations, the main signal modelled with thecanonical HRF was much lower compared to the estimate of theTD. This could indicate that the deactivations are more transient,and may occur in the transition between perceptual states ratherthan in the period when the percept is temporarily stable.

2.4 d is cuss ion

The aim of the rst part of our study was to compare spon-Neural specicity of bistable images taneous perceptual reversals with externally driven changes

in terms of neural activity. In accordance with previous imag-ing studies using bistable percepts [ Kleinschmidt et al. , 1998 ];[Lumer et al. , 1998 ], we observed activations in several frontal

and parietal areas. As expected we did not nd any notableactivations in the primary visual system because the image washeld constant. It is unlikely that our results reect the motor task, because subjects were doing the same key presses during both bistable and stabilised stimuli presentation. The frontoparietalactivations could reect a continuous loop of communicationfrom the visual cortex to higher-order areas. One interpretationis that these areas are involved in a feedback to early visual areas,re-evaluating the multistable percept over: this would supporta top-down explanation. Alternatively, changes are driven by

signals from lower perceptual areas due to destabilization of the current percept, in line with the satiation / bottom-up hy-pothesis. The role of the frontoparietal attention network wouldthen be to detect changes or to momentarily stabilise the cur-rent interpretation. The frontal and parietal regions identiedin our study are similar to the areas found during both volun-tary reversals of bistable images and changes in visual spatialattention [ Slotnick et al. , 2003 ]. In the current study, it is notlikely that frontoparietal regions reect voluntary changes inattention because we specically instructed the subjects not totry to voluntary elicit the ips, but to signal report reversals.The involvement of these regions in this context suggests a cou-pling between spatial visual attention and dynamic changes of visual perception. It is still debatable if these activations reectinitiation or detection of changes in interpretation. We specu-late that the brain is constantly engaged in interpretation andrevaluation of perceptual input, and during this process atten-tion is shifted to different features of the gure. These changesof attended local features could initiate switches. Our expla-


58/116

2.4 discussion 51

nation corresponds to the theoretical framework suggested by[Leopold and Logothetis , 1999], and is supported by behavioralresults showing that attending to different local features of anambiguous gure bias perception [ Peterson and Gibson , 1991 ];[Tsal and Kolbet , 1985 ]. The pre-central gyrus has found to beinvolved in selective processing of relevant visual targets ratherthan directing attention towards a cued target [ Hopnger et al. ,2000 ]. The area also contains the frontal eye elds, an area alsofound in the study by [ Kleinschmidt et al. , 1998 ]. In our studya xation cross was not used, so activation could be due toreversal-related changes of gaze or covert shifts of attention. Sev-eral areas in the cerebellum were identied when contrastingwhole periods of bistable perception with stabilised replays. Thecerebellum is traditionally associated with movement at speech

function, but recent studies also show involvement in mentalactivities such as attention, error detection (see [ Ito, 2008 ] for areview). It has also been demonstrated that the posterior partof cerebellum (lobule VII, crus I) supplies temporal informationto frontoparietal spatial attention network involved in visualattention [ OReilly et al. , 2008 ]. Involvement of the posteriorcerebellum was more active when subjects had to predict thetrajectory of an occluded moving object. In our study, it may bepossible that the cerebellum predicts the upcoming dominantvisual percept.

In the conjunction analysis, larger parts of the overall activa- Levels of ambiguitytions during bistable perception were accounted for by activationduring viewing of intra-categorical gures. There may be severalways to interpret this nding. Our initial hypothesis, in line with[Zeki , 2004], was that reversing gures represent a very simpleform of ambiguity and are not affected by top-down factors,while face/body gures should trigger an activation in frontalareas (top-down). Zekis hypothesis seems to contrast the factthat we found larger areas of frontal

classicthesis.pdf

Documents