001 058

002 059

003 060

004 061

005 062

006 063

007 064

008 065

009 066

010 067

011 068

012 069

013 070

014 071

015 072

016 073

017 074

018 075

019 076

020 077

021 078

022 079

023 080

024 081

025 082

026 083

027 084

028 085

029 086

030 087

031 088

032 089

033 090

034 091

035 092

036 093

037 094

038 095

039 096

040 097

041 098

042 099

043 100

044 101

045 102

046 103

047 104

048 105

049 106

050 107

051 108

052 109

053 110

054 111

055 112

056 113

057 114

ORIGINAL RESEARCH ARTICLEpublished: xx February 2012

doi: 10.3389/fpsyg.2012.00043

Free-energy and illusions: the Cornsweet effectHarriet Brown* and Karl J. Friston

The Wellcome Trust Centre for Neuroimaging, University College London Queen Square, London, UK

Edited by:

Lars Muckli, University of Glasgow,UK

Reviewed by:

Gerrit W. Maus, University ofCalifornia at Berkeley, USAPeter De Weerd, MaastrichtUniversity, Netherlands

*Correspondence:

Harriet Brown, Wellcome Trust Centrefor Neuroimaging, Institute ofNeurology Queen Square, LondonWC1N 3BG, UK.e-mail: harriet.brown.09@ucl.ac.uk

In this paper, we review the nature of illusions using the free-energy formulation of Bayesianperception. We reiterate the notion that illusory percepts are, in fact, Bayes-optimal andrepresent the most likely explanation for ambiguous sensory input. This point is illustratedusing perhaps the simplest of visual illusions; namely, the Cornsweet effect. By usingplausible prior beliefs about the spatial gradients of illuminance and reflectance in visualscenes, we show that the Cornsweet effect emerges as a natural consequence of Bayes-optimal perception. Furthermore, we were able to simulate the appearance of secondaryillusory percepts (Mach bands) as a function of stimulus contrast. The contrast-dependentemergence of the Cornsweet effect and subsequent appearance of Mach bands weresimulated using a simple but plausible generative model. Because our generative modelwas inverted using a neurobiologically plausible scheme, we could use the inversion as asimulation of neuronal processing and implicit inference. Finally, we were able to verify thequalitative and quantitative predictions of this Bayes-optimal simulation psychophysically,using stimuli presented briefly to normal subjects at different contrast levels, in the contextof a fixed alternative forced choice paradigm.

Keywords: free-energy, perception, Bayesian inference, illusions, Cornsweet effect, perceptual priors

INTRODUCTIONIllusions are often regarded as “failures” of perception; however,Bayesian considerations often provide a principled explanation forapparent failures of inference in terms of prior beliefs. This paperis about the nature of illusions and their relationship to Bayes-optimal perception. The main point made by this work is thatillusory percepts are optimal in the sense of explaining sensationsin terms of their most likely cause. In brief, illusions occur when theexperimenter generates stimuli in an implausible or unlikely way.From the subject’s perspective, these stimuli are ambiguous andcould be explained by different underlying causes. This ambiguityis resolved in a Bayesian setting, by choosing the most likely expla-nation, given prior beliefs about the hidden causes of the percept.This key point has been made by many authors (e.g., Purves et al.,1999). Here, we develop it under biologically realistic simulationsof Bayes-optimal perception and try to make some quantitativepredictions about how subjects should make perceptual decisions.We then try to establish the scheme’s validity by showing that thesepredictions are largely verified by experimental data from normalsubjects viewing the same stimuli.

The example we have chosen is the Cornsweet illusion, whichhas a long history, dating back to the days of Helmholtz (Mach,1865; O’Brien, 1959; Craik, 1966; Cornsweet, 1970). This is partic-ularly relevant given our formulation of the Bayesian brain is basedupon the idea that the brain is a Helmholtz or inference machine(von Helmholtz, 1866; Barlow, 1974; Dayan et al., 1995; Fristonet al., 2006). In other words, the brain is trying to infer the hiddencauses and states of the world generating sensory information,using predictions based upon a generative model that includesprior beliefs. We hoped to show that the Cornsweet effect canbe explained in a parsimonious way by some simple prior beliefs

about the way that visual information is generated at differentspatial and temporal scales.

THE CORNSWEET EFFECT AND THE NATURE OF ILLUSIONSFigure 1 provides an illustration of the Cornsweet illusion. Theillusion is the false percept that the peripheral regions of a stimulushave a different brightness, despite the fact they are physically iso-luminant. This illusion is induced by a biphasic luminance “edge”in the centre of the field of view (shown in the right hand col-umn of Figure 1). The four rows of Figure 1 show the Cornsweeteffect increasing in magnitude as we increase the contrast of thestimulus. Interestingly, at high levels of contrast, secondary illu-sions – Mach bands (Mach, 1865; Lotto et al., 1999) – appear atthe para-central points of inflection of the true luminance profile.It is this contrast-dependent emergence of the Cornsweet effectand subsequent Mach bands that we wanted to simulate, underthe assumption that perception is Bayes-optimal.

The Bayesian aspect of perception becomes crucial when weconsider the nature of illusions. Bayesian theories of perceptiondescribe how sensory data (that have a particular likelihood) arecombined with prior beliefs (a prior distribution) to create a per-cept (a posterior distribution). One can regard illusory perceptsas those that are induced by ambiguous stimuli, which can becaused in different ways – in other words, the probability of thedata given different causes or explanations is the same. When facedwith these stimuli, the prior distribution can be used to create aunimodal posterior and an unambiguous percept. If the perceptor inference about the hidden causes of sensory information (theposterior distribution) is different from the true causes used togenerate stimuli, the inference is said to be illusory or false. How-ever, with illusory stimuli the mapping of hidden causes to their

www.frontiersin.org February 2012 | Volume 3 | Article 43 | 1

115 172

116 173

117 174

118 175

119 176

120 177

121 178

122 179

123 180

124 181

125 182

126 183

127 184

128 185

129 186

130 187

131 188

132 189

133 190

134 191

135 192

136 193

137 194

138 195

139 196

140 197

141 198

142 199

143 200

144 201

145 202

146 203

147 204

148 205

149 206

150 207

151 208

152 209

153 210

154 211

155 212

156 213

157 214

158 215

159 216

160 217

161 218

162 219

163 220

164 221

165 222

166 223

167 224

168 225

169 226

170 227

171 228

Brown and Friston Free-energy and illusions: the Cornsweet effect

FIGURE 1 |The Cornsweet illusion and Mach bands. The Cornsweetillusion is the false perception that the peripheral regions of a Cornsweetstimulus have different reflectance values. The magnitude of the effectincreases as the contrast of the stimulus increases. At higher levels ofcontrast, the secondary illusion – Mach bands – appear. The Mach bandsare situated at the point of inflection of the luminance gradient.

sensory consequences is ill-posed (degenerate or many to one),such that a stimulus can have more than one cause. Thus, fromthe point of view of the observer, there can be no “false” infer-ence unless the true causes are known. The perceptual inferencecan be optimal in a Bayesian sense, but is still illusory. However,not all possible causes of sensory input will be equally likely, sothere will be an optimal inference in relation to prior beliefs abouttheir causes. Prior beliefs can be learnt or innate: priors that arelearnt depend upon experience while innate priors can be asso-ciated with architectural features of the visual brain, such as thecomplex arrangement of blobs, interblobs, and stripes in V1, thatmay reflect priors on the statistical structure of visual informationselected by evolutionary pressure.

Prior beliefs are essential when resolving ambiguity or the ill-posed nature of perceptual inverse problems. Put simply, there willalways be an optimal posterior estimate of what caused a sensationthat rests upon prior beliefs. The example in Figure 2 illustratesthis: The central panel shows an ambiguous stimulus (luminanceprofile) that is formally similar to the sort of stimulus that inducesthe Cornsweet illusion. However, this stimulus can be caused in

FIGURE 2 | Contributions to luminance. Luminance values reaching theretina are modeled as a multiplication of illuminance from light sources andreflectance from the surfaces in the environment. The factorization ofluminance is thus an ill-posed problem. One toy example of this degeneracyis shown here; the same stimulus can be produced by (at least) twopossible combinations of illuminance and reflectance. Prior beliefs aboutthe likelihood of these causes can be used to pick the most likely percept.

an infinite number of ways. We have shown two plausible causesby assuming the stimulus is the product of (non-negative) illu-minance and reflectance profiles. The lower two panels show the“true” causes generating stimuli for the Cornsweet illusion. Here,the stimulus has a reflectance profile that reproduces the Corn-sweet stimulus and is illuminated with a uniform illuminant. Analternative explanation for exactly the same stimulus is providedin the upper two panels, in which two isoreflectant surfaces areviewed under a smooth gradient of ambient illumination. In thisexample, we have ensured that both the illuminant and reflectanceare non-negative by applying an exponential transform beforemultiplying them to generate the stimulus.

The key point made by Figure 2 is that there are many pos-sible gradients of illuminance and reflectance that can producethe same pattern of sensory input (luminance). These differ-ent explanations for a particular stimulus can only be distin-guished by priors on the spatial and temporal characteristics ofthe reflectance and illuminance. In this example, the ambiguityabout what caused the stimulus can be resolved if we believe,a priori, that the visual world is composed of isoreflectant sur-faces, as opposed to surfaces that (implausibly) get brighter ordarker nearer their edges or occlusions (as in the lower panels).Under this prior assumption, an observer who infers the pres-ence of spatially extensive isoreflectant surfaces, and explains theedge at the centre with an illuminance gradient, would be infer-ring its most likely cause. The Cornsweet “illusion” is thus only anillusion because the experimenter has chosen an unlikely combi-nation of illuminance and reflectance profiles. In what follows, wewill exploit priors on the spatial composition and generation of

Frontiers in Psychology | Perception Science February 2012 | Volume 3 | Article 43 | 2

229 286

230 287

231 288

232 289

233 290

234 291

235 292

236 293

237 294

238 295

239 296

240 297

241 298

242 299

243 300

244 301

245 302

246 303

247 304

248 305

249 306

250 307

251 308

252 309

253 310

254 311

255 312

256 313

257 314

258 315

259 316

260 317

261 318

262 319

263 320

264 321

265 322

266 323

267 324

268 325

269 326

270 327

271 328

272 329

273 330

274 331

275 332

276 333

277 334

278 335

279 336

280 337

281 338

282 339

283 340

284 341

285 342

Brown and Friston Free-energy and illusions: the Cornsweet effect

visual input to simulate the Cornsweet effect and the emergenceof Mach bands.

The Bayesian approach to visual perception has been exploitedin previous work (Yuille et al., 1991; Knill and Pouget, 2004).In addition, several other visual illusions have been explainedusing Bayesian principles, including motion illusions (Weiss et al.,2002), the sound-induced flash illusion (Shams et al., 2005), andthe Chubb illusion (Lotto and Purves, 2001). Additionally, Purveset al. (1999) demonstrated the Bayesian nature of the Cornsweetillusion: when presented in a context implying an illuminance gra-dient and reflectance step, the Cornsweet illusion is elicited moreeasily.

In terms of the neuronal systems mediating the Cornsweetillusion; some authors have implicated subcortical structures: forexample, Anderson et al. (2009) found that BOLD signal in thelateral geniculate nuclei (LGN) best correlated with perceptionof the Cornsweet illusion, although correlations were also seenin visual cortex. Furthermore, the illusion could be abolished ifthe stimulus was not presented binocularly, suggesting an originbefore V1. Mach bands similarly have been attributed to retinalmechanisms (e.g., Ratliff, 1965); however, Lotto et al. (1999) havesuggested a high-level contextual explanation for their appearance.Irrespective of the cortical or subcortical systems involved, we willassume, in this paper that the same Bayesian principles operateand, crucially, rest on a hierarchical generative model that neces-sarily implicates distributed neuronal processing at the subcorticaland cortical levels.

OVERVIEWThis paper comprises three sections. The first describes a simplegenerative model of visual input that entails prior beliefs abouthow visual stimuli are generated and can be used to infer theircauses. This model is used in the second section to simulatethe perception of the Cornsweet illusion and contrast-dependentemergence of Mach bands. In the third section, we test the pre-dictions of the simulations using a psychophysics study of normalsubjects.

A GENERATIVE MODEL FOR THE CORNSWEET EFFECTOur simulations are based upon the free-energy formulation ofBayes-optimal perception. Put briefly, this is based upon thenotion that self-organizing agents minimize the average surprise(entropy) of sensory inputs through minimizing a free-energybound on surprise. Here, surprise is just the improbability ofsampling some sensory information, in relation to a (generative)model of how those sensations were produced. By adjusting thefree parameters of the model, the sensory information can beexplained or predicted and surprise minimized. Mathematically,surprise is the (negative) log evidence for a model of the worldthat comprises hidden variables (causes and states) that generatesensory information. We have described in many previous publica-tions how this principle leads to active inference and Bayes-optimalperception (Friston et al., 2006; Friston, 2009; Feldman and Fris-ton, 2010). Free-energy is a function of sensory samples and aprobabilistic representation of what caused those samples. Thisrepresentation can be cast in terms of the most likely or expectedstates of the world, under a generative model of how they conspire

to produce sensory inputs. In brief, once we know the agent’s gen-erative model, one can use the free-energy principle to predict itsbehavior and perception. In the present context, our focus willbe on perception and the role of prior beliefs that are an inher-ent part of the generative model. In what follows, we describethe model and then use it to simulate perceptual inference andelectrophysiological responses.

THE GENERATIVE MODELThe generative model we used is straightforward: sensory inputis the product of reflectance and illuminance, where illuminancevaries smoothly over space but can fluctuate with a high frequencyover time. Conversely, the reflectance profile of the visual world iscaused by isoreflectant fields or surfaces that fluctuate smoothlyin time. Crucially, the spatial scales over which these fluctua-tions occur have a scale-free nature, of the sort found in naturalimages (Burton and Moorhead, 1987; Field, 1987; Tolhurst et al.,1992; Ruderman and Bialek, 1994; Ruderman, 1997). To ensurepositivity of the illuminant and reflectance we apply an expo-nential transform to the two factors before multiplying them (asin Figure 2). Equivalently, we can imagine the underlying causes(reflectance and illuminance) as being composed additively in log-space. This model is shown schematically in Figure 3, in termsof hidden causes and states. Mathematically, this model can beexpressed as:

s = g (x , v) = exp (R · x + I · vI ) + ωs

x = f (x , v) = vR − x + ωx(1)

Here, s(t ) are sensory signals generated from hidden statesx(t ) and causes, v(t ) plus some random fluctuations ω(t ). Thedifference between hidden causes and states is that states evolvedynamically, in response to perturbations by hidden causes –these dynamics are described by the equation of motion in thesecond equality above. Hidden causes v(t ) = (vI, vR) have beendivided into those causing changes in luminance and those causingchanges in hidden states that produce changes in reflectance. Thesehidden variables control the amplitude of spatial basis functions(R, I) encoding formal beliefs about the spatial scales of illumi-nance and reflectance. For the illuminant (I) we use a low-orderdiscrete cosine transform, while for the reflectant (R) we use alow-order discrete wavelet transform.

The particular wavelet transform used here is a Haar waveletset that has been thinned by removing high-order wavelets (withhigh spatial frequency) from the periphery of the visual field: Thisrespects, roughly, the increasing size of classical receptive fieldswith retinotopic eccentricity. For simplicity (and ease of report-ing the results), we restrict the simulations to a one-dimensionalvisual field. Because Haar wavelets afford local linear approxima-tions to continuous reflectance profiles, the resulting reflectancehas to be a mixture of isoreflectant surfaces at different spatialscales. To impose the scale-free aspect, we decrease the varianceor, equivalently, increase the precision of the reflectance waveletcoefficients or hidden states in proportion to the order or spatialscale of their wavelet. This is implemented by placing a prior onthe wavelet coefficients with the form p(xk) = N (0, e−3k ), where kis the order of the wavelet. Neuronally, these basis functions could

www.frontiersin.org February 2012 | Volume 3 | Article 43 | 3

343 400

344 401

345 402

346 403

347 404

348 405

349 406

350 407

351 408

352 409

353 410

354 411

355 412

356 413

357 414

358 415

359 416

360 417

361 418

362 419

363 420

364 421

365 422

366 423

367 424

368 425

369 426

370 427

371 428

372 429

373 430

374 431

375 432

376 433

377 434

378 435

379 436

380 437

381 438

382 439

383 440

384 441

385 442

386 443

387 444

388 445

389 446

390 447

391 448

392 449

393 450

394 451

395 452

396 453

397 454

398 455

399 456

Brown and Friston Free-energy and illusions: the Cornsweet effect

FIGURE 3 |The generative model: the generative model employed in thispaper models illuminance as a discrete cosine function and reflectance as aHarr wavelet function with peripheral high frequency wavelets removed. Inaddition, illumination is allowed to change quickly over time, whereasreflectance varies more slowly. This is achieved by making the coefficients on

the reflectance basis functions hidden states, which accumulate hiddencauses to generate changes in reflectance. Inversion of the model providesconditional estimates of the hidden causes and states responsible forsensory input as a function of time. See main text for an explanation of thevariables in this figure.

stand in for a filling-in process such as that described by Gross-berg and Hong (2006). Conversely, the illuminant is modeled asa mixture of smoothly varying cosine functions with a low spatialfrequency. This is easily motivated by the fact that most sourcesof illumination are point sources, which results in smooth illu-minance profiles. These were modeled here with the first threecomponents of a discrete cosine transform (see Figure 4 for agraphical representation of the basis functions and how they areused to generate a stimulus).

By construction, this generative model of visual signals sep-arates the spatial scales or frequencies of the illuminance andreflectance such that all the high frequency components are inthe reflectance profile, while the low frequency components are inilluminance profile. Temporal persistence of reflectance is assuredbecause the reflectance coefficients x(t ) ∈ �16 × 1 are hidden statesthat accumulate hidden causes vR(t ) ∈ �16 × 1. This persistencereflects the prior belief that surfaces move in a continuous fashion.For simplicity, we mapped the hidden causes controlling illumi-nance vI(t ) ∈ �3 × 1 directly to the stimulus (although this is not animportant feature of our model). This can be thought of as accom-modating rapid changes in illuminance of the sort that might beproduced by a flickering candle.

Equation 1 defines our generative model in terms of the jointprobability over sensory information and the hidden variablesproducing fluctuations in reflectance and illuminance. The fluc-tuations in the hidden causes are assumed to be Gaussian with aprecision (inverse variance) of one, while the fluctuations in the

FIGURE 4 |Two-dimension example. The top panels show 2-D examplesof luminance (left) and reflectance (right), created form the generativemodel. The resulting stimulus is the product of the two (bottom panel).

motion of the hidden states are assumed to have a log-precisionof 12. Finally, we assume sensory fluctuations or noise with alog-precision of six. In the next section, we will manipulate the

Frontiers in Psychology | Perception Science February 2012 | Volume 3 | Article 43 | 4

457 514

458 515

459 516

460 517

461 518

462 519

463 520

464 521

465 522

466 523

467 524

468 525

469 526

470 527

471 528

472 529

473 530

474 531

475 532

476 533

477 534

478 535

479 536

480 537

481 538

482 539

483 540

484 541

485 542

486 543

487 544

488 545

489 546

490 547

491 548

492 549

493 550

494 551

495 552

496 553

497 554

498 555

499 556

500 557

501 558

502 559

503 560

504 561

505 562

506 563

507 564

508 565

509 566

510 567

511 568

512 569

513 570

Brown and Friston Free-energy and illusions: the Cornsweet effect

log-precision of the sensory noise as a proxy for changing thecontrast of the stimulus.

Figure 4 shows a snapshot of the sort of visual signals this gen-erative model produces. Here, we have used the outer product ofthe discrete transforms above to generate a two-dimensional stim-ulus. We are not pretending that this is a veridical model of thereal visual world. However, it is sufficient to explain the Cornsweetillusion and related effects by incorporating simple and plausiblepriors on the spatial scales over which illuminance and reflectancechange. In the next section, we use this generative model to sim-ulate perceptual and physiological responses to a stimulus, underthe free-energy principle. This reduces to a Bayesian deconvolu-tion of sensory input that tries to discover the most likely hiddencauses and states generating that input.

PERCEPTION AND PREDICTIVE CODINGThis perceptual deconvolution can be regarded as the inversion ofa generative model that maps from hidden causes (variables in theworld) to sensory consequences. The inverse mapping corresponds

to inferring those variables by mapping back from the sensoryconsequences to the hidden causes and states. This can be imple-mented in a biologically plausible fashion using a generalizedgradient descent on variational free-energy F(s, μ); which is afunction of (generalized) sensory states s(t ) = (s, s ′, s ′′, . . .) andthe expected values μ(t ) = (μx, μv) of hidden variables (see Fris-ton, 2008 for details). In brief, this gradient descent correspondsto a Bayesian filtering, in which expected states of the world arecontinuously optimized using a prediction term and an updateterm:

˙μ = Dμ − ∂

∂μF(s, μ) = Dμ − ∂ εT

∂μξ (2)

Under the simplifying assumption that probabilities are repre-sented as Gaussian densities, this can be regarded as a generalizedform of Kalman filtering, where the second (update or gradient)term can be expressed as a mixture of prediction errors (see theequations in Figure 5). This means that the generalized filtering

FIGURE 5 | Hierarchical message-passing in the brain. The schematicdetails a neuronal architecture that optimizes the conditional expectations ofhidden variables in hierarchical models of sensory input of the sortillustrated in Figure 3. It shows the putative cells of origin of forward drivingconnections that convey prediction error from a lower area to a higher area(red arrows) and non-linear backward connections (black arrows) thatconstruct predictions (Mumford, 1992; Friston, 2008). These predictions tryto explain away (inhibit) prediction error in lower levels. In this scheme, thesources of forward and backward connections are superficial and deeppyramidal cells (triangles) respectively, where state-units are black anderror-units are red. The equations represent a generalized gradient descenton free-energy (see main text) using the generative model of the previousfigure. If we assume that synaptic activity encodes the conditionalexpectation of states, then recognition can be formulated as a gradientdescent on free-energy. Under Gaussian assumptions, these recognitiondynamics can be expressed compactly in terms of precision weightedprediction errors: ξ(i ) :i = s, x, v on the sensory input, motion of hidden

states, and the hidden causes. The ensuing equations suggest two neuronalpopulations that exchange messages; with state-units encoding conditionalpredictions and error-units encoding prediction error. Under hierarchicalmodels, error-units receive messages from the state-units in the same leveland the level above; whereas state-units are driven by error-units in thesame level and the level below. These provide bottom-up messages thatdrive conditional expectations μ(i ) :i = x, v toward better predictions toexplain away prediction error. These top-down predictions correspond tog := g(μ(x), μ(v )) that are specified by the generative model. This schemesuggests the only connections that link levels are forward connectionsconveying prediction error to state-units and reciprocal backwardconnections that mediate predictions. Note that the prediction errors thatare passed forward are weighted by their precisions: Π(i ) :i = s, x, v.Technically, this corresponds to generalized predictive coding because it is afunction of generalized variables, which are denoted by a (∼), such thatevery variable is represented in generalized coordinates of motion: forexample: x = (x , x ′, x ′′, . . .). See Friston (2008) for further details.

www.frontiersin.org February 2012 | Volume 3 | Article 43 | 5

571 628

572 629

573 630

574 631

575 632

576 633

577 634

578 635

579 636

580 637

581 638

582 639

583 640

584 641

585 642

586 643

587 644

588 645

589 646

590 647

591 648

592 649

593 650

594 651

595 652

596 653

597 654

598 655

599 656

600 657

601 658

602 659

603 660

604 661

605 662

606 663

607 664

608 665

609 666

610 667

611 668

612 669

613 670

614 671

615 672

616 673

617 674

618 675

619 676

620 677

621 678

622 679

623 680

624 681

625 682

626 683

627 684

Brown and Friston Free-energy and illusions: the Cornsweet effect

in Eq. 2 to corresponds to a generalized form of predictive coding.Predictive coding has become a popular metaphor for understand-ing perceptual inference in the visual system. For example, Raoand Ballard (1999) used predictive coding to provide a compellingexplanation for extraclassical receptive field effects in striate cortex.

Put simply, in these simulations we assume that neural activ-ity corresponds to the brain’s representation of the most likelyvalues of the hidden causes and states (hidden variables) andthat these are continuously updated to minimize free-energy. Theensuing scheme has been discussed in terms of recurrent message-passing among different cell populations in hierarchical sensorycortex: see Figure 5 and Mumford (1992). This scheme rests uponthe use of bottom-up prediction errors to optimize conditionalestimates of hidden variables. These estimates are then used toproduce top-down predictions that are compared with sensoryinput to form a bottom-up prediction error. In this context, thesum of squared prediction error can be regarded as free-energy.The recursive message-passing used in these schemes tries to min-imize prediction error, such that the predictions approximate thetrue conditional or posterior estimates of the underlying hiddencauses. It is this message-passing that we will stimulate in thenext section and associate with neuronal responses, while usingwhat they represent to predict how real subjects would respondbehaviorally, in terms of their perceptual decisions or inference.

To simulate these responses we simply integrate or solve Eq. 2,using the functions g (x, v) and f(x, v) specified by a generativemodel in Eq. 1. These functions map hidden variables to sen-sory input and encode prior beliefs about the dynamics of hiddenstates. In short, by plugging the equations of our generative modelin Figure 3 into the predictive coding scheme of Figure 5, we cansimulate Bayes-optimal inference about the causes of sensations.Crucially, we can then reconstitute the posterior or conditionalbeliefs about these causes and associate these with percepts. Inparticular, we can take any mixture of the hidden variables andassess the posterior belief about that mixture. We will use thisto quantify the Cornsweet and Mach band percepts, in terms ofreflectance differences among different parts of the visual field.Note that the predictive coding scheme in Figure 5 weights theprediction errors by precision matrices. For example, the preci-sion of sensory signals is Π(s) = I ·exp(γ). These precisions arefunctions of log-precisions γ that encode the expected amplitudeof random fluctuations.

SIMULATED RESPONSESThe simulated responses in Figure 6 were obtained by presentingthe Cornsweet stimulus under uniform illumination. Here, thestimulus was presented transiently by modulating the illuminationwith a Gaussian envelope over time (see image inset). The result-ing predictions are shown in Figure 6A as solid lines, while the reddotted lines correspond to the prediction error. These predictionsare based upon the inferred hidden states and causes shown on theright and lower left respectively. The lines correspond to the poste-rior expectations and the gray regions correspond to 90% Bayesianconfidence intervals. In terms of the underlying causes, the bluecurve in Figure 6B is an estimate of the (log) amplitude of uniformillumination. This should have a roughly quadratic form (giventhe Gaussian envelope), peaking at around bin 30, which indeed

FIGURE 6 | Predictions of the model. (C) Shows the estimates of thehidden states (coefficients of the reflectance basis functions) over time.The hidden state controlling the amplitude of the lowest-frequency basisfunction, which corresponds to the Cornsweet percept, contributessubstantially to the overall perception of the stimulus (green line). Theestimates for the hidden causes are shown in (B). The gray areas are 90%confidence intervals. (A) Shows predictions (solid lines) of sensory inputbased on the estimated hidden causes and states and the resultingprediction error (dotted red lines). The insert on the upper left shows thetime-dependent luminance profile used in this simulation. Please see maintext for further details.

it does. The remaining causes that deviate from zero (Figure 6C)are the perturbations to the hidden states explaining or predictingchanges in reflectance. These drive increases or decreases in theconditional expectations of the hidden states shown on the right.The green line is the coefficient of the second-order basis functionsplitting the visual field into an area of brightness on the left anddarkness on the right. It can be seen that at the point of maximumillumination, there is an extremely high degree of confidence thatthis hidden state is bigger than zero. This is the Cornsweet percept.

The corresponding percepts in sensory space are shown inFigure 7 as a function of peristimulus time. The upper panelsshow the implicit reflectance and illuminance profiles encodedby the conditional expectations of hidden variables respectively.After an exponential transform (and multiplication) these pro-duce the sensory predictions shown on the lower left. By taking aweighted mixture of the perceived reflectance in different regionsof the visual field (shown by the white circles) one can estimate theconditional certainty about both the Cornsweet effect (differencesin perceived reflectance on different sides) and the appearanceof Mach bands (differences in perceived reflectance on the sameside). The weights used to evaluate these mixtures are denoted as

Frontiers in Psychology | Perception Science February 2012 | Volume 3 | Article 43 | 6

685 742

686 743

687 744

688 745

689 746

690 747

691 748

692 749

693 750

694 751

695 752

696 753

697 754

698 755

699 756

700 757

701 758

702 759

703 760

704 761

705 762

706 763

707 764

708 765

709 766

710 767

711 768

712 769

713 770

714 771

715 772

716 773

717 774

718 775

719 776

720 777

721 778

722 779

723 780

724 781

725 782

726 783

727 784

728 785

729 786

730 787

731 788

732 789

733 790

734 791

735 792

736 793

737 794

738 795

739 796

740 797

741 798

Brown and Friston Free-energy and illusions: the Cornsweet effect

FIGURE 7 |The model’s “perceptions.”The upper panels show thepredicted illuminance (left) and reflectance profiles (right), reconstructedfrom the coefficients of the basis functions estimated from the modelinversion shown in the previous figure. An inferred reflectance profiledemonstrating the Cornsweet illusion is apparent, but at this level ofcontrast, Mach bands have not yet appeared. Please see main text furtherdetails.

W corn and W mac for the Cornsweet and Mach band effects respec-tively. The conditional expectation of these mixtures or effectsμmac =W mac·μ(x) and their confidence intervals are shown onthe lower right. At this level of visual contrast or precision (alog-precision of six), the Cornsweet effect is clearly evident witha high degree of certainty, while the confidence interval for theMach band effect always contains zero. In other words, at this con-trast (sensory precision) there is a Cornsweet effect but no Machband effect. In the next section, we repeat the simulation aboveand record the conditional expectations (and confidences) aboutillusory effects at the point of maximum illumination for differentlevels of contrast.

CONTRAST OR PRECISION-DEPENDENT ILLUSORYPERCEPTSUsing the generative model and inversion scheme described above,we repeated the simulations over different levels of sensory pre-cision. This can be regarded as a manipulation of contrast inthe following sense: If we assume that the brain uses divisivenormalization (Weber, 1846; Fechner, 1860; Craik, 1938; Geislerand Albrecht, 1992; Carandini and Heeger, 1994), the key changein sensory information, following an increase in contrast, is anincrease in signal to noise; in other words, its precision increases(see the appendix of Feldman and Friston, 2010 for details). Weuse therefore a manipulation of the log-precision of sensory noiseto emulate changes in visual contrast. It should be noted that wedid not actually add sensory noise to the stimuli. The key quantityhere is the level of precision assumed by the agent which, in these

simulations, we changed explicitly. In more realistic simulations,the log-precision would be itself optimized with respect to free-energy (see Feldman and Friston, 2010 for an example of this inthe modeling of attention).

Figure 8 shows the results of perceptual inference under theBayesian scheme described above. The only thing that we changedwas the log-precision of the sensory input, from minus two (low)through to intermediate levels and ending with a very high log-precision of 16. The two graphs show the conditional expectationand 90% confidence intervals for the Cornsweet effect (upperpanel) and Mach bands (lower panel) respectively, at the pointof maximum illumination. It can be seen in both instances thatunder low levels of contrast (sensory precision) both effects arevery small and inferred with a large degree of uncertainty. How-ever, as contrast increases, conditional uncertainty reduces and,at a critical level, produces a confident inference that the effect isgreater than zero (or some small threshold). Crucially, the pointat which this happens for the Cornsweet effect is at a lower level ofcontrast than for the Mach bands. In other words, the Cornsweetillusion occurs first and then the Mach bands appear as contrastcontinues to rise. The explanation for this is straightforward; theMach band illusion rests upon higher spatial frequencies in thegenerative model, which have a higher prior precision (encodingprior beliefs about the statistical – scale-free – structure of naturalvisual scenes). This means that there needs to be precise sensoryevidence to change them from their prior expectation of zero. Inshort, at high levels of contrast or sensory precision, more andmore fine detail in the posterior percept is recruited to providethe optimum explanation for the stimulus. Interestingly, as thecontrast or sensory precision reaches very high levels, the veridicalreflectance and illuminant profiles are inferred and, quantitatively,both the Cornsweet and mach band effects disappear. The threeimages show exemplar percepts, at low, intermediate and highlevels of contrast respectively. The key difference in the spatialbanding that underlies the Cornsweet and Mach band effects isevident in the difference between the intermediate and high levelsof contrast.

The key prediction of these simulations is that we would expectsubjects to categorize their percepts, following a brief exposureto a Cornsweet stimulus, differently at different levels of contrast.At low levels of contrast, we would expect them to categorize thestimulus as uniformly flat. At intermediate levels of contrast, wewould expect them to categorize the stimulus as a Cornsweet per-cept, with isoluminant and uniform differences in the right andleft hand parts of the visual field; while at higher levels of contrastone would expect the Mach bands to dominate and the stimuliwould be categorized as possessing para-central bands. In princi-ple, at very high levels of contrast, the subject should perceive theveridical stimulus. However, whether this level of contrast can beattained empirically is an open question. In the next section, wetest these hypotheses psychophysically. We conclude this section bylooking not at behavioral responses but at the neuronal responsesimplicit in the simulations.

SIMULATING NEURONAL RESPONSESFigure 9 shows the prediction errors at low, intermediate and highlevels of contrast. These are shown at the sensory level (upper row)

www.frontiersin.org February 2012 | Volume 3 | Article 43 | 7

799 856

800 857

801 858

802 859

803 860

804 861

805 862

806 863

807 864

808 865

809 866

810 867

811 868

812 869

813 870

814 871

815 872

816 873

817 874

818 875

819 876

820 877

821 878

822 879

823 880

824 881

825 882

826 883

827 884

828 885

829 886

830 887

831 888

832 889

833 890

834 891

835 892

836 893

837 894

838 895

839 896

840 897

841 898

842 899

843 900

844 901

845 902

846 903

847 904

848 905

849 906

850 907

851 908

852 909

853 910

854 911

855 912

Brown and Friston Free-energy and illusions: the Cornsweet effect

FIGURE 8 |The effect of contrast. In the results presented here theprecision of the sensory data was used as a proxy for stimulus contrast. Asprecision increases, the strength of both the Cornsweet and Mach effectsincreases until, at very high levels of precision (contrast) the true luminanceprofile is perceived and the illusory percepts fade. Crucially, the Mach bands

appear at higher levels of contrast than the Cornsweet illusion. The inserts inthe lower panels show the inferred reflectance is at different levels of contrast(indicated by the blue dots in the lower graph). The prediction errorsassociated with the processing of stimuli at these three levels are shown inthe next figure. Please see main text for further details.

and at the higher levels of the hidden causes and states (lower row).The key thing to note here is that as contrast increases and thespatial detail of the posterior predictions increases, the sensory

prediction error falls. This is at the expense of inducing predic-tion errors at the higher level, which increase in proportion to theprecision of sensory information. These higher prediction errors

Frontiers in Psychology | Perception Science February 2012 | Volume 3 | Article 43 | 8

913 970

914 971

915 972

916 973

917 974

918 975

919 976

920 977

921 978

922 979

923 980

924 981

925 982

926 983

927 984

928 985

929 986

930 987

931 988

932 989

933 990

934 991

935 992

936 993

937 994

938 995

939 996

940 997

941 998

942 999

943 1000

944 1001

945 1002

946 1003

947 1004

948 1005

949 1006

950 1007

951 1008

952 1009

953 1010

954 1011

955 1012

956 1013

957 1014

958 1015

959 1016

960 1017

961 1018

962 1019

963 1020

964 1021

965 1022

966 1023

967 1024

968 1025

969 1026

Brown and Friston Free-energy and illusions: the Cornsweet effect

FIGURE 9 | Contrast and prediction error. As stimulus contrast increases,prediction error is redistributed from sensory input to hidden variables. Athigh levels of precision, sensory information induces prediction errors athigher levels, which in turn explain away prediction error at the sensorylevel. The higher level prediction errors at high precision reflect increasingconfidence that the reflectance is different from the prior expectation ofzero. Please see main text for further details.

are simply the difference between the posterior and prior expecta-tions and reflect an increasing departure from a prior expectationof zero as contrast (the log-precision of sensory noise) increases.Although these results are interesting in themselves, they can alsobe regarded as a simulation of event related potentials. The reasonthat we can associate prediction error with observed electromag-netic brain responses is that it is usually assumed that predictionerrors are encoded by the activity of superficial pyramidal cells (seeFigure 5). It is these cells that are thought to contribute primarilyto local field potentials and non-invasive EEG signals.

In the high-contrast condition, the prediction errors at thelower level are suppressed by the prediction of the presence ofa Cornsweet stimulus. This sort of phenomenon has been demon-strated using fMRI (Alink et al., 2010; den Ouden et al., 2010);predictable stimuli cause less activation in stimulus-specific areasthan unpredictable stimuli. However, the process simulated hereis likely to produce more complicated neurophysiological corre-lates because of the confounding effect of precision; increasedpredictability (through increasing conditional confidence aboutthe stimulus) will also increase estimates of precision. Since webelieve that the prediction errors reported by superficial pyra-midal cells are precision weighted, decreasing prediction errorin lower sensory areas may be masked by the increasing preci-sion of those errors. We will return to this and related issuesin a subsequent paper looking at the neurophysiological corre-lates of contrast-dependent illusory effects. Here, we focus onpsychophysical correlates:

A PSYCHOPHYSICAL TEST OF THEORETICAL PREDICTIONSIn this section, we report a psychophysics study of normal subjectsexposed to the same stimuli used in the simulations above. Wedepart from the normal procedures for assessing illusions (whichusually involve matching intensity differences) by using a forcedchoice paradigm. This is because we wanted to present stimulibriefly, for several reasons: First, brief presentation avoids the con-founding effects of saccadic eye movements. Second, it allows usto prototype the paradigm for future use in electrophysiological(event related potential) studies, which require transient stimulifor trial averaging. Finally, a forced choice paradigm places con-straints on the subject’s choices that map directly to the modelpredictions. In what follows, we describe the paradigm and inter-pret our results quantitatively, in relation to the predictions of thesimulations above.

SUBJECTS AND EXPERIMENTAL PARADIGMWe studied normal young subjects in accord with guidelines estab-lished by the local ethical committee and after obtaining informedconsent. Eight participants (4 female) completed the Mach bandparadigm; 19 (12 female) completed the Cornsweet paradigm.

EXPERIMENT 1 (CORNSWEET PARADIGM)The Cornsweet illusion was assessed using a two-interval forcedchoice procedure. Subjects were presented with a (set contrast)Cornsweet stimulus and real luminance step for 200 ms (a Gauss-ian temporal envelope was not used), separated by an interval of200 ms. One stimulus appeared to the left of fixation and one tothe right; this was randomized across trials, as was the order ofthe stimuli. Subjects were asked to report the side on which thestimulus with the greatest contrast had appeared (Figure 10).

Six blocks were completed, using Cornsweet stimuli with Webercontrasts of 0.0073–0.734. A Quest procedure (Watson and Pelli,1983) was used to select each step stimulus for comparison. Themean of the psychometric function was taken as the point ofsubjective equality between the Cornsweet stimulus and a realluminance. There were 200 presentations per block.

EXPERIMENT 2 (MACH BAND PARADIGM)The same method could not be used to identify the strength of theMach bands percept, as there is no non-illusory stimulus that canbe used for matching. Consequently, we used a two-alternativeforced choice paradigm: A single Cornsweet stimulus was dis-played for 200 ms to the left or right of fixation and participantswere asked to report if the stimulus contained Mach bands or not.Each participant completed six runs of 200 presentations at 10Weber contrast levels from 0.0204 to 0.2038. The probability ofreporting a Mach band was assessed as the relative frequency ofreporting its presence over trials, within subject (Figure 10).

In both experiments, stimuli were displayed on an LCD mon-itor under ambient room lighting. Subjects were seated 60 cmaway from the monitor, such that stimuli subtended an angle of14.21˚ vertically and 6.10˚ horizontally, at 2.96˚–7.06˚ eccentricity.The luminance ramp of the Cornsweet stimulus profile occupied2.42˚. Only the lowest levels of contrast the monitor was able toproduce were employed; thus, luminance values were linearizedpost hoc.

www.frontiersin.org February 2012 | Volume 3 | Article 43 | 9

1027 1084

1028 1085

1029 1086

1030 1087

1031 1088

1032 1089

1033 1090

1034 1091

1035 1092

1036 1093

1037 1094

1038 1095

1039 1096

1040 1097

1041 1098

1042 1099

1043 1100

1044 1101

1045 1102

1046 1103

1047 1104

1048 1105

1049 1106

1050 1107

1051 1108

1052 1109

1053 1110

1054 1111

1055 1112

1056 1113

1057 1114

1058 1115

1059 1116

1060 1117

1061 1118

1062 1119

1063 1120

1064 1121

1065 1122

1066 1123

1067 1124

1068 1125

1069 1126

1070 1127

1071 1128

1072 1129

1073 1130

1074 1131

1075 1132

1076 1133

1077 1134

1078 1135

1079 1136

1080 1137

1081 1138

1082 1139

1083 1140

Brown and Friston Free-energy and illusions: the Cornsweet effect

RESULTS AND DISCUSSIONThe results of the psychophysics experiments are shown inFigure 11, as a function of empirical (Weber) contrast levels. Theseresults are expressed as the mean over all subjects and associated

FIGURE 10 |Time courses of trials. Experiment 1: before the start of eachtrial, participants fixated a central cross. One Cornsweet and one realluminance step stimulus appeared for 200 ms each, with a 200 ms intervalbetween them. The order of the stimuli, their orientation and the side onwhich each appeared were randomized (although they were constrained toappear on opposite sides within each trial). Participants then had 1750 msto report the side on which the stimulus with the greatest contrast hadappeared (using the arrow keys of the keyboard). Experiment 2: each trialstared with fixation. A Cornsweet stimulus then appeared for 200 ms with arandom orientation to the left or right of fixation. Participants had 1750 msto report, with the “Y” and “N” keys, if they perceived Mach bands.

SE. The reported Cornsweet effect (as indexed by the point of sub-jective equality) peaked, on average over subjects, at a contrast ofabout 0.0025. At higher levels of contrast, as in the simulations,the effect fell quantitatively, plateauing at the highest contrastused in Experiment 1. Conversely, the probability of reporting aMach band increased monotonically as a function of the empiricalcontrast, reaching about 75% at a Weber contrast of about 0.15.

Qualitatively, these empirical results compare well with the the-oretical predictions shown in Figure 9: that is, the subjective orinferred Cornsweet effect emerged before the Mach bands, as con-trast increases. We also see the characteristic “inverted U” depen-dency of the Cornsweet effect on contrast levels. The empiricalprofile is somewhat compromised by the small range of contrastsemployed, however, this range was sufficient to disclose an unam-biguous peak. Clearly, it would be nice to relate these empiricalresults quantitatively to the simulations shown in Figure 9. Thispresents an interesting challenge because the psychophysical dataconsist of reported levels of an effect and the probability of aneffect for the Cornsweet and Mach Band illusions respectively.However, because our simulations provide a conditional or pos-terior probability over the effects reported, we can simulate bothsorts of reports and see how well they explain the psychophysicaldata. This quantitative analysis is now considered in more detail.

A FORMAL BEHAVIORAL ANALYSISThe simulations provide conditional expectations (and precisions)of both the Cornsweet and Mach band effects over a number ofsimulated (Weber) contrast levels, as modeled with the precisionof sensory noise. This means that one can compute a psychome-tric function of contrast c that returns the behavioral predictionsof both illusions respectively; namely, the level of the illusion andthe probability of inferring a Mach band. To predict the reported

FIGURE 11 | experimental results. This figure shows the results of theempirical psychophysical study for the Cornsweet illusion (left panel) andthe Mach band illusion (Right panel). Both report the average effect oversubjects and the SE (bars). The Cornsweet illusion is measured in termsof the subjective contrast level (quantified in terms of subjectiveequivalence using psychometric functions). The Mach band illusion is

quantified in terms of probability that the illusion is reported to bepresent. Both results are shown as functions of empirical (Weber)stimulus contrast levels. We have used the same range for both graphsso that the dependency on contrast levels can be compared. The keything to note here is that the Cornsweet illusion peaks before the Machband illusion (vertical line).

Frontiers in Psychology | Perception Science February 2012 | Volume 3 | Article 43 | 10

1141 1198

1142 1199

1143 1200

1144 1201

1145 1202

1146 1203

1147 1204

1148 1205

1149 1206

1150 1207

1151 1208

1152 1209

1153 1210

1154 1211

1155 1212

1156 1213

1157 1214

1158 1215

1159 1216

1160 1217

1161 1218

1162 1219

1163 1220

1164 1221

1165 1222

1166 1223

1167 1224

1168 1225

1169 1226

1170 1227

1171 1228

1172 1229

1173 1230

1174 1231

1175 1232

1176 1233

1177 1234

1178 1235

1179 1236

1180 1237

1181 1238

1182 1239

1183 1240

1184 1241

1185 1242

1186 1243

1187 1244

1188 1245

1189 1246

1190 1247

1191 1248

1192 1249

1193 1250

1194 1251

1195 1252

1196 1253

1197 1254

Brown and Friston Free-energy and illusions: the Cornsweet effect

level of the Cornsweet illusion we can simply use the conditionalexpectation μcorn(c) scaled by some (unknown) coefficient β1. Topredict the probability of reporting the presence of Mach bands,one can integrate the conditional probability distribution over theMach band effect above some (unknown) threshold β2. However,to do this, we need to know the relationship between the simulatedand empirical contrasts:

As noted above, we used the log-precision of sensory noiseγ to model log-contrast in accord with Weber’s law. This meanswe can assume a linear relationship between the empirical log-contrast and simulated log-precision. This induces two furtherunknown coefficients – the slope and intercept (β3 and β4) thatparameterise the relationship between the simulated and empiri-cal contrasts. Finally, we need to relate the conditional probabilityof a suprathreshold Mach band effect to the probability of report-ing its presence. Here, we assumed a simple, monotonic sigmoidrelationship, under the constraint that when the conditional prob-ability was 50:50, the report probability was also 50:50. The preciseform of this mapping is provided in Figure 11 (left panel) and has a

single (unknown) slope coefficient β5. These relationships providea mapping between the results of the simulations and the observedresponses averaged over subjects (under the assumptions of addi-tive prediction errors). This is known as a response model and isdetailed schematically in Figure 11. The predictions are based onthe simulated responses in Figure 9, assuming a smooth psycho-metric function of contrast that was modeled as a linear mixture ofcosine functions: Xk(γ) = cos(πγk) for k = 1, . . ., 6). The coeffi-cients of this discrete cosine set were estimated with ordinary leastsquares, using the responses of the model (μcorn(γ), μmac(γ)) overdifferent precision levels, at the time of maximum luminance.

Given the form of the relationships between the simulatedand empirical contrasts and between the report probability andconditional probabilities for Mach bands, one can use the psy-chophysical data to estimate the unknown coefficients of theserelationships: βi for = 1,. . ., 5. The results of this computation-ally informed response modeling are shown in the right panel ofFigure 12. The upper panels show the same data as in Figure 10placed over the theoretical psychometric functions based on the

FIGURE 12 | Predicted and experimental results. Theoretical predictionsof the empirical results reported in the previous figure: These predictions arebased upon a response model that maps from the conditional expectationsand precisions in the simulations to the behavioural responses of subjects.This mapping rests on some unknown parameters or coefficients βi:i = 1,. . ., 5 that control the relationship between the simulated γ(c) and empiricalcontrast levels c used for stimuli and the relationship between theprobabilities of reporting a Mach band σ(pmac(c)) and the conditionalprobability that it exceeds some threshold pmac(c) at contrast level c. Theserelationships form the basis of a response model, whose equations areprovided in the left panel (for simplicity, we have omitted the expressions forconditional variance of the Mach band contrast). Given empirical responsesfor the mean Cornsweet effect M (c) and probability of reporting a Mach

band P (c), the coefficients βi can be estimated under the assumption ofadditive prediction errors ε. The predicted responses following thisestimation are shown in the graphs on the right hand side. The upper panelsshow the empirical data superimposed upon conditional predictions fromthe model. The gray lines are the predicted psychometric functions, μcorn(c)and σ(pmac(c)). The red dots correspond to the predictions at levels ofcontrast used in the simulations (as shown in Figure 9), while the black dotscorrespond to the empirical responses: M (c) and P (c). The lower left panelshow the relationship between the empirical and simulated contrast levels(as a semi-log plot of the empirical contrast against the log-precision ofsensory noise). The lower right panel shows the relationship between theprobability of reporting a Mach band is present and the underlyingconditional probability that it is above threshold.

www.frontiersin.org February 2012 | Volume 3 | Article 43 | 11

1255 1312

1256 1313

1257 1314

1258 1315

1259 1316

1260 1317

1261 1318

1262 1319

1263 1320

1264 1321

1265 1322

1266 1323

1267 1324

1268 1325

1269 1326

1270 1327

1271 1328

1272 1329

1273 1330

1274 1331

1275 1332

1276 1333

1277 1334

1278 1335

1279 1336

1280 1337

1281 1338

1282 1339

1283 1340

1284 1341

1285 1342

1286 1343

1287 1344

1288 1345

1289 1346

1290 1347

1291 1348

1292 1349

1293 1350

1294 1351

1295 1352

1296 1353

1297 1354

1298 1355

1299 1356

1300 1357

1301 1358

1302 1359

1303 1360

1304 1361

1305 1362

1306 1363

1307 1364

1308 1365

1309 1366

1310 1367

1311 1368

Brown and Friston Free-energy and illusions: the Cornsweet effect

simulations of the previous section. These predictions are basedon the mapping from simulated to empirical contrast levels (lowerleft) and the relationship between the probability of reporting aMach band and the conditional confidence that it is present (lowerright).

By construction, the relationship between the simulated andempirical contrasts is linear when plotted on a log-log scale. Theslope of this plot suggests that the higher contrasts used in the sim-ulations are, practically, not realizable in an empirical setting. Thisis because as the simulated contrast increases the correspondingempirical contrast increases much more quickly. The implicationof this is that the contrasts employed in the psychophysics studycorrespond to the first few levels of the simulated contrasts. Thismeans that it may be difficult to demonstrate the theoretically pre-dicted attenuation of the Cornsweet illusion at very high levels ofcontrast.

The relationship between the report and conditional probabil-ities suggests that subjects have a tendency to “all or nothing”reporting; in the sense that a conditional confidence that theprobability of reporting a Mach band is slightly greater than theconditional confidence it is above threshold. Conversely, subjectsappear to report the absence of Mach bands with a probabil-ity that is slightly greater than the conditional probability it isbelow threshold. The resulting psychometric predictions (in theupper panels) show a remarkable agreement between the predictedand observed probabilities of reporting a Mach band. The corre-spondence between the predicted and empirical results for theCornsweet illusion are less convincing but show that both asymp-tote to a peak level much more quickly than the probability ofreporting a Mach band.

In summary, this analysis suggests that there is a reasonablequantitative agreement between the theoretical predictions andempirical results. Furthermore, in practical terms, it appears thatthe normal range of Weber contrasts that can be usefully employedcorresponds to a relatively low level of sensory precision in the

simulations. This means that it may be difficult to demonstrate the“inverted U” behavior for the Cornsweet illusion seen in Figure 9.This is because it may be difficult to present stimuli at the ultrahigh levels of contrast required. Note that the empirical proba-bility of reporting a Mach band does not decrease as a functionof contrast level. This is to be anticipated from the theoreticalpredictions: increasing the contrast level increases the conditionalprecision about the inferred level of the Mach band effect, whichmeans that the probability that is above threshold can still increaseeven if the conditional expectation decreases (as in Figure 9).

CONCLUSIONThis paper has reviewed the nature of illusions, in the contextof Bayes-optimal perception. We reiterate the notion that illusorypercepts are optimal in that they may represent the most likelyexplanation for ambiguous sensory input. We have illustrated thisusing the Cornsweet illusion. By using simple and plausible priorexpectations about the spatial deployment of illuminance andreflectance, we have shown show that the Cornsweet effect emergesas a natural consequence of Bayes-optimal perception. Further-more, we were able to simulate a contrast-dependent emergenceof the Cornsweet effect and subsequent appearance of Mach bandsthat was verified psychophysically using a forced choice paradigm.

SOFTWARE NOTEThe simulations and graphics presented in this paper can bereproduced with the DEM toolbox distributed with the academicfreeware SPM from http://www.fil.ion.ucl.ac.uk/spm/. The anno-tated files that implement the Cornsweet illusion simulations andthe more general routines used for model inversion are providedas Matlab code.

ACKNOWLEDGMENTSThis work was funded by the Wellcome Trust. We thank MarciaBennett for helping prepare this manuscript.

REFERENCESAlink, A., Schwiedrzik, C. M., Kohler,

A., Singer, W., and Muckli, L. (2010).Stimulus predictability reducesresponses in primary visual cortex.J. Neurosci. 30, 2960–2966.

Anderson, E. J., Dakin, S. C., andRees, G. (2009). Monocular sig-nals in human lateral genicu-late nucleus reflect the Craik-Cornsweet-O’Brien effect. J. Vis. 9,1–18.

Barlow, H. B. (1974). Inductive infer-ence, coding, perception, and lan-guage. Perception 3, 123–134.

Burton, G. J., and Moorhead, I. R.(1987). Color and spatial structurein natural scenes. Appl. Opt. 26,157–170.

Carandini, M., and Heeger, D. J. (1994).Summation and division by neuronsin primate visual cortex. Science 264,1333–1336.

Cornsweet, T. (1970). Visual Perception.New York: Academic Press.

Craik, K. J. (1938). The effect of adap-tation on differential brightness dis-crimination. J. Physiol. (Lond.) 92,406–421.

Craik, K. J. W. (1966). The Natureof Psychology, ed. S. L. Sherwood.(Cambridge: Cambridge UniversityPress).

Dayan, P., Hinton, G. E., and Neal, R.M. (1995). The Helmholtz machine.Neural Comput. 7, 889–904.

den Ouden, H. E., Daunizeau, J., Roiser,J., Friston, K. J., and Stephan, K.E. (2010). Striatal prediction errormodulates cortical coupling. J. Neu-rosci. 30, 3210–3219.

Fechner, G. T. (1860). “Specielles zurMethode richtiger und falscher Fälle,in Anwendung auf die Gewichtsver-suche,” in Elemente der Psychophysik(Leipzig: Breitkopf and Härtel).

Feldman, H., and Friston, K. J. (2010).Attention, uncertainty, and free-energy. Front. Hum. Neurosci. 4:215.doi:10.3389/fnhum.2010.00215

Field, D. J. (1987). Relations between thestatistics of natural images and theresponse properties of cortical cells.J. Opt. Soc. Am. A 12, 2379–2394.

Friston, K. (2008). Hierarchical mod-els in the brain. PLoS Comput.Biol. 4, e1000211. doi:10.1371/jour-nal.pcbi.1000211

Friston, K. (2009). The free-energyprinciple: a rough guide to the brain?Trends Cogn Sci. (Regul. Ed.) 13,293–301.

Friston, K., Kilner, J., and Harrison, L.(2006). A free energy principle forthe brain. J. Physiol. Paris 100, 70–87.

Geisler, W. S., and Albrecht, D. G.(1992). Cortical neurons: isolationof contrast gain control. Vision Res.32, 1409–1410.

Grossberg, S., and Hong, S. (2006).A neural model of surface per-ception: lightness, anchoring,and filling-in. Spat. Vis. 19,263–321.

Knill, D. C., and Pouget, A. (2004).The Bayesian brain: the role ofuncertainty in neural coding andcomputation. Trends Neurosci. 27,712–719.

Lotto, R. B., and Purves, D. (2001).An empirical explanation of theChubb illusion. J. Cogn. Neurosci. 13,547–555.

Lotto, R. B., Williams, S. M., andPurves, D. (1999). Mach bandsas empirically derived associations.Proc. Natl. Acad. Sci. U.S.A. 96,5245–5250.

Mach, E. (1865). Über die Wirkungder raumlichen Verteilung desLichtreizes auf die Netzhaut.Sitzungsber. Heidelb. Akad.

Frontiers in Psychology | Perception Science February 2012 | Volume 3 | Article 43 | 12

1369 1426

1370 1427

1371 1428

1372 1429

1373 1430

1374 1431

1375 1432

1376 1433

1377 1434

1378 1435

1379 1436

1380 1437

1381 1438

1382 1439

1383 1440

1384 1441

1385 1442

1386 1443

1387 1444

1388 1445

1389 1446

1390 1447

1391 1448

1392 1449

1393 1450

1394 1451

1395 1452

1396 1453

1397 1454

1398 1455

1399 1456

1400 1457

1401 1458

1402 1459

1403 1460

1404 1461

1405 1462

1406 1463

1407 1464

1408 1465

1409 1466

1410 1467

1411 1468

1412 1469

1413 1470

1414 1471

1415 1472

1416 1473

1417 1474

1418 1475

1419 1476

1420 1477

1421 1478

1422 1479

1423 1480

1424 1481

1425 1482

Brown and Friston Free-energy and illusions: the Cornsweet effect

Wiss. Math. Naturwiss. Kl. 52,303–322.

Mumford, D. (1992). On the com-putational architecture of the neo-cortex. II. The role of cortico-cortical loops. Biol. Cybern. 66,241–251.

O’Brien,V. (1959). Contrast by contour-enhancement. Am. J. Psychol. 72,299–300.

Purves, D., Shimpi, A., and Lotto, R. B.(1999). An empirical explanation ofthe Cornsweet effect. J. Neurosci. 19,8542–8551.

Rao, R. P., and Ballard, D. H. (1999).Predictive coding in the visualcortex: a functional interpretationof some extra-classical receptive-field effects. Nat. Neurosci. 2,79–87.

Ratliff, F. (1965). Mach Bands: Quanti-tative Studies on Neural Networks in

the Retina. San Francisco: Holden-Day.

Ruderman, D. L. (1997). Origins of scal-ing in natural images. Vision Res. 37,3385–3398.

Ruderman, D. L., and Bialek, W. (1994).Statistics of natural images: Scalingin the woods. Phys. Rev. Lett. 73,814–817.

Shams, L., Ma, W. J., and Beierholm, U.(2005). Sound-induced flash illusionas an optimal percept. Neuroreport16, 1923–1927.

Tolhurst, D. J., Tadmor, Y., and Chao, T.(1992). Amplitude spectra of naturalimages. Ophthalmic Physiol. Opt. 12,229–232.

von Helmholtz, H. (1866). Treatise onPhysiological Optics, Vol. III, 3rd Edn(trans. by J. P. C. Southall 1925 Opt.Soc. Am. Section 26). New York:Dover.

Watson, A. B., and Pelli, D. G.(1983). QUEST: a Bayesianadaptive psychometric method.Percept. Psychophys. 33,113–120.

Weber, E. (1846). “Der Tastsinn and dasGemeingefuhl,” in Handwörterbuchder Physiologie, ed. R. Wagner(Leipzig: Wilhelm Engelmann).

Weiss, Y., Simoncelli, E. P., and Adel-son, E. H. (2002). Motion illusionsas optimal percepts. Nat. Neurosci.5, 598–604.

Yuille, A., Geiger, D., and Bulthoff,H. (1991). Stereo integration, meanfield theory and psychophysics. Net-work 2, 423–442.

Conflict of Interest Statement: Theauthors declare that the research wasconducted in the absence of any com-mercial or financial relationships that

could be construed as a potential con-flict of interest.

Received: 30 November 2011; paperpending published: 21 December 2011;accepted: 07 February 2012; publishedonline: xx February 2012.Citation: Brown H and Friston KJ (2012)Free-energy and illusions: the Corn-sweet effect. Front. Psychology 3:43. doi:10.3389/fpsyg.2012.00043This article was submitted to Frontiers inPerception Science, a specialty of Frontiersin Psychology.Copyright © 2012 Brown and Friston.This is an open-access article distributedunder the terms of the Creative CommonsAttribution Non Commercial License,which permits non-commercial use, dis-tribution, and reproduction in otherforums, provided the original authors andsource are credited.

www.frontiersin.org February 2012 | Volume 3 | Article 43 | 13