Andreas KleinschmidtINSERM U992CEA NeuroSpinSaclay, France

Mind Reading - Can Imaging Tell What You Are Thinking?

CognitiveNeuroscience In Humans –Many Modalities

One Imaging Modality – Many Signals

Trait vs. State - Structural and Functional Abnormalities in Adolescents with Conduct Disorder

-0,2

-0,15

-0,1

-0,05

0

0,05

0,1

0,15

0,2

40 60 80 100

aggression score

gre

y m

atte

r co

nce

ntr

atio

n

r = -5.4

p = 0.006

Voxel-based morphometry in 12 adolescents with conduct disorder compared with 12 controls

(Sterzer et al., NeuroImage 2007)

C

ageIQ

attention problems

anxiety/depression

socio-economic state

x = 9 y = 36

A B

-5

-4

-3

-2

-1

0

1

2

3

controls patients

size

of e

ffect

neutral

negative

C

ageIQ

attention problems

anxiety/depression

socio-economic state

x = 9 y = 36

A B

-5

-4

-3

-2

-1

0

1

2

3

controls patients

size

of e

ffect

neutral

negative

x = 9 y = 36

A B

-5

-4

-3

-2

-1

0

1

2

3

controls patients

size

of e

ffect

neutral

negative

(Sterzer et al., Biol. Psychiatry 2005, Stadler et al., J Psych Res 2006)

Different responsesto affective stimuli

Reversing the Direction of Inference –From Brain Mapping to Mind Reading

(Dehaene et al., Nature Neuroscience 1998)

From Simple Actions toCognitive Categories

From Single Voxels orRegions of Interest toMulti-Voxel Patterns

Classification(Decoding)

(Norman et al., 2006)

From categories to exemplars and invariance

from Op de Beeck et al., 2008

= ≠

(Eger et al. J Neurophysiol, 2008)

Training and Testingthe Classifier (SVM)

(Miyawiki et al., Neuron 2008)

Decoding Images From Visual Cortex Activity

Decoding: From Individual Natural Images to Individual Art Painters

(Yamamura et al., Neuroreport 2009)

Thoughts are forms created in the mind, rather than the forms perceived through the five senses. Thought and thinking are the processes by which these imaginary sense perceptions arise and are manipulated.

Cognitive psychology: Mental processes which mediate between stimulus and response, including the psychology of reasoning, and how people make decisions and choices, solve problems, as well as engage in creative discovery and imaginative thought.

Developmental psychology (Piaget): Thought evolves from being based on perceptions and actions at the sensorimotor stage to internal representations. Subsequently, representations are gradually organized into logical structures which first operate on the concrete properties of the reality, in the stage of concrete operations, and then operate on abstract principles that organize concrete properties, in the stage of formal operations.

From Actions and Images to Thoughts

Decoding Untrained Items

(Mitchell et al., Science 2008)

Sample numerosities: 4 8 16 32either item size, or overall luminance matched between sample numerosities

Delayed number comparison task:

Sample stimulus 200 ms

+Delay period 4-7 s

Match stimulus 200 ms -> response: smaller or larger number?(50 % difference)

Measuring signals from individual numerosities

high-res fMRI 1.5 mm voxels

at 3T

Eger et al., Curr Biol 2009

Supra-modal representation of individual numbers

From Classification of Noisy Evoked Response Patterns To Predictions from Noise

(Norman et al., 2006)

Spontaneous local variations in ongoing pre-stimulus neural activity bias perceptual decisions

(Hesselmann et al., PNAS, 2008)

right hMT+

z = 3

** *

[46 ± 4, -69 ± 6, 2 ± 3]

coherentrandom

(Hesselmann et al., J Neurosci, 2008)

t

100x

355ms motion

20 – 40s static (ISI)

The Brain builds a Predictive Model of the World

(Friston, Trends Cogn Sci, 2009)

Thinking allows beings to model the world and to represent it according to their objectives, plans, intentions and desires. (Wikipedia)

Brain Imaging and Its Impact on Society

Who has the right to know?

What are we thinking?

Cooperation Consent