Sensation & Perception

• Exam 3: Review

© Takashi Yamauchi (Dept. of Psychology, Texas A&M University)

Sensation & Perception

Ch. 7: Action

© Takashi Yamauchi (Dept. of Psychology, Texas A&M University)

Main topicsEcological approach

Optic flow

The physiology of navigation

Skilled action

Linking sensory and motor functions

Perception reconsidered

• J. J. Gibson’s ecological approach– All the studies we have seen so far are

conducted in laboratory settings, in which stationary stimuli were given.

– Perception is not a stationary experience.

– Perception should be studied as it occurs in the natural environment.

Optic flow and a gradient of flow

• A gradient of flow gives a strong cue for motion and depth perception

Demonstration:

http://www.rdg.ac.uk/arl/clips/demo_of_displays.htm#opticflow

The Physiology of Navigation

• Optic flow neurons - neurons in the medial superior temporal area (MST) of monkeys respond to flow patterns

Neuron 1 in the monkey’s MST responds to an expanding stimulus but not a stimulus with circular movement. Neuron 2 responds to circular movement but not to expansion.

Physiological Links Between Sensory and Motor Functions

• Optic ataxia

– Patients suffering from this syndrome produce inaccurate reaching movements towards a target or object in space.

Experiment by Schindler et al.• Participants:2 groups

• Patients with parietal lobe damage and normal control participants.

• Participants performed two tasks• Bisection task - point to position between cylinders• Reaching task - reach between cylinders and touch a gray

strip

Bisection task

In each trial, you saw two cylinders that were shifted different positions.

You had to point a place exactly midway between the cylinders.

Experiment by Schindler et al.

Bisection task

In each trial, you saw two cylinders that were shifted different positions.

You had to point to a place exactly midway between the cylinders.

So, for each trial, you had to point different positions.

Trial 1

Trial 2

point

point

Experiment by Schindler et al.

Reaching task

In each trial, you saw two cylinders that were shifted different positions.

You had to touch a gray strip that were 20 cm behind the cylinders.

So, for each trial, you had to extend your hand to different positions.

Trial 1

Trial 2

touch

touch

Results• Pointing task

– The patients with parietal lobe damage and normal control were equally accurate

• Reaching task– The patients kept

reaching the same place even though cylinders were shifted in each trial.

Trial 1

Trial 2

touch

touch

Mirror neurons

Mirror neurons• Mirror neurons respond to a particular

“kind” of action (e.g., grasping) AND observing someone doing the action.

The activity of a particular neuron in the premotor area of a monkey.

• Video clip from NOVA (PBS)– 15 min

– http://www.pbs.org/wgbh/nova/sciencenow/3204/01.html

Sensation & Perception

Ch. 8: Motion

© Takashi Yamauchi (Dept. of Psychology, Texas A&M University)

Main topicsThe functions of motion perception

The direction of movement

4 different ways to create the perception of movement

• Real movement– The object is physically moving

• Apparent movement– Displacement of objects

• Induced movement

• Movement aftereffect

Functions of Movement Perception

• Survival in the environment

– Predators use movement of prey as a primary means to location in hunting

– Motion agnosia

• Damage to the cortex resulting in inability to perceive movement

• Extremely debilitating and dangerous for the patient

Functions of Movement Perception - continued

• Perceiving objects

– Movement of objects or the observer’s movement through objects help perceive the 3D organization of stimuli

– Kinetic depth effect - movement of an object’s 2-D shadow can change into perception of a 3-D object

• This is an example of structure-from-motion

– Perceptual organization

Figure 9.4 Setup similar to the one used by Wallach and O’Connell (1953) to demonstrate the kinetic depth effect.

– Demonstration: structure-from-motion

• http://www.cs.ubc.ca/nest/imager/contributions/flinn/Illusions/Illusions.html

Visual Pathway (MT)

The Physiology of Navigation

• Optic flow neurons - neurons in the medial superior temporal area (MST) of monkeys respond to flow patterns

• Neurons that are sensitive to a specific direction of motion.

• Inhibitory connections

The intelligence of movement perception

• Top-down processes

– Certain stimuli (like the human body) have special meaning that affects perception

Figure 9.18 Frames from the stimuli used by Grossman and Blake (2001). (a) Sequence from the point-light walker stimulus. (b) Sequence from the scrambled point-light stimulus.

Biological Motion (Demonstration)

http://www.biomotionlab.ca/

Depth and size

• Monocular depth cues

• Binocular depth cues

Monocular pictorial cues

• Occlusion

• Relative height

• Relative size

• Familiar size

• Atmospheric perspective

• Linear perspective

Leonardo da Vinci (1452-1519)There has never been an artist who was more fittingly, and without qualification, described as a genius. Like Shakespeare, Leonardo were from an insignificant background and rose to universal acclaim. Leonardo was the illegitimate son of a local lawyer in the small town of Vinci in the Tuscan region….

Mona Lisa (1503)

The Dreyfus Madonna: da Vinci 1469

The Santa Trinita Madonna: Cimabue (1260/80)

Linear perspective

Things get smaller when they are away.

Occlusion

Relative height

Relative size

Atmospheric perspective

Texture gradient

• Two types of perspectives

– Linear perspective– Atmospheric perspective

Belvedere: Escher

Binocular disparity

• The binocular disparity arises when a given point in the external world does not project to the corresponding points on the left and right retinae – (Palmer, 1999, “Vision Science”)

Binocular disparity• Your two eyes are getting

different images disparity double images

Corresponding points

Corresponding points

No disparity disparity

Small disparity

Large disparity No disparity

Horopter (Vieth-Muller circle)

Small disparitylarge disparity

small

distance

large

distance

Sensation & Perception

Ch. 11: Sound, The Auditory System, and Pitch Perception

© Takashi Yamauchi (Dept. of Psychology, Texas A&M University)

Main topicsSound stimuli, amplitude and frequency

Sound quality (timbre)

Ear (structure)

Place theory

Central auditory processing

Sound wave• Figure 1: Graphic

representations of a sound wave. (A) Air at equilibrium, in the absence of a sound wave; (B) compressions and rarefactions that constitute a sound wave; (C) transverse representation of the wave, showing amplitude (A) and wavelength (taken from Britannica Online)

A sound wave is determined by two factors

• Magnitude– Y axis

• Frequency– X axis

Specifying a sound stimulus

• Amplitude– Y axis– Decibel (dB)– Number of dB = 20 x log(P/P0)– (P: the sound pressure of the

stimulus, P0: a standard pressure)– P0:=the pressure of a 1000Hz tone at

threshold.

• Frequency– X axis– Hertz (Hz) one cycle per

second

dB: Decibel

• dB? With p=200, p0(standard pressure level)=20

dB=20 x log (200/20)= 20 x log (10) = 20 x 1= 20

• with p = 2000dB = 20 x log (2000/20)= 20 x log (100) = 20x 2 = 40

With p = 20000dB = 20 x log (20000/20)= 20 x log (1000) = 20x 3 = 60

• create a complex sound by combining simple sound waves

Additive synthesis

• reduce a complex sound wave into a collection of simple sound waves.

Fourier analysis

• A sound wave from clarinet.

• Simple sound waves that make a sound of clarinet

Figure 11.9 The frequency spectrum for the tone in Figure 11.8d. The heights of the lines indicate the amplitude of each of the frequencies that make up the tone.

Ear

Ear

Bekesy’s place theory

• A wave spreads.• The wave reached the

peak at a particular location.

The height of the wave reaches the peak at P, and then gradually subsides.

• Different locations of vibration peak are produced by different spatial frequencies.

• Different frequencies of sound waves activate hair cells in different locations

• A complex tone (440Hz, 880Hz, and 1320Hz)..

• The auditory system basically carry out a “Fourier analysis” treat a complex sound as a composite of simple waves.

ch 12 73

Sensation & Perception

Ch. 12: Sound Localization

© Takashi Yamauchi (Dept. of Psychology, Texas A&M University)

Main topicsAuditory localization

Perceptual grouping

ch 12 74

Identifying the sound source in the horizontal coordinate

• Interaural differences

– Interaural time differences• Capture the difference in the time that a sound

reaches the left and right ears

– Interaural level differences• Capture the difference in the level of the sound

intensity (sound pressure level) that a sound reaches the left and right ears

ch 12 75

Sensation & Perception

Ch. 13: Speech Perception

© Takashi Yamauchi (Dept. of Psychology, Texas A&M University)

Main topicsPhoneme

Speech spectrogram

The segmentation and variability problems

Categorical perception

Multimodal speech perception, Top-down process

Speech perception and the brain

Some major problems in speech perception

• The segmentation problem

– How do we segment the individual words?

• The variability problem

– Variability from different speakers

– Variability from a phoneme’s context • E.g., /d/ differs in “dig” “dug”

Stimuli

• Phonemes:• The smallest unit of the sound system in a language

– consonants and vowels– If you change a phoneme in a word, the meaning of the word is

also changed. (e.g., hip tip; cat hat; dog dig)

• English has about 40 different phonemes– Every English word is produced by a combination of phonemes.

• Different languages have different phonemes– E.g., Japanese does not distinguish r and l.– E.g., rice / lice; election / erection

Some major problems in speech perception

• The segmentation problem

– How do we segment the individual words?

• The variability problem

– Variability from different speakers

– Variability from a phoneme’s context • E.g., /d/ differs in “dig” “dug”

Question:

• How does the perceptual system handle these problems, and give us a coherent speech perception?

Using visual cues in speech perception

• The McGurk effect (demonstration)– http://www.media.uio.no/personer/arntm/McGurk_english.html

– What am I saying? Play the clip several times, alternating between looking at the talking head while listening, and listening with your eyes shut. Most adults (98%) think they are hearing "DA" - a so called "fused respons" - where the "D" is a result of an audio-visual illusion. In reality you are hearing the sound "BA", while you are seing the lip movements "GA". The "McGurk effect" was first described by Harry McGurk and John MacDonald in "Hearing lips and seeing voices", Nature 264, 746-748 (1976).

Using visual cues in speech perception

• The McGurk effect (demonstration)– http://www.youtube.com/watch?v=I1XWDOwH47Y– http://www.youtube.com/watch?v=jtsfidRq2tw&NR=1

– The "McGurk effect" was first described by Harry McGurk and John MacDonald in "Hearing lips and seeing voices", Nature 264, 746-748 (1976).

Speech perception and the brain

• Broca’s area

• Wernicke’s area

Speech Perception and the Brain

• Broca’s aphasia - individuals have damage in Broca’s area (in frontal lobe)– Labored and stilted speech and short sentences but they

understand others– http://video.google.com/videoplay?

docid=9178936581276081395&q=Broca%27s+aphasia&total=6&start=0&num=10&so=0&type=search&plindex=0

• Wernicke’s aphasia - individuals have damage in Wernicke’s area (in temporal lobe)– Speak fluently but the content is disorganized and not

meaningful– They also have difficulty understanding others– http://video.google.com/videoplay?

docid=7590914168187986085&q=Wernicke%27s+aphasia&total=3&start=0&num=10&so=0&type=search&plindex=0