1

Depth Perception

Perception & Attention Course

George Mather

Depth cues

• Retinal images are two-dimensional, yet the world is three-dimensional. The visual system must recover the missing dimension.

• A depth cue can be defined as anything that is used by the visual system to estimate the depth of a point in space, or to perceive depth in a 3-D shape.

• Retinal images contain multiple visual cues. Oculomotor cues are also available.

• Visual cues can be divided into monocular and binocular.

Height

• Height in the image, relative to the horizon, gives a cue to distance.

Linear Perspective

• Parallel lines sloping away from the observer converge as they recede into the distance.

• Similarly sized shapes decrease in retinal size as they recede into the distance.

Blur

• Eyes (and cameras) have limited depth of field –objects nearer or farther than the point of fixation or focus are blurred.

• Blur variation across the image gives a cue to distance.

Aerial Perspective

• Particles in the atmosphere scatter light, particularly in the blue region of the spectrum.

• As a result, distant objects appear lower in contrast, and slightly blue.

2

Accommodation

• As fixation shifts from far to near, the shape of the lens must be changed to maintain a sharply focused image on the retina (accommodation).

• Cili ary muscles are used to control lens shape.

• Information on the state of the muscles offers a cue to fixation distance.

FAR NEAR

Motion Parallax

• As the observer moves, points at different distances move at different velocities over the retina.

• Points beyond fixation (red) move in the direction of observer motion; points nearer than fixation (green) move in the direction opposite to observer motion.

F

T im e 1 T im e 2

Fie ld o f v iew

Shadows

• Shading reveals 3-D shape.

• The visual system assumes that light is falli ng from above.

Interposition

• Near objects often partially obscure far objects.

• The retinal image contains ‘T-junctions’ at points where contours of the nearer object intersect contours of the far object.

Convergence

• Convergence angle is the angle formed by the two eyes when fixating an object at a specific distance.

• As fixation shifts from a far object to a near object, convergence angle increases.

• Extra-ocular muscles control eye position, including convergence angle.

• Information on the state of the muscles offers a cue to fixationdistance.

Fa r F ixatio n Nea r F ixation

F

F

Binocular Disparity

• The two eyes receive slightly different views of the world.

• These slight differences provide a powerful depth cue.

Natu ra l S te reo Im ag e

L eft Eye V iew Rig h t E ye V ie w

3

Disparity in detail• Images of points at the same

distance fall on corresponding retinal positions in the two eyes.

• The horopter is a line drawn through all such points.

• Images of points nearer or farther than fixation fall on non-corresponding or disparate retinal locations.

• The sign or direction of the disparity signifies near vs. far depth.

• The magnitude of the disparity signifies magnitude of depth.

Horop ter

LeftRetina

RightRetina

Fov ea

F

Farther aw ay:Uncros se d o r far d isparity

F ixa ted :Corre spond ing po in tsor zero dis pa rity

Nea rer:Crosse d o r ne ar d isparity

LeftRetina

RightRetina

Fov ea

F

Stereoscopes

• A variety of devices simulate the slightly different views created by a real stereo image.

Natu ra l S tere o Im ag e

P rism Stereo sco p e Red -G re en Ana g ly p h

L eft Ey e V iew Rig h t E ye V iew

Autostereograms

• Designed to allow stereo viewing without equipment.• L and R eye views are interlaced in vertical columns.• Convergence in front or behind the image brings the two eyes’ columns into

alignment.

Autostereogram

LR

Disparity coding in the cortex

• Primate cortex contains binocular cells that respond selectively to stimuli falli ng on disparate retinal locations in the two eyes.

• Position of right-eye receptive field (RED) relative to left-eye receptive field (GREEN) determines preferred disparity.

• Data shown are based on Poggio & Talbot (1981) J. Physiol. 315, 469-492.

Stimulus Cell Response

L R

0.2 deg(n ear )

0 deg

-0.2 deg(fa r )

Disparity Cell 1 Cell 2

Psychophysical evidence for disparity coding cells

• Random dot stereograms (RDS) contain only disparity cues.

• A sub-set of dots in one eye’s view is displaced relative to the other eye’s view.

• Subjects perceive the displaced dots as standing out in depth against the other dots.

• The success of RDS is evidence for a pure disparity coding mechanism.

The correspondence problem in RDS

• In a RDS, how does the visual system correctly match dots in the left-eye image with dots in the right-eye image?

• With just two dots in each eye there are 2 correct and 2 false matches.

• Image information alone is not suff icient to solve the problem.

• The visual system uses constraints or assumptions based on properties of real-world objects and stereo projection to rule out the majority of false matches.

• These can be embodied in interactions between disparity coding cells.

C o rrec t M atch

F a ls e M atch

4

Cue combination

• Natural images contain multiple depth cues. How are they combined?

• The magnitude of different cues is very highly correlated: cue magnitude is proportional to (1/distance).

• To derive a single estimate of depth from multiple cues, the visual system seems to take the average of different cue values.

• Cues are weighted according to their reliabilit y in a given set of stimulus conditions.