depth perception cs790 presentation andrew eggebeen 5/6/2015

Depth PerceptionCS790 PresentationAndrew Eggebeen

5/6/2015

Depth perception

• “Cognitive processes that lead to the inference of the third dimension from 2D retinal images” [1]• Multiple cues are perceived• Depth can be perceived incorrectly• Relevance to Computer Science• More realistic 3-D graphics• More efficient vision systems• Visual prosthesis (bionic eye)

Depth cues

• Monocular cues• Gathered from one eye (one image)• Used to perceive depth from a ‘flat’ image (photo, computer screen, etc.)• Makes vision system robust

• Not entirely impaired if one eye is closed, covered, or damaged

• Binocular cues• Gathered from both eyes (two images)• Used in Virtual Reality headsets• Gives more information about depth (different info from both eyes)

• Oculomotor cues• Physical (non-visual) feedback

Motion parallax

• During lateral motion, closer objects move across field of view more rapidly than distant objects• Used in 2-D side-scrolling computer games to give feeling of depth• Still used in 3-D graphics

• Pigeons bob their heads perceive depth from parallax

Size

• Relative size• For objects of the same size, distant

objects will appear smaller and closer objects will appear larger

• Familiar size• ‘Known’ size of an object suggests

how distant it is

• Useful effect for films• Can create illusion of object size• Can create illusion of object distance

Perspective

• Linear perspective• Parallel lines converge at vanishing points• Significant depth cue used in 3-D graphics

• Aerial perspective (atmospheric perspective)• Contrast of distant features decreases• Moisture in atmosphere scatters light and veils

distant features• Popular effect in paintings/artwork

Muller Lyer illusion

• Are the center lines the same length?• Yes

• Explanation• Angles on left suggest a ‘protruding’ (closer)

feature, angles on right suggest a ‘receding’ (more distant) feature• A ‘close’ line and a ‘far’ line with the same

length are perceived as shorter and longer, respectively

• People from different cultures perceive this illusion to varying degrees [2]

Ames room illusion

• Are these people the same size?• Yes

• Explanation• Room is not actually square• People appear to be the same distance

from viewer, but one is closer• Size suggests closer person is much larger

than distant person

Lighting and shading

• Surfaces facing light sources are more well lit, surfaces facing away from light sources are less well lit• Visual system assumes that “light is shining from above” [3]• Visual system assumes that “there is only one light source illuminating

the entire visual image” [3]• Another significant depth cue in 3-D graphics

Image from [3]

Defocus blur

• Vision system has limited depth of focus• Objects at focused distance are clear, others

are blurry• Popular effect in photography• Jumping spiders use defocus blur for depth

perception• But only with certain wavelengths of light

“When using just green light to illuminate prey, the spiders were always accurate in their strikes, just like in full spectrum light. However, when green light was excluded and red light exclusively was used to illuminate prey, the spiders attacks on prey were typically short.” [5]

Image from [5]

• “Estimating Spatially Varying Defocus Blur From A Single Image” [5]• Xiang Zhu, Member, IEEE, Scott Cohen, Member, IEEE, Stephen Schiller, Member, IEEE, and Peyman Milanfar, Fellow, IEEE• December 2013

• Algorithm analyzes defocus blur in an image• Can be used to segment foreground image from background

Binocular disparity

• Difference in images received by each eye• Disparity is more significant for closer objects• Less effective at long distances• Less disparity of distant objects

• Correspondence problem• How to merge the two images into a single

image?

• Virtual reality headsets (e.g. Oculus Rift) project offset images onto each eye

Autostereogram

• Converging eyes behind image overlaps patterns• Patterns repeating across

smaller distances appear closer• Patterns repeating across

larger distances appear farther• Same pattern appear flat

Sailboat

Stereoscopic images

• Displays two images on the same surface• Viewer wears glasses to separate images• Anaglyph

• Red/blue images are superimposed• Red/blue lenses separate images

• Polarized• Images are projected with polarized light• Polarizes lenses separate images

• Eclipse• Images are displayed alternately• Active lenses shutter correspondingly to separate

images

• Stereoblindness - inability to perceive 3D from stereo vision

• 5% - 10% of the population is stereoblind

• In 2012, a man unexpectedly overcame stereoblindness while watching a 3D film. “It was just literally like a whole new dimension of sight.” [6]

Accommodation

• Ciliary body• Part of eye which changes lens shape to

focus image

• Feedback from eye muscles suggests how near or far image in focus is• Lens shape correlates with distance

• Works with convergence• Accommodation vs Defocus blur• Defocus blur is a visual cue• Accommodation is an oculomotor cue

Convergence

• Eyes move inward (converge) to focus on closer objects• Amount of convergence suggest how close

object is• Only useful at very close distances• Less convergence change at further distances

• Convergence vs Binocular disparity• Binocular disparity is a visual cue• Convergence is an oculomotor cue

Predators and prey

• Predators• Forward facing eyes• More stereo vision, less monocular vision• Narrower field of view• More focus on target

• Prey• Side facing eyes• Less stereo vision, more monocular vision• Larger field of view• Wider range of threat detection

Prey Predator

• Peregrine falcons [7]• Use monocular vision to see

sideways• Fly lateral to prey to surveil

with side view• Use binocular vision to see

forward• Dive at prey using front view

End

References[1] L. Renier et al, “Cross-modal activation of visual cortex during depth perception using auditory substitution of vision”, March 2005, NeuroImage 26

[2] Cultural Differences in the Perception of Geometric Illusions, M. Segall et al, February 1963, Science, New Series, Vol. 139, No. 3556

[3] On the Perception of Shape from Shading, D. Kleffner and V.S. Ramachandran, 1992, Perception & Psychophysics, Vol. 52 (1)

[4] http://webvision.med.utah.edu/2012/01/jumping-spiders-use-image-defocusing-for-depth-perception/ Referenced: 5/3/2015

[5] Estimating Spatially Varying Defocus Blur From A Single Image, X. Zhu et al, December 2013, IEEE Transactions on Image Processing, Vol. 22, No. 12

[6] http://www.bbc.com/future/story/20120719-awoken-from-a-2d-world Referenced: 5/4/2015

[7] Peregrine Falcons: Stories of the Blue Meanie, J. H. Enderson, March 2005, University of Texas Press