user issues in 3d tv & cinema
DESCRIPTION
User Issues in 3D TV & Cinema. Martin S. Banks Vision Science Program UC Berkeley. Issues in 3D TV & Cinema. Technical Issues Developing content Sufficient resolution over time: temporal aliasing Sufficient separation between two eyes’ images: “ghosting” User Issues - PowerPoint PPT PresentationTRANSCRIPT
User Issues in 3D TV & Cinema
Martin S. BanksVision Science Program
UC Berkeley
Issues in 3D TV & Cinema
Technical Issues• Developing content• Sufficient resolution over time: temporal aliasing• Sufficient separation between two eyes’ images: “ghosting”
User Issues• Perceptual distortions due to incorrect viewing position• Flicker & motion judder due to temporal sampling• Maintaining depth across scene cuts• Window violations• Residual ghosting• Visual discomfort due to vergence-accommodation conflict• Appropriate blur relative to other depth signals• Conflict between visually-induced motion & vestibular signals
Technical Issues• Developing content• Sufficient resolution over time: temporal aliasing• Sufficient separation between two eyes’ images: “ghosting”
User Issues• Perceptual distortions due to incorrect viewing position• Flicker & motion judder due to temporal sampling• Maintaining depth across scene cuts• Window violations• Residual ghosting• Visual discomfort due to vergence-accommodation conflict• Appropriate blur relative to other depth signals• Conflict between visually-induced motion & vestibular signals
Issues in 3D TV & Cinema
Technical Issues• Developing content• Sufficient resolution over time: temporal aliasing• Sufficient separation between two eyes’ images: “ghosting”
User Issues• Perceptual distortions due to incorrect viewing position• Flicker & motion judder due to temporal sampling• Maintaining depth across scene cuts• Window violations• Residual ghosting• Visual discomfort due to vergence-accommodation conflict• Appropriate blur relative to other depth signals• Conflict between visually-induced motion & vestibular signals
Issues in 3D TV & Cinema
Foca
l dis
tanc
e
Verg
ence
dis
tanc
eVergence & Accommodation: Natural Viewing
3 6
1.5
3
4.5
6
Vergence Distance (diopters)00
Foca
l Dis
tanc
e (d
iopt
ers)
Foca
l dis
tanc
e
Verg
ence
dis
tanc
e
1.5 4.5
Vergence & Accommodation: Natural Viewing
3 6
1.5
3
4.5
6
Vergence Distance (diopters)00
zone of clear singlebinocular vision
Foca
l Dis
tanc
e (d
iopt
ers)
Foca
l dis
tanc
e
Verg
ence
dis
tanc
e
1.5 4.5
Vergence & Accommodation: Natural Viewing
3 6
1.5
3
4.5
6
Vergence Distance (diopters)00
Percival's zoneof comfort
zone of clear singlebinocular vision
Foca
l Dis
tanc
e (d
iopt
ers)
Foca
l dis
tanc
e
Verg
ence
dis
tanc
e
1.5 4.5
Vergence & Accommodation: Natural Viewing
Foca
l dis
tanc
e
Verg
ence
dis
tanc
eVergence & Accommodation: Stereo Display
3 6
1.5
3
4.5
6
Vergence Distance (diopters)00
Foca
l Dis
tanc
e (d
iopt
ers)
1.5 4.5
Foca
l dis
tanc
e
Verg
ence
dis
tanc
eVergence & Accommodation: Stereo Display
Percival's zoneof comfort
zone of clear singlebinocular vision
Displays with Nearly Correct Focus Cues
Two multi-focal displays we’ve developed:1.Fixed-viewpoint, volumetric display with mirror system & 3 focal
planes (Akeley, Watt, Girshick, & Banks, SIGGRAPH, 2004).
2.Fixed-viewpoint, volumetric display with switchable lens & 4 focal planes (Love, Hoffman, Kirby, Hands, Gao, & Banks, Optics Express, 2009)
Multi-focal Display
Akeley, Watt, Girshick & Banks (2004), SIGGRAPH.
Akeley, Watt, Girshick & Banks (2004), SIGGRAPH.
Multi-focal Display
Akeley, Watt, Girshick & Banks (2004), SIGGRAPH.
Multi-focal Display
Depth-weighted Blending
• Depth-weighted blending along lines of sight
• Weights dependent on dioptric distances to planes
Akeley, Watt, Girshick, & Banks (2004), SIGGRAPH.
Do V-A Conflicts Cause Fatigue/Discomfort?
• 600-ms stimulus at near or far vergence-specified distance• Appeared at each focal distance
Hoffman, Girshick, Akeley, & Banks (2008), Journal of Vision
Do V-A Conflicts Cause Fatigue/Discomfort?
** ** ** **
cues-inconsistentcues-consistent
Sev
erity
of S
ympt
om
1
3
5
7
9
How tired are your eyes?
How clear is your vision?
How tired or sore are your
neck & back?
How do your eyes feel?
How does your head feel?
** = p < 0.01 (Wilcoxen test)
Hoffman, Girshick, Akeley, & Banks (2008), Journal of Vision
Do V-A Conflicts Cause Fatigue/Discomfort?
*******Which session was
more fatiguing?
Which session irritated
your eyes more?
Which session gave you
more headache?
Which session did
you prefer?
cues-consistent much worse than
inconsistent
cues-inconsistent much worse than
consistent
no difference
** = p < 0.01 (Wilcoxen test)
Hoffman, Girshick, Akeley, & Banks (2008), Journal of Vision
Do V-A Conflicts Cause Fatigue/Discomfort?
Discomfort & 3D Cinema
Discomfort & 3D Cinema
Discomfort & 3D Cinema
Discomfort & 3D Cinema
Discomfort & 3D Cinema & TV
Technical Issues• Developing content• Sufficient resolution over time: temporal aliasing• Sufficient separation between two eyes’ images: “ghosting”
User Issues• Perceptual distortions due to incorrect viewing position• Flicker & motion judder due to temporal sampling• Maintaining depth across scene cuts• Window violations• Residual ghosting• Visual discomfort due to vergence-accommodation conflict• Appropriate blur relative to other depth signals• Conflict between visually-induced motion & vestibular signals
Issues in 3D TV & Cinema
Almost never view pictures from correct position.
Retinal image thus specifies different scene than depicted.
Do people compensate, and if so, how?
Viewing Pictures
Stimuli
Vishwanath, Girshick, & Banks (2005), Nature Neuroscience.
Stimulus: simulated 3D ovoid with variable aspect ratio.Task: adjust ovoid until appears spherical.
Vary monitor slant Sm to assess compensation for oblique viewing positions.
Spatial calibration procedure.If compensate, will set ovoid to sphere on screen (ellipse on retina).
Observation Point
Sm
CRT
Experimental Task
Vishwanath, Girshick, & Banks (2005), Nature Neuroscience.
Center of Projection
Observation Point
No compensation: set ovoid to make image on retina circular:
retinal coordinates
screen coordinates
Predictions
Center of Projection
Observation Point
Compensation: Set ovoid to make image on screen circular:
Predictions
retinal coordinates
screen coordinates
Asp
ect R
atio
(scr
een
coor
ds)
1
1.2
1.4
-40 -20 0 20 40
invariance predictions
Viewing Angle Sm (deg)
Sm
Predictions
Predictions
1
1.2
1.4
-40 -20 0 20 40
invariance predictionsretinal predictions
Asp
ect R
atio
(scr
een
coor
ds)
Viewing Angle Sm (deg)
Sm
Results
1
1.2
1.4
-40 -20 0 20 40
monoc-aperture
invariance predictsretinal predicts
Asp
ect R
atio
(scr
een
coor
ds)
Viewing Angle (deg)
JLL
Vishwanath, Girshick, & Banks (2005), Nature Neuroscience.
Results
1
1.2
1.4
-40 -20 0 20 40
monoc-aperture
binoc-no aperture invariance predictsretinal predicts
JLL
Results
Asp
ect R
atio
(scr
een
coor
ds)
Viewing Angle (deg)Vishwanath, Girshick, & Banks (2005), Nature Neuroscience.
Compensation for Incorrect Viewing Position
• Pictures not useful unless percepts are robust to changes in viewing position.
• People compensate for oblique viewing position when viewing 2d pictures.
• Two theories of compensation: pictorial & surface. Data clearly favor surface compensation.
• Two versions of surface method: global & local. Data clearly favor local slant.
2D Pictures vs 3D Pictures
• Two eyes presented same image
• Binocular disparities specify orientation & distance of picture surface; hence useful for compensation
2D
• Two eyes presented different images
• Binocular disparities specify orientation & distance of picture surface and layout of picture contents; hence not useful for compensation
• Two eyes presented same image
• Binocular disparities specify orientation & distance of picture surface; hence
useful for compensation
3D
• Two eyes presented same image
• Binocular disparities specify orientation & distance of picture surface; hence useful for compensation
2D
2D Pictures vs 3D Pictures
Stereo (3D) Pictures
• For most applications, viewers will not be at correct position.
• Retinal disparities thus specify a different layout than depicted.
• Do people compensate?• Is correct seating position for a
3D movie more important than for 2D movie?
Stereo Picture Geometry
display surface
stereo projectors
display surface
stereo projectors
depicted hinge
Stereo Picture Geometry
display surface
stereo projectors
depicted hinge
Stereo Picture Geometry
display surface
stereo projectors
depicted hinge
disparity-specified hinge
Stereo Picture Geometry
perceiveddihedralangle?
display surface
stereo projectors
depicted hinge
disparity-specified hinge
Stereo Picture Geometry
Predictions
0 25 450
30
60
90
120
Viewing Angle (deg)
35°
17.5°
0°
-17.5°
-35°
Hin
ge S
ettin
g (d
eg) Invariance: Hinge settings
are 90° for all viewing angles and base slants
Retinal disparity: Hinge settings vary significantly with viewing angle & base slant
Viewing Angle (deg)
0 25 45
30
60
90
120
Hin
ge S
ettin
g (d
eg)
Results
non-stereo pictures
Viewing Angle (deg)
0 25 45
30
60
90
120
stereo pictures
0 25 45
30
60
90
120
Hin
ge S
ettin
g (d
eg)
Results
non-stereo pictures
Viewing Angle (deg)
0 25 45
30
60
90
120
0 25 45
30
60
90
120
Hin
ge S
ettin
g (d
eg)
non-stereo pictures stereo pictures
Results
Summary
• User issues in 3D cinema & TV• Vergence-accommodation conflicts cause visual fatigue &
discomfortCan be handled by attending to viewer’s distance from screen & range of disparities presented relative to screen
• Perceptual distortions due to incorrect viewing positionCompensation is good with non-stereo pictures
Compensation is significantly poorer with stereo pictures suggesting that viewer position could be more important
Acknowledgments
• Kurt Akeley (Microsoft)• Simon Watt (Univ. of Wales, Bangor)• Ahna Girshick (NYU)• David Hoffman (UC Berkeley)• Robin Held (UC Berkeley)• Funding from NIH, NSF, & Sharp Labs