stereo and 3d displays - matt hirsch
DESCRIPTION
TRANSCRIPT
![Page 1: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/1.jpg)
➢Introduction: History and Physiology Display Taxonomy Multi-view Rendering using OpenGL/GLSL Designing Content for Glasses-free 3D Displays Emerging Technology
Stereo and 3D Displays
![Page 2: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/2.jpg)
Monocular Depth Cues Supported by Conventional Displays relative and familiar size perspective and occlusion texture gradient, shading and lighting, atmospheric effects
Limitations of Conventional Displays
![Page 3: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/3.jpg)
Limitations of Conventional DisplaysLimitations of Conventional Displays
Monocular Depth Cues with Conventional Displays relative and familiar size perspective and occlusion texture gradient, shading and lighting, atmospheric effects
![Page 4: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/4.jpg)
Additional Monocular Depth Cues motion parallax [Hermann von Helmholtz, 1866] accommodation
What is missing?
![Page 5: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/5.jpg)
Binocular Depth Cues retinal disparity [Charles Wheatstone, 1838] convergence
“It being thus established that the mind perceives an object of three dimensions by means of the two dissimilar pictures projected by it on the two retinae, the following question occurs: What would be the visual effect of simultaneously presenting to each eye, instead of the object itself, its projection on a plane surface as it appears to that eye?”
Binocular Depth Cues
![Page 6: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/6.jpg)
American Civil War-era stereoscopic photos
• Available from the US library of congress•http://www.loc.gov/pictures/search - Search for “stereographs civil war prints”
• Lincoln in 3D• Selection of stereographs converted to red-
cyan anaglyph images•John J. Richter: ISBN 978-0811872317
Interesting Historical Example
![Page 7: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/7.jpg)
l The HVS can ignore conflicting or missing depth cuesl Understand depth in 2D (monocular) videol Perceive shape in “noise”
Ponzo Illusion: © Walt Anthony 2006 magiceye.com
Conflicting Cues
![Page 8: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/8.jpg)
Stereo and 3D Displays
Introduction: History and Physiology
➢Display Taxonomy Multi-view Rendering using OpenGL/GLSL Designing Content for Glasses-free 3D Displays Emerging Technology
![Page 9: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/9.jpg)
Taxonomy of 3D Displays:Glasses-bound vs. Unencumbered Designs
Glasses-boundStereoscopic
Immersive(blocks direct-viewing of real world)
See-through(superimposes synthetic images onto real world)
Head-mounted(eyepiece-objective and microdisplay)
Multiplexed (stereo pair with same display surface)
Spatially-multiplexed (field-concurrent)(color filters, polarizers, autostereograms, etc.)
Temporally-multiplexed (field-sequential)(LCD shutter glasses)
UnencumberedAutomultiscopic
Parallax-based(2D display with light-directing elements)
Volumetric(directly illuminate points within a volume)
Holographic(reconstructs wavefront using 2D element)
Parallax Barriers(uniform array of 1D slits or 2D pinhole arrays)
Integral Imaging(lenticular sheets or fly’s eye lenslet arrays)
Multi-planar(time-sequential projection onto swept surfaces)
Transparent Substrates(intersecting laser beams, fog layers, etc.)
Static(holographic films)
Dynamic(holovideo)
Taxonomy adapted from Hong Hua
![Page 10: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/10.jpg)
Taxonomy of 3D Displays:Immersive Head-mounted Displays (HMDs)
Glasses-boundStereoscopic
Immersive(blocks direct-viewing of real world)
Head-mounted(eyepiece-objective and microdisplay)
Multiplexed (stereo pair with same display surface)
![Page 11: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/11.jpg)
Taxonomy of 3D Displays:See-through Head-mounted Displays (HMDs)
Glasses-boundStereoscopic
Immersive(blocks direct-viewing of real world)
See-through(superimposes synthetic images onto real world)
Head-mounted(eyepiece-objective and microdisplay)
Multiplexed (stereo pair with same display surface)
![Page 12: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/12.jpg)
Taxonomy of 3D Displays:Spatial Multiplexing (e.g., Anaglyphs)
Glasses-boundStereoscopic
Immersive(blocks direct-viewing of real world)
See-through(superimposes synthetic images onto real world)
Head-mounted(eyepiece-objective and microdisplay)
Multiplexed (stereo pair with same display surface)
Spatially-multiplexed (field-concurrent)(color filters, polarizers, etc.)
![Page 13: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/13.jpg)
Taxonomy of 3D Displays:Temporal Multiplexing (e.g., Shutter Glasses)
Glasses-boundStereoscopic
Immersive(blocks direct-viewing of real world)
See-through(superimposes synthetic images onto real world)
Head-mounted(eyepiece-objective and microdisplay)
Multiplexed (stereo pair with same display surface)
Spatially-multiplexed (field-concurrent)(color filters, polarizers, autostereograms, etc.)
Temporally-multiplexed (field-sequential)(LCD shutter glasses)
![Page 14: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/14.jpg)
Taxonomy of 3D Displays:Parallax Barriers
UnencumberedAutomultiscopic
Parallax-based(2D display with light-directing elements)
Volumetric(directly illuminate points within a volume)
Holographic(reconstructs wavefront using 2D element)
Parallax Barriers(uniform array of 1D slits or 2D pinhole arrays)
NewSight MV-42AD3 42''(1920x1080, 1x8 views)
![Page 15: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/15.jpg)
Taxonomy of 3D Displays:Integral Imaging
UnencumberedAutomultiscopic
Parallax-based(2D display with light-directing elements)
Volumetric(directly illuminate points within a volume)
Holographic(reconstructs wavefront using 2D element)
Parallax Barriers(uniform array of 1D slits or 2D pinhole arrays)
Integral Imaging(lenticular sheets or fly’s eye lenslet arrays)
Alioscopy 3DHD 42''(1920x1200, 1x8 views)
![Page 16: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/16.jpg)
Taxonomy of 3D Displays:Multi-planar Volumetric Displays
UnencumberedAutomultiscopic
Parallax-based(2D display with light-directing elements)
Volumetric(directly illuminate points within a volume)
Holographic(reconstructs wavefront using 2D element)
Parallax Barriers(uniform array of 1D slits or 2D pinhole arrays)
Integral Imaging(lenticular sheets or fly’s eye lenslet arrays)
Multi-planar(time-sequential projection onto swept surfaces)
![Page 17: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/17.jpg)
Taxonomy of 3D Displays:Transparent-substrate Volumetric Displays
UnencumberedAutomultiscopic
Parallax-based(2D display with light-directing elements)
Volumetric(directly illuminate points within a volume)
Holographic(reconstructs wavefront using 2D element)
Parallax Barriers(uniform array of 1D slits or 2D pinhole arrays)
Integral Imaging(lenticular sheets or fly’s eye lenslet arrays)
Multi-planar(time-sequential projection onto swept surfaces)
Transparent Substrates(intersecting laser beams, fog layers, etc.)
![Page 18: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/18.jpg)
Taxonomy of 3D Displays:Static Holograms
UnencumberedAutomultiscopic
Parallax-based(2D display with light-directing elements)
Volumetric(directly illuminate points within a volume)
Holographic(reconstructs wavefront using 2D element)
Parallax Barriers(uniform array of 1D slits or 2D pinhole arrays)
Integral Imaging(lenticular sheets or fly’s eye lenslet arrays)
Multi-planar(time-sequential projection onto swept surfaces)
Transparent Substrates(intersecting laser beams, fog layers, etc.)
Static(holographic films)
capture reconstruction
![Page 19: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/19.jpg)
Taxonomy of 3D Displays:Dynamic Holograms (Holovideo)
UnencumberedAutomultiscopic
Parallax-based(2D display with light-directing elements)
Volumetric(directly illuminate points within a volume)
Holographic(reconstructs wavefront using 2D element)
Parallax Barriers(uniform array of 1D slits or 2D pinhole arrays)
Integral Imaging(lenticular sheets or fly’s eye lenslet arrays)
Multi-planar(time-sequential projection onto swept surfaces)
Transparent Substrates(intersecting laser beams, fog layers, etc.)
Static(holographic films)
Dynamic(holovideo)
Tay et al. [Nature, 2008]
MIT Media Lab Spatial Imaging Group[Holovideo, 1989 – present]
![Page 20: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/20.jpg)
Stereo and 3D Displays
Introduction: History and Physiology Display Taxonomy
➢Multi-view Rendering using OpenGL/GLSL Designing Content for Glasses-free 3D Displays Emerging Technology
![Page 21: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/21.jpg)
Overview:GLSL: Programmable Pipeline
Fixed Function Pipeline
Simple 1-Slide Explanati
on!
Drawing APIDrawing API
Process VerticesProcess Vertices
Process PixelsProcess Pixels
FramebufferFramebuffer
Programmable Pipeline
Vertex ProgramVertex Program
Fragment ProgramFragment Program
![Page 22: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/22.jpg)
l Some graphics cards have support for stereo 3D (Not on mobile)l Double buffered stereo = Quad buffered
voiddisplay(void){ glDrawBuffer(GL_BACK_LEFT);
<Draw left eye here>
glDrawBuffer(GL_BACK_RIGHT); <Draw right eye here>
glutSwapBuffers();}
intmain(int argc, char **argv){ glutInit(&argc, argv); glutInitDisplayMode(
GLUT_DOUBLE | GLUT_RGB | GLUT_STEREO); glutCreateWindow("stereo example"); glutDisplayFunc(display); glutMainLoop(); return 0;}
Anaglyphic Model Viewer:Stereo 3D in OpenGL
![Page 23: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/23.jpg)
Overview:Multi-View Rendering in OpenGL
OpenGL Draw Calls
Render
Standard Pipeline
Output
Multi-View Pipeline
Loop Over Views
BackbufferFramebuffer Object Array
Render View
Change Camera
Screen:Memory:
![Page 24: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/24.jpg)
Overview:Multi-View Interlacing using GLSL Shaders
Framebuffer Object Array
Framebuffer Object Array
View 1View 2
View 3
GLSL ProgramTranslate views appropriately for
output device
Translate views appropriately for
output device
BackbufferBackbufferAnaglyph GlassesAnaglyph Glasses LenticularLenticular
Shown in this course…
The model can apply to many others
![Page 25: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/25.jpg)
Multi-View Rendering in OpenGL:Off-Axis Perspective Projection with glFrustum()
Output
![Page 26: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/26.jpg)
Anaglyphic Model Viewer:Anaglyph Compositing Algorithms
LL RR3x3 Color Transform Matrix Pair3x3 Color Transform Matrix Pair
Full Color
Half Color
Optimized
L= R=1 0 00 0 00 0 0
0 0 00 1 00 0 1
L= R=0.299 0 00.587 0 00.114 0 0
0 0 00 1 00 0 1
L= R=0 0 00.7 0 00.3 0 0
0 0 00 1 00 0 1
=
Source: http://3dtv.at/Knowhow/AnaglyphComparison_en.aspx
![Page 27: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/27.jpg)
% read in imagesImL = imread('l.png');ImR = imread('r.png');
% define "half color" matrices (see slides)L = [.299 0 0 .587 0 0 .114 0 0];
R = [0 0 0 0 1 0 0 0 1];
% create a pixel x color arrayImL1d = double(reshape(ImL,prod(size(ImL(:,:,1))),3));ImR1d = double(reshape(ImR,prod(size(ImR(:,:,1))),3));
% perform per pixel color permutationImL1d = ImL1d*L;ImR1d = ImR1d*R;
Anaglyphic Model Viewer:Making an Anaglyph Image in MATLAB
% convert back to 2d x color imageImL = uint8(reshape(ImL1d,size(ImL)));ImR = uint8(reshape(ImR1d,size(ImR)));
% create outputIout = ImL + ImR;
anaglyph.m
![Page 28: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/28.jpg)
Anaglyphic Model Viewer:Demonstration
![Page 29: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/29.jpg)
Stereo and 3D Displays
Introduction: History and Physiology Display Taxonomy Multi-view Rendering using OpenGL/GLSL
➢Designing Content for Glasses-free 3D Displays Emerging Technology
![Page 30: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/30.jpg)
• Stereo cameras (commercial and improvised) are common
Source DataStereo Cameras
![Page 31: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/31.jpg)
• Many researchers/hobbyists have built their own solutions to capture light fields
• The PointGrey ProFusion is one of the few commercially available
PointGrey ProFusion
Stanford
Source DataLight Field Cameras
MIT
![Page 32: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/32.jpg)
Focal Plane
Example in Anaglyph Viewer
Screen
Virtual Object
Placing objects farm from the plane of focus is uncomfortable
Displays with limited DOF: objects further from screen plane are blurred
Rendering TipsAccommodation & Object Placement
![Page 33: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/33.jpg)
Focal Plane
Screen
Kirshnan, V. V., Stark, L. A heuristic model for the human vergence eye movement system, IEEE Trans. BioMed, 1977.
Limit distance of virtual object to viewerLimit rate of change in scene distance
<1 m/s for distant objects
Rendering TipsComfortable Vergence
![Page 34: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/34.jpg)
Off-axis parallel projection Rotate and translate – Toe-in
• Puts ‘infinity’ at axis of rotation•Requires user to focus beyond infinity•Some find diverged eyes uncomfortable
Disadvantages of toe-in • Distortion between views•Camera distance to most objects change•Off axis objects will have different perspective projection
Rendering TipsCamera Model Choice
![Page 35: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/35.jpg)
Avoid cases that cause a view to differ greatly from its neighbor Left Right
Pillar pointing at viewer
Left Right
Clipped by edge of screen
Also watch out for• Far objects clipped by near object• Edges of hallways, tunnels, tubes,
etc
Also watch out for• Don’t exit in front of screen plane• More comfortable behind screen• Avatar does a good job with this
Rendering TipsClipping and Degenerate Cases
![Page 36: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/36.jpg)
Warzone 2100: GL Game Conversion
![Page 37: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/37.jpg)
Stereo and 3D Displays
Introduction: History and Physiology Display Taxonomy Multi-view Rendering using OpenGL/GLSL Designing Content for Glasses-free 3D Displays
➢Emerging Technology
![Page 38: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/38.jpg)
Tensor Display
![Page 39: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/39.jpg)
Tensor Display
![Page 40: Stereo and 3D Displays - Matt Hirsch](https://reader034.vdocuments.site/reader034/viewer/2022042713/5404fa558d7f729b768b49de/html5/thumbnails/40.jpg)
Stereo and 3D Displays Resources
SIGGRAPH 2010/2011 Course: BYO3Dhttp://web.media.mit.edu/~mhirsch/byo3d/index.html Long-form slides Code and examples
display blocks
Display Blocks bloghttp://displayblocks.org Tutorials Building blocks explained
Gordon [email protected]
Matt [email protected]