seeing 3d from 2d imagesasdad
TRANSCRIPT
-
8/13/2019 Seeing 3D From 2D Imagesasdad
1/47
Seeing 3D from 2D
ImagesWilliam and Craig 115 - 164
-
8/13/2019 Seeing 3D From 2D Imagesasdad
2/47
How to make a 2D image appear as3D!
Output is typically 2D Images
Yet we want to show a 3D world!
How can we do this? We can include cues in the image that give our
brain 3D information about the scene
These cues are visual depth cues
-
8/13/2019 Seeing 3D From 2D Imagesasdad
3/47
Visual Depth Cues
Monoscopic Depth Cues (single 2D image)
Stereoscopic Depth Cues (two 2D images)
Motion Depth Cues (series of 2D images)Physiological Depth Cues (body cues)
-
8/13/2019 Seeing 3D From 2D Imagesasdad
4/47
Monoscopic Depth Cues
Interposition An object that occludes another is closer
Shading Shape info. Shadows are included here
Size Usually, the larger object is closer
Linear Perspective parallel lines converge at a single point
Surface Texture Gradient more detail for closer objects
Height in the visual field
Higher the object is (vertically), thefurther it is
Atmospheric effects further away objects are blurrier
Brightness further away objects are dimmer
-
8/13/2019 Seeing 3D From 2D Imagesasdad
5/47
Stereoscopic Display Issues
Stereopsis
Stereoscopic Display Technology
Computing Stereoscopic ImagesStereoscopic Display and HTDs.
Works for objects < 5m. Why?
-
8/13/2019 Seeing 3D From 2D Imagesasdad
6/47
Stereopsis
The result of the two slightly different views of theexternal world that our laterally-displaced eyes
receive.
-
8/13/2019 Seeing 3D From 2D Imagesasdad
7/47
Retinal Disparity
f1f2
Left Eye Right Eye
Retinal disparity =
If both eyes are fixated on apoint, f1, in space, then animage of f1 if focused atcorresponding points in the
center of the fovea of eacheye. Another point, f2, at adifferent spatial location wouldbe imaged at points in eacheye that may not be the same
distance from the fovea. Thisdifference in distance is theretinal disparity.
-
8/13/2019 Seeing 3D From 2D Imagesasdad
8/47
Disparity
If an object is closer than the fixation point, theretinal disparity will be a negative value. Thisis known as crossed disparitybecause the twoeyes must cross to fixate the closer object.
If an object is farther than the fixation point,the retinal disparity will be a positive value.This is known as uncrossed disparitybecausethe two eyes must uncross to fixate the fartherobject.
An object located at the fixation point or whoseimage falls on corresponding points in the tworetinae has a zero disparity.
-
8/13/2019 Seeing 3D From 2D Imagesasdad
9/47
Convergence Angles
i
f2
f1
D1
D2a b
c d
1
a+a+c+b+d = 180
b+c+d = 180
a-b= a+(-b) = 1+2
= Retinal Disparity
2
-
8/13/2019 Seeing 3D From 2D Imagesasdad
10/47
Miscellaneous Eye Facts
Stereoacuity- the smallest depth that canbe detected based on retinal disparity.
Visual Direction- Perceived spatiallocation of an object relative to an observer.
-
8/13/2019 Seeing 3D From 2D Imagesasdad
11/47
Horopters
Corresponding points onthe two retinae are definedas being the same verticaland horizontal distancefrom the center of the
fovea in each eye. Horopter - the locus of
points in space that fall oncorresponding points inthe two retinae when the
two eyes binocularly fixateon a given point in space(zero disparity).
Points on the horopterappear at the same depth
as the fixation point.
f1
f2
Vieth-Mueller
Circle
-
8/13/2019 Seeing 3D From 2D Imagesasdad
12/47
Stereoscopic Display
Stereoscopic images are easy to do badly,hard to do well, and impossible to do
correctly.
-
8/13/2019 Seeing 3D From 2D Imagesasdad
13/47
Stereoscopic Displays
Stereoscopic display systems create a three-dimensional image (versus a perspectiveimage) by presenting each eye with a
slightly different view of a scene.
Time-parallel
Time-multiplexed
-
8/13/2019 Seeing 3D From 2D Imagesasdad
14/47
Time Parallel StereoscopicDisplay
Two Screens
Each eye sees adifferent screen
Optical system directseach eye to the correctview.
HMD stereo is donethis way.
Single Screen
Two different images
projected on the samescreen
Images are polarizedat right angles to each
other.
User wears polarizedglasses (passive
glasses).
-
8/13/2019 Seeing 3D From 2D Imagesasdad
15/47
Passive Polarized Projection Issues
Linear Polarization
Ghosting increases when you tilt head
Reduces brightness of image by about
Potential Problems with Multiple Screens (nextslide)
Circular Polarization
Reduces ghosting but also reduces brightnessand crispness of image even more
-
8/13/2019 Seeing 3D From 2D Imagesasdad
16/47
Problem with Linear Polarization
With linear polarization,the separation of the leftand right eye images isdependent on the
orientation of the glasseswith respect to theprojected image.
The floor image cannot be
aligned with both the sidescreens and the frontscreens at the same time.
-
8/13/2019 Seeing 3D From 2D Imagesasdad
17/47
Time Multiplexed Display
Left and right-eye views of an image arecomputed and alternately displayed on thescreen.
A shuttering system occludes the right eyewhen the left-eye image is being displayedand occludes the left-eye when the right-
eye image is being displayed.
-
8/13/2019 Seeing 3D From 2D Imagesasdad
18/47
Stereographics Shutter Glasses
-
8/13/2019 Seeing 3D From 2D Imagesasdad
19/47
Screen Parallax
P
Left eye
position
Right eyeposition
Pleft
Pright
Pright
Pleft
P
Display
Screen
Object withpositive
parallax
Object with
negative parallax
The screen parallax is the distance between the projected locationof P on the screen, Pleft, seen by the left eye and the projected
location, Pright, seen by the right eye (different from retinal disparity).
-
8/13/2019 Seeing 3D From 2D Imagesasdad
20/47
Screen Parallax (cont.)
f1
p
i
d
Left
eyepoint
Right
eyepoint
Projection
Plane
D
p = i(D-d)/D
where p is the amount of screenparallax for a point, f1, whenprojected onto a plane adistance d from the plane
containing two eyepoints.i is the interocular distance
between eyepoints and
D is the distance from f1 to thenearest point on the plane
containing the two eyepointsd is the distance from the
eyepoint to the nearest pointon the screen
-
8/13/2019 Seeing 3D From 2D Imagesasdad
21/47
Screen Parallax
-65.00
-55.00
-45.00
-35.00
-25.00
-15.00
-5.00
5.00
0 50 100 150 200 250 300 350
Distance from Eye
Screen
Parallax
Zero parallax at screen, max positive parallax
is i, negative parallax is equal to I halfway
between eye and screen
-
8/13/2019 Seeing 3D From 2D Imagesasdad
22/47
Stereoscopic Voxels
Left EyePoint
Right EyePoint
1
1
1
1
1
1
1
1
1
2
2
2
2
2
2
2
3
3
3
3
3
3
3
4
4
4
4
4
4
5
5
5
5
5
2
A
B
-
8/13/2019 Seeing 3D From 2D Imagesasdad
23/47
Screen Parallax and ConvergenceAngles
f1f2 f3
ProjectionPlane
a
Screen parallax dependson closest distance toscreen.
Different convergenceangles can all have thesame screen parallax.
Also depends onassumed eyeseparation.
-
8/13/2019 Seeing 3D From 2D Imagesasdad
24/47
How to create correct left- andright-eye views
To specific a single view in almost allgraphics software or hardware you mustspecify:
Eyepoint
Look-at Point
Field-of-View or location of Projection Plane
View Up Direction
-
8/13/2019 Seeing 3D From 2D Imagesasdad
25/47
Basic Perspective Projection SetUp from Viewing Paramenters
Y
Z
X
Projection Plane is orthogonal to one of the major axes
(usually Z). That axis is along the vector defined by the
eyepoint and the look-at point.
-
8/13/2019 Seeing 3D From 2D Imagesasdad
26/47
What doesnt work
Each view has a different
projection plane
Each view will be presented
(usually) on the same plane
-
8/13/2019 Seeing 3D From 2D Imagesasdad
27/47
What Does Work
i i
-
8/13/2019 Seeing 3D From 2D Imagesasdad
28/47
Setting Up Projection Geometry
Look at pointEyeLocations
Look at points
Eye
Locations
No
Yes
-
8/13/2019 Seeing 3D From 2D Imagesasdad
29/47
Screen Size
The size of the window doesnot affect the retinal disparity
for a real window.
Once computed, the screen parallaxis affected by the size of the displayscreen
-
8/13/2019 Seeing 3D From 2D Imagesasdad
30/47
Visual Angle Subtended
Screen parallax is measured in terms of visual angle. This is a screenindependent measure. Studies have shown that the maximum anglethat a non-trained person can usually fuse into a 3D image is about
1.6 degrees. This is about 1/2 the maximum amount of retinal disparityyou would get for a real scene.
-
8/13/2019 Seeing 3D From 2D Imagesasdad
31/47
Accommodation/ Convergence
Display Screen
-
8/13/2019 Seeing 3D From 2D Imagesasdad
32/47
Position Dependence(without head-tracking)
l d
-
8/13/2019 Seeing 3D From 2D Imagesasdad
33/47
Interocular Dependance
F
Modeled Point
PerceivedPoint
Projection Plane
-
8/13/2019 Seeing 3D From 2D Imagesasdad
34/47
Obvious Things to Do
Head tracking
Measure Users Interocular Distance
-
8/13/2019 Seeing 3D From 2D Imagesasdad
35/47
Another Problem
Many people can not fuse stereoscopicimages if you compute the images withproper eye separation!
Rule of Thumb: Compute with about thereal eye separation.
Works fine with HMDs but causes image
stability problems with HTDs (why?)
-
8/13/2019 Seeing 3D From 2D Imagesasdad
36/47
Two View Points with Head-Tracking
Projection Plane
Modeled Point
Perceived Points
Modeled Eyes
True Eyes
-
8/13/2019 Seeing 3D From 2D Imagesasdad
37/47
Maximum Depth Plane
Maximum Depth PlaneModeled Eyes
True Eyes
EF
Modeled
Point
PerceivedPoint
ProjectionPlane
-
8/13/2019 Seeing 3D From 2D Imagesasdad
38/47
Can we fix this?
Zachary Wartell, "Stereoscopic Head-Tracked Displays: Analysis andDevelopment of Display Algorithms," Ph.D. Dissertation, GeorgiaInstitute of Technology, August 2001.
Zachary Wartell, Larry F. Hodges, William Ribarsky. "An AnalyticComparison of Alpha-False Eye Separation, Image Scaling and Image
Shifting in Stereoscopic Displays," IEEE Transactions on Visualizationand Computer Graphics,April-June 2002, Volume 8, Number 2, pp.129-143. (related tech report is GVU Tech Report 00-09 (Abstract ,PDF, Postscript.)
Zachary Wartell, Larry F. Hodges, William Ribarsky. "BalancingFusion, Image Depth, and Distortion in Stereoscopic Head-Tracked
Displays." SIGGRAPH 99 Conference Proceedings, Annual ConferenceSeries. ACM SIGGRAPH, Addison Wesley, August 1999, p351-357.(Paper:Abstract, PDF, Postscript; SIGGRAPH CD-ROM Supplement,supplement.zip,supplement.tar.Z).
http://www.gvu.gatech.edu/gvu/reports/2000/abstracts/00-09.htmlftp://ftp.cc.gatech.edu/pub/gvu/tr/2000/00-09.pdfftp://ftp.cc.gatech.edu/pub/gvu/tr/2000/00-09.ps.Zhttp://www.cc.gatech.edu/gvu/reports/1999/abstracts/99-32.htmlftp://ftp.cc.gatech.edu/pub/gvu/tr/1999/99-32.pdfftp://ftp.cc.gatech.edu/pub/gvu/tr/1999/99-32.ps.Zhttp://www.cc.gatech.edu/people/home/wartell/supplement.ziphttp://www.cc.gatech.edu/people/home/wartell/supplement.tar.Zhttp://www.cc.gatech.edu/people/home/wartell/supplement.tar.Zhttp://www.cc.gatech.edu/people/home/wartell/supplement.zipftp://ftp.cc.gatech.edu/pub/gvu/tr/1999/99-32.ps.Zftp://ftp.cc.gatech.edu/pub/gvu/tr/1999/99-32.pdfhttp://www.cc.gatech.edu/gvu/reports/1999/abstracts/99-32.htmlftp://ftp.cc.gatech.edu/pub/gvu/tr/2000/00-09.ps.Zftp://ftp.cc.gatech.edu/pub/gvu/tr/2000/00-09.pdfhttp://www.gvu.gatech.edu/gvu/reports/2000/abstracts/00-09.htmlhttp://www.gvu.gatech.edu/gvu/reports/2000/abstracts/00-09.html -
8/13/2019 Seeing 3D From 2D Imagesasdad
39/47
Point of fixation
Distance in centimeters from eye plane
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.40
10
20
30
40
50
60
70
80
90
10
0
11
0
12
0
13
0
14
0
15
0
Symmetric convergence
Convergence 20 centime ters to the left of the lef t eye
Change in eyepoint separation with change in point of fixation.Centers of rotation of the eyes are assumed to be 6.4 centimeters apart.
-
8/13/2019 Seeing 3D From 2D Imagesasdad
40/47
Ghosting
Affected by the amount of light transmittedby the LC shutter in its off state.
Phosphor persistence
Vertical screen position of the image.
-
8/13/2019 Seeing 3D From 2D Imagesasdad
41/47
Ghosting (cont.)
Extinction Ratio =Luminance of the correct eye image
------------------------------------------------------------Luminance of the opposite eye ghost image
Image Position Red White
Top 61.3/1 17/1
Middle 50.8/1 14.4/1
Bottom 41.1/1 11/1
-
8/13/2019 Seeing 3D From 2D Imagesasdad
42/47
-
8/13/2019 Seeing 3D From 2D Imagesasdad
43/47
Time-parallel stereoscopic images
Image quality may also be affected by
Right and left-eye images do not match in color,size, vertical alignment.
Distortion caused by the optical system
Resolution
HMDs interocular settings
Computational model does not match viewinggeometry.
-
8/13/2019 Seeing 3D From 2D Imagesasdad
44/47
Motion Depth Cues
Parallax createdby relative headposition and
object beingviewed.
Objects nearer tothe eye move a
greater distance
-
8/13/2019 Seeing 3D From 2D Imagesasdad
45/47
Pulfrich Effect
Neat trick
Different levels of illumination requireadditional time (your frame rates differ base
of amount of light)What if we darken one image, and brighten
another?
http://dogfeathers.com/java/pulfrich.htmlwww.cise.ufl.edu/~lok/multimedia/videos/p
ulfrich.avi
http://dogfeathers.com/java/pulfrich.htmlhttp://dogfeathers.com/java/pulfrich.html -
8/13/2019 Seeing 3D From 2D Imagesasdad
46/47
Physiological Depth Cues
Accommodationfocusing adjustmentmade by the eye to change the shapeofthe lens. (up to 3 m)
Convergencemovement of the eyes tobring in the an object into the same locationon the retina of each eye.
-
8/13/2019 Seeing 3D From 2D Imagesasdad
47/47
Summary
MonoscopicInterposition is strongest.
Stereopsis is very strong.
Relative Motion is also very strong (orstronger).
Physiological is weakest (we dont even usethem in VR!)
Add as needed
ex. shadows and cartoons