gordon wetzstein douglas lanman wolfgang heidrich ramesh ... · ubc mit media lab. 3 overview...
TRANSCRIPT
Layered 3D Tomographic Image Synthesis for Attenuation-
based Light Field and High Dynamic Range Displays
Gordon Wetzstein
Wolfgang Heidrich
Douglas Lanman
Ramesh Raskar
UBC MIT Media Lab
3
Overview
Attenuation Layers
with Spacers
Backlight
4
Key Insights
Multi-Layer Display
Computed
Tomography
4D Light Field
5
Glasses-Free 3D Display
Lenslet Arrays Parallax Barriers Layered 3D
Nin
ten
do 3
DS
Alio
sco
py 3
DH
D 4
2”
Ives 1903 Lippmann 1908
sourc
e: w
ikip
edia
6
Gotoda 2010, 2011
Generalizing Parallax Barriers
mask 1
mask 2
mask 1
mask 2
mask 2
mask 3
mask K
mask 1
… Conventional Parallax Barriers
Lanman et al. 2010
Layered 3D
Tem
pora
l M
ultip
lexin
g
Multiple Layers
We are first to analyze problem and build prototypes
7
Computed Tomography (CT)
sourc
e: w
ikip
edia
x-ray source
x-ray sensor
3D Reconstruction
Reconstructed 2D Slices
8
Tomographic Light Field Synthesis
q
2D Light Field
x
x q
Backlight
Attenuation Volume
Virtual Planes Image Formation
L(x,q ) = e- m (r )dr
cò
c
drrxL )(,log q
P)log( L2
20
P)log(argmin
L
Tomographic Synthesis
9
CT vs. Layered 3D
Computed Tomography Layered 3D
reconstruct physical volume
sensor noise
thin stack of optimized layers
no noise
10
Multi-Layer Decomposition
Input 4D Light Field
Optimized Attenuation Layers
1 2
3 4
5 1 2 3 4 5
Photographs of Prototype
viewer moves right
vie
wer m
oves d
ow
n
11
Depth of Field for 3D Displays Integral Imaging Parallax Barriers
Antialiasing + Display Prefilter Antialiasing + Display Prefilter Zwicker et al. 2006
Display Thickness
Distance of Virtual Plane from Middle of Display (cm)
Cu
toff (
cycle
s/c
m)
Maximum Resolution
12
Integral Imaging Parallax Barriers Layered 3D
?
Distance of Virtual Plane from Middle of Display (cm)
Cu
toff (
cycle
s/c
m)
Display Thickness
Maximum Resolution
How Do Layers Increase Depth of Field?
13
h u
a
backlight Layered 3D
do
ξ
Chai et al. 2000; Durand et al. 2005;
Veeraraghavan et al. 2007;
Lanman et al. 2008; Ihrke et al. 2010
Fourier Transform
u
a
Light Field
fu
fa
f0 -f0
Light Field Spectral Support
attenuator
Review of Frequency-Domain Light Field Analysis
?
14
u
a
h u
a
u
a
Layered 3D
Ä
Fourier Transform
Rear Layer Light Field Front Layer Light Field
fu
fa
f0 -f0
Rear Layer Light Field Spectral Support
fu
fa
f0 -f0
Front Layer Light Field Spectral Support
backlight
attenuator
Multi-Layer Frequency-Domain Analysis
15
u
a
h
u hb
backlight Parallax Barrier
Layered 3D
fu
fa
Emitted Light Field Spectral Support
backlight
attenuator
Multi-Layer Frequency-Domain Analysis
16
u
a
h
fξ
do
ξ
fu
fa
fξ
Layered 3D
Two-Layer Depth of Field
Two-Layer Spectral Support
backlight
attenuator
Distance of Virtual Plane from Middle of Display (cm)
Cuto
ff (
cycle
s/c
m)
Depth of Field
Multi-Layer Frequency-Domain Analysis
17
u
a
h
do
ξ
fu fu
fa fa
fξ
fξ
Layered 3D
Two-Layer Spectral Support Three-Layer Spectral Support
backlight
attenuator
fξ
Distance of Virtual Plane from Middle of Display (cm)
Cuto
ff (
cycle
s/c
m)
Depth of Field
Three-Layer Depth of Field
Multi-Layer Frequency-Domain Analysis
18
Multi-Layer Depth of Field
Conventional*
Layered 3D *Includes integral imaging and parallax barriers
Distance of Virtual Plane from Middle of Display (cm)
Cu
toff (
cycle
s/c
m)
19
Optimization: Number of Layers
Two Layers Three Layers Five Layers
20
Optimization: Display Thickness
Number of Layers
PS
NR
in d
B
Average Reconstruction PSNR for All Scenes
21
Application to HDR Display
“Square Root” Layers
22
Application to HDR Display
“Square Root” Layers
23
Application to HDR Display
Optimized Layers
25
Limitations: Field of View
FOV 10º FOV 20º FOV 45º
26
Personal Glasses-Free 3D Display
Challenges for dynamic display:
Real-time computation
Engineering issues, moiré
27
Dynamic Multi-Layer LCDs
Dynamic Light Field Display using Multi-Layered LCDs, to appear in Siggraph Asia 2011 Douglas Lanman, Gordon Wetzstein, Matthew Hirsch, Wolfgang Heidrich, Ramesh Raskar
28
Photo-stéréo-synthesis Louis Lumière 1920
Focal Stack
Layered
Transparencies 3D effect, but not correct!
Computational Displays
Computational Photography
29
www.layered3d.com
Datasets, code & videos on the website!
30
31
3D objects only inside enclosure
mostly mechanically moving parts
all depth cues
computationally expensive Holograms
inexpensive fabrication
computationally efficient
3D objects outside enclosure
no moving parts
Volumetric Displays
Glasses-Free 3D Display
Zebra Imaging MIT Holovideo Jones et al. 2007 Sony LightSpace
Layered 3D
32
Limitations: “Flip Animations”
Parallax Barriers Multi-Layer
Full Resolution
Multi-Layer
Reduced Resolution
33
Performance Assessment
Prototype Prototype Prototype
Input Input Input Expected Error Expected Error Expected Error
34
Application to HDR Display
bottom bottom
bottom bottom
top top
top top
Se
etz
en
20
04
, D
olb
y C
an
ad
a
LCD
LED Array
35
Implementation
Prototype in Front of LCD (Backlight)
Epson Stylus 2200 inkjet (300 dpi, six color primaries)
5 layers (5.7×7.6 cm),1.27 cm thickness, 10° field of view
Transparencies and Acrylic Spacers Inkjet Transparency Printer *
*not „90s, try eBay
36
Implementation: Software viewer moves right
vie
wer m
oves d
ow
n 1
2 3
4 5
1 2 3
4 5
POV-Ray: 7×7 views (512×384 pixels), 10° field of view Depth of field tuned for combined antialiasing and display prefilter
MATLAB: LSQLIN (independently for each color channel)
12 minutes on 2.4 GHz Intel Core 2 with 8 GB RAM