image-based rendering of real objects with complex brdfs
Post on 21-Dec-2015
231 views
TRANSCRIPT
![Page 1: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/1.jpg)
Image-based Rendering of Real Objects Image-based Rendering of Real Objects with Complex BRDFswith Complex BRDFs
![Page 2: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/2.jpg)
Intensity of One PixelIntensity of One Pixel
s1(, )
q
p
Consider the measured intensity at one pixelConsider the measured intensity at one pixel
I1(1, 1)
as the isotropic point source is moved over as the isotropic point source is moved over the surface.the surface.
I1(, )
Note similarity to Note similarity to
•(Levoy, Hanrahan, 1996) (Levoy, Hanrahan, 1996)
•(Gortler et al, 1996)(Gortler et al, 1996)
![Page 3: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/3.jpg)
Phong Intensity of One Pixel: Phong Intensity of One Pixel: II11((, , ))
This is effectively a 2-D slice of a point’s BRDF except for This is effectively a 2-D slice of a point’s BRDF except for
• ShadowingShadowing
• 1/r1/r22 falloff from the source falloff from the source
![Page 4: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/4.jpg)
Intensity of One Pixel: Intensity of One Pixel: II11((, , ))
This is effectively a 2-D slice of a point’s BRDF except for This is effectively a 2-D slice of a point’s BRDF except for
• ShadowingShadowing
• 1/r1/r22 falloff from the source falloff from the source
![Page 5: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/5.jpg)
Image AcquisitionImage Acquisition
![Page 6: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/6.jpg)
Intensity Over Second SurfaceIntensity Over Second Surface
s1(, )
s2(, )
p
Now, consider moving an isotropic point Now, consider moving an isotropic point source over a second surface and measuring source over a second surface and measuring the intensity of the same pixel: the intensity of the same pixel:
I2(, )I1(, )
I2(, )
![Page 7: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/7.jpg)
II11((, , ) and I) and I22((, , ))
Inner
Sphere:
I1(, )
Outer
Sphere:
I2(, )
![Page 8: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/8.jpg)
Relation Between Intensity MapsRelation Between Intensity Maps
When the surface point p, When the surface point p, s1() and s2() are collinear (in correspondence), the measured pixel intensities are simply related by the relative 1/r2 losses.
s1(, )
s2(, )
p
![Page 9: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/9.jpg)
Depth EstimationDepth Estimation
s1(, )
s2(, )
p()
This correspondence can be expressed as a This correspondence can be expressed as a change of coordinates change of coordinates 22((; ; )) and and
22((; ; ) parameterized by depth ) parameterized by depth .
We can then estimateWe can then estimate by minimizing by minimizing:
O()= [I2(2(), 2 ()) - r2 I1(1,1) ]2d1d1
![Page 10: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/10.jpg)
A Reconstructed Depth MapA Reconstructed Depth Map
143 Images on 143 Images on each surfaceeach surface
![Page 11: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/11.jpg)
Rendering Synthetic Images: Point SourcesRendering Synthetic Images: Point Sources
New light position
Intersection with the sphere
• Intersect light ray throughIntersect light ray through P P with sphere.with sphere.
• Find triangle of light sources Find triangle of light sources containing containing PP..
• Interpolate pixel intensities of Interpolate pixel intensities of images corresponding to the images corresponding to the triangle vertices.triangle vertices.
• For a given image point, there For a given image point, there is a scene point: is a scene point: PP
PP
![Page 12: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/12.jpg)
Rendered ImagesRendered Images
![Page 13: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/13.jpg)
Rendered Image: A Sea ShellRendered Image: A Sea Shell
Isotropic point light source Isotropic point light source located between acquisition located between acquisition spheres.spheres.
![Page 14: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/14.jpg)
Rendered Image: A PearRendered Image: A Pear
• Two light sourcesTwo light sources• Point source to the leftPoint source to the left• 3 by 5 cm area source3 by 5 cm area source to the rightto the right
![Page 15: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/15.jpg)
Video Compositing of Real ObjectsVideo Compositing of Real Objects
Video Frame #567 Radiance Map Frame #567
![Page 16: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/16.jpg)
Video CompositingVideo Compositing
Background Image #2313 Object Image #2313
![Page 17: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/17.jpg)
Video CompositingVideo Compositing
Composite Frame #567 Composite Frame #567
![Page 18: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/18.jpg)
Lighting Sensitive Displays
Shree Nayar Peter Belhumeur Terry Boult
Columbia Yale Lehigh
Computer Vision Laboratory
Columbia University
Sponsor: NSF ITR
![Page 19: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/19.jpg)
Displays Everywhere
![Page 20: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/20.jpg)
But, Displays are Passive
brightness
contrast
![Page 21: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/21.jpg)
display
content
Lighting Sensitive Display (LSD)
• Senses the Environmental Illumination
• Modifies Displayed Content Accordingly
illumination
: Perception
: Reaction
![Page 22: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/22.jpg)
State of the Art
brightness
contrastphotodetector
adjustment
Heijligers 62 ; Thomas 63; Gibson 64; Korda 65; Biggs 65; Szermy 68Newman 72; Constable 78; Fitzgibbon 82; Antwerp 85; Otenstein 93
![Page 23: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/23.jpg)
Display’s Illumination Field
display
content
display
content
(s,t)(u,v)
L(s,t,u,v,
• Wide Range of Sources: Sunlight, Overcast, Halogen, Fluorescent ...
• Arbitrarily Complex : Point/Extended/Multiple Sources, Scene Radiance ...
(s,t)(u,v)
L(s,t,u,v,
Four-Dimensional Ray Manifold
![Page 24: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/24.jpg)
Methods for Sensing the Illumination Field
photodetectorsoptical fibers
hemispherical camera
??
![Page 25: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/25.jpg)
probe video
Compact Hemispherical Illumination Probe
compact wide angle optics
color video camera
neutral density filters
![Page 26: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/26.jpg)
LSD Prototype
Sony 15” LCDFlat Display
HemisphericalProbe Camera
Matting
Wooden Frame
![Page 27: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/27.jpg)
Content Modification : Rendering
• Power Efficiency
• Brighter in Sunlight
• Dimmer Indoors
• Compensation
• Spatially Varying Brightness
• Spatially Varying Color
• Photorealism
• Consistent Colors and Shadings
• Consistent Highlights and Shadows
All Modifications in Real-Time
![Page 28: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/28.jpg)
Rendering Using Explicit Models: 2D+
v
s1
n
s2
O
viewer
source
source
display
rendered image
content: surface
Algorithms: Ray Tracing, Radiosity
![Page 29: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/29.jpg)
Rendering Using Explicit Models: 3D
v
s1
n
s2
O
viewer
source
source
display
rendered image
content:shape, BRDF
Algorithms: Ray Tracing, Radiosity
![Page 30: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/30.jpg)
Image based Rendering
probe cameracapture camera
Off-line Scene Capture
(with Kudelka and Swaminathan)
![Page 31: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/31.jpg)
Efficient Representation and Rendering
Image Bases E
source directions
40
30
1616
Captured Images
i=1 i=4096
I
40
30
16
16
k=1 k=10
bloc
ks x
bas
is
3
0 x
40 x
10
Lighting Coefficient Vectors L
source directionsi=1 i=4096
SVD
![Page 32: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/32.jpg)
Efficient Representation and Rendering
Compressed Coefficient Vectors
Coeff. Bases U
source directions
Coefficient Vectors L
i=1 i=4096
V
q=1 b=200
bloc
ks x
bas
is
3
0 x
40 x
10
SVD
source directionsi=1 i=4096
![Page 33: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/33.jpg)
Real-Time Rendering
Compressed Coefficient Vector
Coeff Eigenvectors U
Illumination Field
Vs
Coefficient Vector
Image Eigenvectors E
Display
I
Compressed Coefficient Vectors V
s
X X X
UVs
![Page 34: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/34.jpg)
Efficient Representation and RenderingCaptured Data
ComputeLocal Subspaces
Local Bases and Coefficients
source direction
Image Reconstruction
Display Illumination Field
Display
4 Gb
10 Mb 8 fps (laptop)
![Page 35: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/35.jpg)
Efficient Representation and RenderingCaptured Data
ComputeLocal Subspaces
Local Bases and Coefficients
source direction
Image Reconstruction
Display Illumination Field
Display
4 Gb
10 Mb 8 fps (laptop)
![Page 36: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/36.jpg)
Face
![Page 37: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/37.jpg)
Still Life: Scene Capture
![Page 38: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/38.jpg)
Still Life
![Page 39: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/39.jpg)
![Page 40: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/40.jpg)
![Page 41: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/41.jpg)
Summary
• Lighting Sensitive Display:
• Senses Environmental Illumination
• Modifies Displayed Content
• Applications:
• Compensation: Computers, PDA’s, Televisions, Billboards
• Photorealism: Digital Art, E-Commerce, Future Homes
![Page 42: Image-based Rendering of Real Objects with Complex BRDFs](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d575503460f94a35ccc/html5/thumbnails/42.jpg)
Capturing Scenes for Image based Rendering
probe cameracapture camera