11/16/10
DESCRIPTION
Image-based Lighting. 11/16/10. T2. Computational Photography Derek Hoiem, University of Illinois. Many slides from Debevec, some from Efros. Next two classes. Today Details on final projects Go over some exam questions Start on ray tracing, environment maps, and relighting 3D objects - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/1.jpg)
11/16/10
Image-based Lighting
Computational PhotographyDerek Hoiem, University of Illinois
Many slides from Debevec, some from Efros
T2
![Page 2: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/2.jpg)
Next two classes
Today• Details on final projects• Go over some exam questions• Start on ray tracing, environment maps, and
relighting 3D objects
Thursday• Finish relighting• Start on hardware side of computational
photography: plenoptic camera, coded aperture
![Page 3: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/3.jpg)
Final Projects• To do (11/24 or sooner): Send me an e-mail with
1. a one-paragraph summary of your project idea 2. when you’d like to meet with me
- either this Thursday afternoon except 2:30-3:30, this Friday, or the week after break
• Ideas – Implement one or more related papers: texture synthesis, image analogies,
single-view 3D reconstruction, inserting 3D models, hole-filling, etc., etc.– Extend one of the projects, bringing in ideas from a couple related papers– Do something that seems fun/interesting to you or that builds on
interests/research in other areas
• What is the scope?– 3 credit version: it’s sufficient to get something working that has been done
before– 4 credit version: try something new, or try experimenting with variations on an
idea, integrate some ideas into your own research, or provide a thorough experimentation, perhaps comparing two methods
![Page 4: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/4.jpg)
Final Projects• Project page write-up due on Dec 13 (11:59pm)
– Similar to other projects, but with some more detail on how to do it – Can be in html or pdf– Store at url below or include a link from there
http://<username>.projects.cs.illinois.edu/cs498dwh/final/
• Class presentations on Dec 14 (1:30-4:30pm)– ~7 minutes per presentation
• Project Grade– 80%: implementation, results– 20%: presentation
![Page 5: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/5.jpg)
How to render an object inserted into an image?
What’s wrong with the teapot?
![Page 6: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/6.jpg)
How to render an object inserted into an image?
Traditional graphics way• Manually model BRDFs of all room surfaces• Manually model radiance of lights• Do ray tracing to relight object, shadows, etc.
![Page 7: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/7.jpg)
How to render an object inserted into an image?
Image-based lighting• Capture incoming light with a
“light probe”• Model local scene• Ray trace, but replace distant
scene with info from light probe
Debevec SIGGRAPH 1998
![Page 8: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/8.jpg)
Key ideas for Image-based Lighting• Environment maps: tell what light is entering
at each angle within some shell
+
![Page 9: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/9.jpg)
Key ideas for Image-based Lighting• Light probes: a way of capturing environment
maps in real scenes
![Page 10: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/10.jpg)
Key ideas for Image-based Lighting• Capturing HDR images: needed so that light
probes capture full range of radiance
![Page 11: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/11.jpg)
Key ideas for Image-based Lighting• Relighting: environment map acts as light
source, substituting for distant scene
![Page 12: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/12.jpg)
A photon’s life choices• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection
λ
light source
?
![Page 13: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/13.jpg)
A photon’s life choices• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection
λ
light source
![Page 14: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/14.jpg)
A photon’s life choices• Absorption• Diffuse Reflection• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection
λ
light source
![Page 15: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/15.jpg)
A photon’s life choices• Absorption• Diffusion• Specular Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection
λ
light source
![Page 16: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/16.jpg)
A photon’s life choices• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection
λ
light source
![Page 17: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/17.jpg)
A photon’s life choices• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection
λ
light source
![Page 18: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/18.jpg)
A photon’s life choices• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection
λ1
light source
λ2
![Page 19: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/19.jpg)
A photon’s life choices• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection
λ
light source
![Page 20: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/20.jpg)
A photon’s life choices• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection
t=1
light source
t=n
![Page 21: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/21.jpg)
A photon’s life choices• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection
λ
light source
(Specular Interreflection)
![Page 22: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/22.jpg)
Where are the light sources are in this room?
http://www.flickr.com/photos/chrisdonbavand/493707413/sizes/z/in/photostream/
![Page 23: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/23.jpg)
Rendering Equation
Incoming LightBRDF Incident angle
Generated lightTotal reflected light
Outgoing light
![Page 24: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/24.jpg)
Rendering a scene with ray tracing
http://en.wikipedia.org/wiki/File:Glasses_800_edit.png
![Page 25: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/25.jpg)
Ray tracing: basics
Illustration from http://www.groovyvis.com/other/raytracing/basic.html
camera center
image plane
λ
light source
![Page 26: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/26.jpg)
Ray casting• Store colors of surfaces and see what you hit
Wolfenstein 3D (1992)
![Page 27: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/27.jpg)
Ray tracing: fast approximation• Cast diffuse ray: go towards light and see if an object
is in the way• Cast reflection ray: see what reflected object is
http://en.wikipedia.org/wiki/File:Ray_trace_diagram.svg
![Page 28: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/28.jpg)
Ray tracing: interreflections• Reflect light N times before heading to light
source
N=16
N=2
![Page 29: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/29.jpg)
Ray tracing• Conceptually simple but hard to do fast
• Full solution requires tracing millions of rays for many inter-reflections
• Design choices– Ray paths: Light to camera vs. camera to light?– How many samples per pixel (avoid aliasing)?– How to sample diffuse reflections?– How many inter-reflections to allow?– Deal with subsurface scattering, etc?
![Page 30: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/30.jpg)
Environment Maps
• The environment map may take various forms:– Cubic mapping– Spherical mapping– other
• Describes the shape of the surface on which the map “resides”
• Determines how the map is generated and how it is indexed
![Page 31: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/31.jpg)
Cubic Map Example
![Page 32: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/32.jpg)
Cubic Mapping
• The map resides on the surfaces of a cube around the object– Typically, align the faces of the cube with the coordinate
axes
• To generate the map:– For each face of the cube, render the world from the
center of the object with the cube face as the image plane• Rendering can be arbitrarily complex (it’s off-line)
• To use the map:– Index the R ray into the correct cube face– Compute texture coordinates
![Page 33: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/33.jpg)
Spherical Map Example
![Page 34: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/34.jpg)
Sphere Mapping
• Map lives on a sphere• To generate the map:
– Render a spherical panorama from the designed center point
• To use the map:– Use the orientation of the R ray to index directly into the
sphere
![Page 35: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/35.jpg)
Using the Environment Map
camera center
image plane
![Page 36: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/36.jpg)
What approximations are made?
• The map should contain a view of the world with the point of interest on the object as the Center of Projection– We can’t store a separate map for each point, so one map
is used with the COP at the center of the object– Introduces distortions in the reflection, but we usually
don’t notice– Distortions are minimized for a small object in a large
room
• The object will not reflect itself!
![Page 37: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/37.jpg)
What about real scenes?
From Flight of the Navigator
![Page 38: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/38.jpg)
What about real scenes?
from Terminator 2
![Page 39: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/39.jpg)
Real environment maps
• We can use photographs to capture environment maps– The first use of panoramic mosaics
• Several ways to acquire environment maps:– Mirrored balls (light probes)– Stitching mosaics– Fisheye lens
![Page 40: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/40.jpg)
Mirrored SphereMirrored Sphere
![Page 41: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/41.jpg)
![Page 42: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/42.jpg)
![Page 43: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/43.jpg)
Sources of Mirrored BallsSources of Mirrored Balls
2-inch chrome balls ~ $20 ea. McMaster-Carr Supply Company
www.mcmaster.com
6-12 inch large gazing balls Baker’s Lawn Ornaments
www.bakerslawnorn.com
Hollow Spheres, 2in – 4in Dube Juggling Equipment
www.dube.com
FAQ on www.debevec.org/HDRShop/
2-inch chrome balls ~ $20 ea. McMaster-Carr Supply Company
www.mcmaster.com
6-12 inch large gazing balls Baker’s Lawn Ornaments
www.bakerslawnorn.com
Hollow Spheres, 2in – 4in Dube Juggling Equipment
www.dube.com
FAQ on www.debevec.org/HDRShop/
![Page 44: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/44.jpg)
0.34 0.34
0.580.58
=> 59% Reflective=> 59% Reflective
Calibrating Mirrored Sphere Reflectivity
Calibrating Mirrored Sphere Reflectivity
![Page 45: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/45.jpg)
One small snag• How do we deal with light sources? Sun, lights, etc?
– They are much, much brighter than the rest of the environment
• Use High Dynamic Range photography!
1
46
1907
15116
18
.
..
.
.
![Page 46: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/46.jpg)
Problem: Dynamic RangeProblem: Dynamic Range
![Page 47: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/47.jpg)
Problem: Dynamic RangeProblem: Dynamic Range
15001500
11
25,00025,000
400,000400,000
2,000,000,0002,000,000,000
The real world ishigh dynamic
range.
The real world ishigh dynamic
range.
![Page 48: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/48.jpg)
Long ExposureLong Exposure
10-6 106
10-6 106
Real world
Picture
0 to 255
High dynamic range
![Page 49: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/49.jpg)
Short ExposureShort Exposure
10-6 106
10-6 106
Real world
Picture
High dynamic range
0 to 255
![Page 50: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/50.jpg)
Camera CalibrationCamera Calibration
• Geometric– How pixel coordinates relate to directions in the
world
• Photometric– How pixel values relate to radiance amounts in
the world
• Geometric– How pixel coordinates relate to directions in the
world
• Photometric– How pixel values relate to radiance amounts in
the world
![Page 51: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/51.jpg)
The Image Acquisition PipelineThe Image Acquisition Pipeline
sceneradiance
(W/sr/m )
sceneradiance
(W/sr/m )
òsensorirradiance
sensorirradiance
sensorexposuresensor
exposurelatentimagelatentimage
LensLens ShutterShutter FilmFilm
Electronic CameraElectronic Camera
22
Dt
filmdensity
filmdensity
analogvoltagesanalog
voltagesdigitalvaluesdigitalvalues
pixelvaluespixel
values
DevelopmentDevelopment CCDCCD ADCADC RemappingRemapping
![Page 52: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/52.jpg)
log Exposure = log (Radiance * Dt)
Imaging system response function
Pixelvalue
0
255
(CCD photon count)
![Page 53: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/53.jpg)
Varying ExposureVarying Exposure
![Page 54: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/54.jpg)
Camera is not a photometer!Camera is not a photometer!
• Limited dynamic rangeÞ Perhaps use multiple exposures?
• Unknown, nonlinear response Þ Not possible to convert pixel values to
radiance
• Solution:– Recover response curve from multiple
exposures, then reconstruct the radiance map
• Limited dynamic rangeÞ Perhaps use multiple exposures?
• Unknown, nonlinear response Þ Not possible to convert pixel values to
radiance
• Solution:– Recover response curve from multiple
exposures, then reconstruct the radiance map
![Page 55: 11/16/10](https://reader035.vdocuments.site/reader035/viewer/2022070412/56814ab1550346895db7c38a/html5/thumbnails/55.jpg)
To be continued…