Download - Recap of Monday
![Page 1: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/1.jpg)
Recap of MondayPlenoptic function
Light fields
![Page 2: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/2.jpg)
High Dynamic Range ImagesHigh Dynamic Range Images
cs195g: Computational PhotographyJames Hays, Brown, Spring 2010
slides from Alexei A. Efrosand Paul Debevec
![Page 3: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/3.jpg)
Problem: Dynamic RangeProblem: Dynamic Range
15001500
11
25,00025,000
400,000400,000
2,000,000,0002,000,000,000
The real world ishigh dynamic
range.
The real world ishigh dynamic
range.
![Page 4: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/4.jpg)
Long ExposureLong Exposure
10-6 106
10-6 106
Real world
Picture
0 to 255
High dynamic range
![Page 5: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/5.jpg)
Short ExposureShort Exposure
10-6 106
10-6 106
Real world
Picture
High dynamic range
0 to 255
![Page 6: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/6.jpg)
pixel (312, 284) = 42pixel (312, 284) = 42
ImageImage
42 photons?42 photons?
![Page 7: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/7.jpg)
Camera CalibrationCamera Calibration
• Geometric– How pixel coordinates relate to directions in the
world
• Photometric– How pixel values relate to radiance amounts in
the world
• Geometric– How pixel coordinates relate to directions in the
world
• Photometric– How pixel values relate to radiance amounts in
the world
![Page 8: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/8.jpg)
Camera CalibrationCamera Calibration
• Geometric– How pixel coordinates relate to directions in the
world in other images.
• Photometric– How pixel values relate to radiance amounts in
the world in other images.
• Geometric– How pixel coordinates relate to directions in the
world in other images.
• Photometric– How pixel values relate to radiance amounts in
the world in other images.
![Page 9: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/9.jpg)
The ImageAcquisition Pipeline
The ImageAcquisition Pipeline
sceneradiance
(W/sr/m )
sceneradiance
(W/sr/m )
òsensorirradiance
sensorirradiance
sensorexposuresensor
exposure
LensLens ShutterShutter
22
Dt
analogvoltagesanalog
voltagesdigitalvaluesdigitalvalues
pixelvaluespixel
values
ADCADC RemappingRemappingCCDCCD
Raw Image
JPEGImage
![Page 10: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/10.jpg)
log Exposure = log (Radiance * Dt)
Imaging system response function
Pixelvalue
0
255
(CCD photon count)
![Page 11: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/11.jpg)
Varying ExposureVarying Exposure
![Page 12: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/12.jpg)
Camera is not a photometer!Camera is not a photometer!
• Limited dynamic rangeÞ Perhaps use multiple exposures?
• Unknown, nonlinear response Þ Not possible to convert pixel values to
radiance
• Solution:– Recover response curve from multiple
exposures, then reconstruct the radiance map
• Limited dynamic rangeÞ Perhaps use multiple exposures?
• Unknown, nonlinear response Þ Not possible to convert pixel values to
radiance
• Solution:– Recover response curve from multiple
exposures, then reconstruct the radiance map
![Page 13: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/13.jpg)
Recovering High Dynamic RangeRadiance Maps from PhotographsRecovering High Dynamic RangeRadiance Maps from Photographs
Paul DebevecJitendra MalikPaul DebevecJitendra Malik
August 1997August 1997
Computer Science DivisionUniversity of California at Berkeley
Computer Science DivisionUniversity of California at Berkeley
![Page 14: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/14.jpg)
Ways to vary exposureWays to vary exposure Shutter Speed (*)
F/stop (aperture, iris)
Neutral Density (ND) Filters
Shutter Speed (*)
F/stop (aperture, iris)
Neutral Density (ND) Filters
![Page 15: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/15.jpg)
Shutter SpeedShutter Speed
• Ranges: Canon D30: 30 to 1/4,000 sec.• Sony VX2000: ¼ to 1/10,000
sec.• Pros:• Directly varies the exposure• Usually accurate and repeatable• Issues:• Noise in long exposures
• Ranges: Canon D30: 30 to 1/4,000 sec.• Sony VX2000: ¼ to 1/10,000
sec.• Pros:• Directly varies the exposure• Usually accurate and repeatable• Issues:• Noise in long exposures
![Page 16: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/16.jpg)
Shutter SpeedShutter Speed
• Note: shutter times usually obey a power series – each “stop” is a factor of 2
• ¼, 1/8, 1/15, 1/30, 1/60, 1/125, 1/250, 1/500, 1/1000 sec
• Usually really is:
• ¼, 1/8, 1/16, 1/32, 1/64, 1/128, 1/256, 1/512, 1/1024 sec
• Note: shutter times usually obey a power series – each “stop” is a factor of 2
• ¼, 1/8, 1/15, 1/30, 1/60, 1/125, 1/250, 1/500, 1/1000 sec
• Usually really is:
• ¼, 1/8, 1/16, 1/32, 1/64, 1/128, 1/256, 1/512, 1/1024 sec
![Page 17: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/17.jpg)
• 3• 3
• 1• 1 •
2• 2
Dt =1 sec
• 3• 3
• 1• 1 •
2• 2
Dt =1/16 sec
• 3• 3
• 1• 1
• 2• 2
Dt =4 sec
• 3• 3
• 1• 1 •
2• 2
Dt =1/64 sec
The AlgorithmThe Algorithm
Image seriesImage series
• 3• 3
• 1• 1 •
2• 2
Dt =1/4 sec
Exposure = Radiance * DtExposure = Radiance * Dtlog Exposure = log Radiance + log Dtlog Exposure = log Radiance + log Dt
Pixel Value Z = f(Exposure)Pixel Value Z = f(Exposure)
![Page 18: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/18.jpg)
Response CurveResponse Curve
ln Exposureln Exposure
Assuming unit radiancefor each pixel
Assuming unit radiancefor each pixel
After adjusting radiances to obtain a smooth response
curve
After adjusting radiances to obtain a smooth response
curve
Pix
el v
alue
Pix
el v
alue
33
11
22
ln Exposureln Exposure
Pix
el v
alue
Pix
el v
alue
![Page 19: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/19.jpg)
The MathThe Math• Let g(z) be the discrete inverse response function• For each pixel site i in each image j, want:
• Solve the overdetermined linear system:
• Let g(z) be the discrete inverse response function• For each pixel site i in each image j, want:
• Solve the overdetermined linear system:
fitting term smoothness term
ln Radiance i lnt j g(Zij ) 2j1
P
i1
N
g (z)2
zZ min
Zmax
ln Radiance i ln t j g(Zij )
![Page 20: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/20.jpg)
MatlabCode
MatlabCode
function [g,lE]=gsolve(Z,B,l,w)
n = 256;A = zeros(size(Z,1)*size(Z,2)+n+1,n+size(Z,1));b = zeros(size(A,1),1);
k = 1; %% Include the data-fitting equationsfor i=1:size(Z,1) for j=1:size(Z,2) wij = w(Z(i,j)+1); A(k,Z(i,j)+1) = wij; A(k,n+i) = -wij; b(k,1) = wij * B(i,j); k=k+1; endend
A(k,129) = 1; %% Fix the curve by setting its middle value to 0k=k+1;
for i=1:n-2 %% Include the smoothness equations A(k,i)=l*w(i+1); A(k,i+1)=-2*l*w(i+1); A(k,i+2)=l*w(i+1); k=k+1;end
x = A\b; %% Solve the system using SVD
g = x(1:n);lE = x(n+1:size(x,1));
![Page 21: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/21.jpg)
Results: Digital CameraResults: Digital Camera
Recovered response curve
log Exposure
Pix
el v
alue
Kodak DCS4601/30 to 30 sec
![Page 22: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/22.jpg)
Reconstructed radiance map
![Page 23: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/23.jpg)
Results: Color FilmResults: Color Film• Kodak Gold ASA 100, PhotoCD• Kodak Gold ASA 100, PhotoCD
![Page 24: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/24.jpg)
Recovered Response CurvesRecovered Response Curves
Red Green
RGBBlue
![Page 25: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/25.jpg)
The Radiance
Map
The Radiance
Map
![Page 26: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/26.jpg)
How do we store this?How do we store this?
![Page 27: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/27.jpg)
Portable FloatMap (.pfm)Portable FloatMap (.pfm)• 12 bytes per pixel, 4 for each channel
sign exponent mantissa
PF768 5121<binary image data>
Floating Point TIFF similarFloating Point TIFF similar
Text header similar to Jeff Poskanzer’s .ppmimage format:
![Page 28: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/28.jpg)
(145, 215, 87, 149) =
(145, 215, 87) * 2^(149-128) =
(1190000, 1760000, 713000)
(145, 215, 87, 149) =
(145, 215, 87) * 2^(149-128) =
(1190000, 1760000, 713000)
Red Green Blue ExponentRed Green Blue Exponent
32 bits / pixel32 bits / pixel
(145, 215, 87, 103) =
(145, 215, 87) * 2^(103-128) =
(0.00000432, 0.00000641, 0.00000259)
(145, 215, 87, 103) =
(145, 215, 87) * 2^(103-128) =
(0.00000432, 0.00000641, 0.00000259)
Ward, Greg. "Real Pixels," in Graphics Gems IV, edited by James Arvo, Academic Press, 1994
Radiance Format(.pic, .hdr)
Radiance Format(.pic, .hdr)
![Page 29: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/29.jpg)
ILM’s OpenEXR (.exr)ILM’s OpenEXR (.exr)• 6 bytes per pixel, 2 for each channel, compressed
sign exponent mantissa
• Several lossless compression options, 2:1 typical• Compatible with the “half” datatype in NVidia's
Cg• Supported natively on GeForce FX and Quadro FX
• Available at http://www.openexr.net/
![Page 30: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/30.jpg)
Now What?Now
What?
![Page 31: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/31.jpg)
Tone MappingTone Mapping
10-6 106
10-6 106
Real WorldRay Traced World (Radiance)
Display/Printer
0 to 255
High dynamic range
• How can we do this?Linear scaling?, thresholding? Suggestions?
• How can we do this?Linear scaling?, thresholding? Suggestions?
![Page 32: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/32.jpg)
TheRadiance
Map
TheRadiance
Map
Linearly scaled todisplay device
![Page 33: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/33.jpg)
Simple Global OperatorSimple Global Operator
• Compression curve needs to
– Bring everything within range– Leave dark areas alone
• In other words
– Asymptote at 255– Derivative of 1 at 0
• Compression curve needs to
– Bring everything within range– Leave dark areas alone
• In other words
– Asymptote at 255– Derivative of 1 at 0
![Page 34: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/34.jpg)
Global Operator (Reinhart et al)Global Operator (Reinhart et al)
world
worlddisplay L
LL
1
![Page 35: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/35.jpg)
Global Operator ResultsGlobal Operator Results
![Page 36: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/36.jpg)
Darkest 0.1% scaledto display device
Reinhart Operator
![Page 37: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/37.jpg)
What do we see?What do we see?
Vs.
![Page 38: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/38.jpg)
Issues with multi-exposure HDRIssues with multi-exposure HDR
• Scene and camera need to be static• Camera sensors are getting better and better• Display devices are fairly limited anyway
(although getting better).
![Page 39: Recap of Monday](https://reader036.vdocuments.site/reader036/viewer/2022062323/56815b29550346895dc8eb22/html5/thumbnails/39.jpg)
What about local tone mapping?
What about avoiding radiance map reconstruction entirely?
What about local tone mapping?
What about avoiding radiance map reconstruction entirely?