1 practical scene illuminant estimation via flash/no-flash pairs cheng lu and mark s. drew simon...
Post on 18-Dec-2015
218 views
TRANSCRIPT
1
Practical Scene Practical Scene Illuminant Estimation via Illuminant Estimation via
Flash/No-Flash PairsFlash/No-Flash Pairs
Cheng Lu and Mark S. Drew
Simon Fraser University{clu, mark}@cs.sfu.ca
2
Flash/No-flash Imagery – a Brief History
diCarlo, Xiao, & Wandell, CIC 2001Combine flash/no-flash images to produce a pure-flash image.Use dim=3 FDM + knowledge of flash SPD and sensor curves to estimate surface reflectance most likely ambient illuminant
Raskar et al., Non-Realistic Rendering 2004
Filling in night-time imagery with daytime image info.
Copy edges from cloned image region into edge-map of target background; re-integrate.
Blake et al., Poisson Image Editing, Siggraph 2004
Szeliski et al., Siggraph 2004
Transfer lower-noise information from flash image to higher-noise ambient-light image.
Find shadow-mask, copy edges inside shadow from flash image into ambient image, re-integrate.
Drew,Lu,Finlayson, Removing Shadows using Flash/Noflash Image Edges , ICME 2006
3
This paper:
Estimate Ambient Illuminant, usingFlash/No-flash Pairs
Like diCarlo&Wandell approach, but replace knowledge of camera sensor curves with a camera RGB-based calibration using difference of with-flash and no-flash images. How?
- Spectral sharpening- Subtract “both” – “no-flash” pure-flash image- Log’s- Project difference of flash minus ambient into geometric-mean chromaticity color space
Calibrate such to get illuminant chromaticity.
4
What’s the point?:
Can estimate scene (ambient) illuminant without knowing:
- Flash SPD- Camera sensors- Surface reflectance
5
Why estimate the illuminant?
White balance, plus many computer vision applications == intrinsic imageswithout illumination.
- Simple- Fast
What’s good about this method?
6
The set-up:
2 images , one under ambient lighting, & another under flash.
Under Ambient: Image “A”. Under Both: Image “B”.
+
7
The Key: Pure-Flash Image The ambient light from “A” is also in “B”. Therefore if we subtract the two, we have “F”: the pure-flash image.
Under Flash: Image “F”:
+ - =)(
8
Incidentally, note that there are now extra shadows, from the flash(since it’s offset from the lens).
Image “F”: the scene as imaged under Flash light only.
9
1. Lambertian surface:
RGB =
Shading = normal effective light-direction
Illum
inant
Surfa
ceSen
sors
Simple Image Formation Model
will guide us.
Assumptions: 1., 2., 3.
10
)(kQ
2. Narrow-band sensors:
so then
is exactly a single-spike sensor:
11
3. Planckian light:
But, can violate 1., 2., 3. and still succeed.
(in Wien’s approximation)
Gives
12
-Now take Log’s, to pull apart multiplications:
Camera-dep’t vector
Camera-dep’t vector
Intensity and shading
Surface
Color-temperature of light
where
13
Camera-dep’t vector
Surface
Color-temperature of light
So form geometric-mean chromaticity:
-We’d like to remove intensity/shading term:
In logs:
where
14
-The point:
As temp (light color) changes, move along straight line.
-But, we have “A” and “F” images: Subract them, and use same chromaticity trick Only illumination is left!
15
Log-difference Geometric-Mean Chromaticity
So log-log delivers inverse-temperature difference:
-Calibrate for 1/TA-1/TF, then in new scene obtain TA!
{
16
What does this
look like?
Moved to 2D; color-matching functions in geo-mean chromaticity. (9 Planckians, Macbeth ColorChecker, spike sensors, xenon flash SPD)
17
Sony DXC930 sensors, Daylights+F2, actual xenon flash SPD:
“Reference locus”
How to proceed: -Sharpen- Find closest cluster
18
Effect of sharpening:
Poor clusters Better clusters#’ing
Kodak DCS420:
19
Test: can we determine the illuminant?
102 illuminants, Sony camera, Munsell patches
102 illuminants, Sony camera, Macbeth patches
Estimate illum.from Munsell to Macbeth
Nearly 100% correctly identified.
20
Application: White Balance
4 calibration illuminants, HP camera, Macbeth chart(each cluster has 24 dots)
No flash
With flash
- Sharpen- Sample image at 24 locationsevenly over image-Same (“daylight”) color balancefor training and for testing
Image under CWF; CWF+Xenon
21
Overlaps best with CWF, so usewhite patch of Macbeth under CWF for white balance:
“Auto” balance – Wrong.
“Fluor” balance – Correct.
Our color-balance– Much closer.
22
Thanks!To Natural Sciences and
Engineering Research Council of Canada