shadow removal algorithms shadow removal seminar pavel knur
Post on 19-Dec-2015
238 views
TRANSCRIPT
Shadow removal algorithms
Shadow removal seminarPavel Knur
Deriving intrinsic images from image sequences
Yair WeissJuly 2001.
History
• “intrinsic images” by Barrow and Tenenbaum , 1978
Constraints
• Fixed viewpoint• Works only for static objects• Cast shadows
Classic ill-posed problem
•Denote– the input image– the reflectance image– the illumination image
Number of Unknowns is twice the number of equations.
),( yxR),( yxI
),( yxL
),(),(),( yxRyxLyxI
The problem
Given a sequence of T imagesin which reflectance is constant over the time and only the illuminationchanges, can we solve for a singlereflectance image and T illumination images ?
Still completely ill-posed : at every pixel there are T equations and T+1 unknowns.
)},,({1
tyxIT
t
)},,({1
tyxLT
t
Maximum-likelihood estimation
• Log domain :
),(),(
),(),(
),(),(
log
log
log
yxlyxL
yxryxR
yxiyxI
),,(),(),,( tyxlyxrtyxi
Assumptions
When derivative filters are applied to natural images, the filter outputs tend to be sparse.
Laplacian distribution
Can be well fit by laplacian distribution
xZ exP 1)(
Claim 1
Denote :• N filters – • Filter outputs – • Filtered reflectance image –
ML estimation of filtered reflectance image
is given by
}{ nf
nn ftyxityxo ),,(),,(
nn frr
nr̂
),,(ˆ tyxomedianr ntn
Estimated reflectance function
Recover ML estimation of r
is reversed filter of
nn rrf ˆˆ
)ˆ(ˆ n
nrn rfgr
rnf nf
)(n
nrn ffg
ML estimation algorithm
ML estimation algorithm – cont.
• Ones we have estimated ),( yxr
),(),,(),,( yxrtyxityxl
Claim 2
•What if does not have exactly a Laplasian distribution ?
Let
Then estimated filtered reflectance are within with probability at least:
),,( tyxlfn
)),,(( tyxlfPp i
2/
1
)1(T
k
kkT ppk
T
Claim 2 - proof
If more than 50% of the samples ofare within of some value, then by definition of median, the median must be within of that value.
),,( tyxlfn
Example 1
• Einstein image is translated diagonally
• 4 pixels per frame
Example 2
• 64 images with variable lighting from Yale Face Database
Illumination Normalization with Time-Dependent Intrinsic Images for Video SurveillanceY.Matsushita,K.Nishito,K.IkeuchiOct. 2004
Illumination Normalization algorithm
• Preprocessing stage for robust video surveillance.
• Causes– Illumination conditions– Weather conditions– Large buildings and trees
• Goal– To “normalize” the input image
sequence in terms of incident lighting.
Constraints
• Fixed viewpoint• Works only for static objects• Cast shadows
Background images
• Remove moving objects from the input image sequence
Input images
Background images
Off-line
Estimation of Intrinsic Images
Denote• input image• time-varying reflectance image• time-varying illumination image• reflectance image estimated by ML• illumination image estimated by
ML
• Filters
• Log domain
Input images
Background images
Off-line
Estimation of Intrinsic Images
),,( tyxR
),,( tyxL
),( yxRw),,( tyxLw
),,( tyxI
),,(),,(),,( tyxRtyxLtyxI
1100 f
Tf 1101
ww lrlri ,,,,
Estimation of Intrinsic Images – cont.
• In Weiss’s original work
• The goal is to find estimation of and
Input images
Background images
Off-line
Estimation of Intrinsic Images
),,(),(ˆ tyxifmedianyxr ntwn
),(ˆ),,(),,(ˆ yxrtyxiftyxl wnnwn
ril
),,( tyxR ),,( tyxL
Estimation of Intrinsic Images – cont.
Basic idea:• Estimate time-varying reflectance
components by canceling the scene texture from initial illumination images
Define:
Input images
Background images
Off-line
Estimation of Intrinsic Images
otherwisetyxl
Tyxriftyxl
wn
wnn ),,,(
),(,0),,(
otherwiseyxr
Tyxriftyxlyxrtyxr
wn
wnwnwnn ),,(
),(),,,(),(),,(
),,(),,(),,(),(),,( tyxltyxrtyxlyxrtyxif nnwnwnn
Estimation of Intrinsic Images – cont.
Finally :
Where : is reversed filter of
Input images
Background images
Off-line
Estimation of Intrinsic Images
nn
rn
nn
rn
lfgtyxl
rfgtyxr
ˆ),,(ˆ
ˆ),,(ˆ
r
nf nf
)(n
nrn ffg
Shadow Removal
Denote - background image - illuminance-invariant image
Input images
Background images
Off-line
Estimation of Intrinsic Images
),,( tyxB
),,(),,(),,( tyxLtyxRtyxB
),,( tyxN
),,(/),,(),,( tyxLtyxBtyxN
Illumination Eigenspace
• PCA – Principle component analysisBasic components -
Input images
Background images
Off-line
Estimation of Intrinsic Images
nsss ,...,, 21
IlluminationEigenspace
Illumination Eigenspace – cont.
• Average is
• P is MxN matrix where– N – number of pixels in illumination
image– M – number of illumination images
• Covariance matrix Q of P is
Input images
Background images
Off-line
Estimation of Intrinsic Images
n ww L
nL
1
IlluminationEigenspace
wwwwww LLLLLLPn ,...,,
21
TPPQ
iii Qee
Direct Estimation of Illumination Images
• Pseudoillumination image
• Direct Estimation is
• Where– F is a projection function onto the j’s
eigenvector
-
Input images
Background images
Off-line
Estimation of Intrinsic Images
),(/),,(* yxRtyxIL w
IlluminationEigenspace
j
wjLw jLFjLFwLiiw
2* ),(),(minargˆ
i
jjw
Direct Estimation of Illumination Images
• Results
Input images
Background images
Off-line
Estimation of Intrinsic Images
IlluminationEigenspace
Shadow interpolation
probability density functioncumulative probability functionshadowed arealit area
mean
optimum threshold value
Input images
Background images
Off-line
Estimation of Intrinsic Images
IlluminationEigenspace
ShadowInterpolation
T
iis ipTP
min
)()(
max
)()(i
Til ipTP
T
iis iipT
min
)()(
max
)()(i
Til iipT
2)()()()(maxarg TTTPTPT lsls
T
opt
)(ip
Ps
l
optT
The whole algorithmInput images
Background images
Off-line
Estimation of Intrinsic Images
IlluminationEigenspace
/
IlluminationImages
Normalization
ShadowInterpolation
Example
Questions ?
References
[1] Y.Weiss,”Deriving Intrinsic Images from Image Sequences”, Proc. Ninth IEEE Int’l Conf. Computer Vision, pp. 68-75, July 2001.
[2] Y.Matsushita,K.Nishito,K.Ikeuchi,“Illumination Normalization with Time-Dependent Intrinsic Images for Video Surveillance”,Oct. 2004.