presentation for thesis defense by xinshuo weng
Post on 16-Jan-2017
85 Views
Preview:
TRANSCRIPT
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Photometric Stereo ReconstructionBased on Sparse Representation
Xinshuo Weng
Wuhan UniversitySchool of Electronic Information
Instructor: Lei Yu
xinshuo.weng@whu.edu.cn
May 17, 2016
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 1 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Overview
1 Motivation
2 Basic Photometric Stereo: Woodham’s Method
3 Modeling with Sparse Regularized Algorithms
4 Representation of Sparse Bayesian Learning Model
5 Extension with Piecewise Linear Model
6 Summary
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 2 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Overview
1 Motivation
2 Basic Photometric Stereo: Woodham’s Method
3 Modeling with Sparse Regularized Algorithms
4 Representation of Sparse Bayesian Learning Model
5 Extension with Piecewise Linear Model
6 Summary
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 3 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Motivation
Monocular vision versus binocular vision
Basic photometric stereo: only ideal Lambertian diffuse model
Considering non-diffuse corruption (e.g. shadows and specularities)SparsityExtending to non-Lambertian diffuse reflection (e.g. specular reflectionand mixed reflection)
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 4 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Motivation
Monocular vision versus binocular vision
Basic photometric stereo: only ideal Lambertian diffuse model
Considering non-diffuse corruption (e.g. shadows and specularities)SparsityExtending to non-Lambertian diffuse reflection (e.g. specular reflectionand mixed reflection)
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 4 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Motivation
Monocular vision versus binocular vision
Basic photometric stereo: only ideal Lambertian diffuse model
Considering non-diffuse corruption (e.g. shadows and specularities)SparsityExtending to non-Lambertian diffuse reflection (e.g. specular reflectionand mixed reflection)
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 4 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Motivation
Monocular vision versus binocular vision
Basic photometric stereo: only ideal Lambertian diffuse model
Considering non-diffuse corruption (e.g. shadows and specularities)SparsityExtending to non-Lambertian diffuse reflection (e.g. specular reflectionand mixed reflection)
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 4 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Motivation
Monocular vision versus binocular vision
Basic photometric stereo: only ideal Lambertian diffuse model
Considering non-diffuse corruption (e.g. shadows and specularities)SparsityExtending to non-Lambertian diffuse reflection (e.g. specular reflectionand mixed reflection)
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 4 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Overview
1 Motivation
2 Basic Photometric Stereo: Woodham’s Method
3 Modeling with Sparse Regularized Algorithms
4 Representation of Sparse Bayesian Learning Model
5 Extension with Piecewise Linear Model
6 Summary
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 5 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Basic Photometric Stereo: Woodham’s MethodOutline
Procedure:Recovery of the surface normalDepth estimation
Assumption:Relative position between camera and object is fixed across all imagesObject is illuminated by a varying light source at known directions
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 6 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Basic Photometric Stereo: Woodham’s MethodOutline
Procedure:Recovery of the surface normalDepth estimation
Assumption:Relative position between camera and object is fixed across all imagesObject is illuminated by a varying light source at known directions
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 6 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Basic Photometric Stereo: Woodham’s MethodRecovery of Surface Normal
Recovering surface normal while ignoring non-diffuse corruption
Lambertian diffuse model:
I = ρnT lwhere
n ∈ R3 is the surface normal at the pointI ∈ R1xm is the observed intensity at this point in m imagesl ∈ R3xm is the incoming lighting direction in m imagesρ ∈ R is the diffuse albedo
Degree of freedom: 4
Given 4 images, n can be recovered via solving linear system
Given more than 4 images(m > 4): least square (LS)Close-form solution according to normal equation:
nT = I · lT · (llT )−1ρ−1
Limitation: non-Lambertian diffuse reflection, non-diffuse corruption
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 7 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Basic Photometric Stereo: Woodham’s MethodRecovery of Surface Normal
Recovering surface normal while ignoring non-diffuse corruption
Lambertian diffuse model:
I = ρnT lwhere
n ∈ R3 is the surface normal at the pointI ∈ R1xm is the observed intensity at this point in m imagesl ∈ R3xm is the incoming lighting direction in m imagesρ ∈ R is the diffuse albedo
Degree of freedom: 4
Given 4 images, n can be recovered via solving linear system
Given more than 4 images(m > 4): least square (LS)Close-form solution according to normal equation:
nT = I · lT · (llT )−1ρ−1
Limitation: non-Lambertian diffuse reflection, non-diffuse corruption
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 7 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Basic Photometric Stereo: Woodham’s MethodRecovery of Surface Normal
Recovering surface normal while ignoring non-diffuse corruption
Lambertian diffuse model:
I = ρnT lwhere
n ∈ R3 is the surface normal at the pointI ∈ R1xm is the observed intensity at this point in m imagesl ∈ R3xm is the incoming lighting direction in m imagesρ ∈ R is the diffuse albedo
Degree of freedom: 4
Given 4 images, n can be recovered via solving linear system
Given more than 4 images(m > 4): least square (LS)Close-form solution according to normal equation:
nT = I · lT · (llT )−1ρ−1
Limitation: non-Lambertian diffuse reflection, non-diffuse corruption
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 7 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Basic Photometric Stereo: Woodham’s MethodRecovery of Surface Normal
Recovering surface normal while ignoring non-diffuse corruption
Lambertian diffuse model:
I = ρnT lwhere
n ∈ R3 is the surface normal at the pointI ∈ R1xm is the observed intensity at this point in m imagesl ∈ R3xm is the incoming lighting direction in m imagesρ ∈ R is the diffuse albedo
Degree of freedom: 4
Given 4 images, n can be recovered via solving linear system
Given more than 4 images(m > 4): least square (LS)Close-form solution according to normal equation:
nT = I · lT · (llT )−1ρ−1
Limitation: non-Lambertian diffuse reflection, non-diffuse corruption
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 7 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Basic Photometric Stereo: Woodham’s MethodDepth Estimation
Surface normal is approximately perpendicular to the vector formedwith adjacent pixel
Least square again
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 8 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Basic Photometric Stereo: Woodham’s MethodReconstruction Result
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 9 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Basic Photometric Stereo: Woodham’s MethodReconstruction Result
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 9 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Overview
1 Motivation
2 Basic Photometric Stereo: Woodham’s Method
3 Modeling with Sparse Regularized Algorithms
4 Representation of Sparse Bayesian Learning Model
5 Extension with Piecewise Linear Model
6 Summary
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 10 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Modeling with Sparse Regularized AlgorithmsRepresentation of Non-Diffuse Corruption
Motivation: introduce a parameter to interpret corruption
New reflectance model:
I = ρnT l+ e
where
e ∈ R1xm is the additive corruption at this point in m images
Degree of freedom: m + 4 > m forever, unconstrained problem
Property: exhibiting dominant diffuse reflection while non-diffuseeffects emerge primarily in limited areas =⇒ e is sparse
Reformulating as following
minx,e
∥e∥0 s.t. y = Ax+ e
With relaxation for compensate more modeling errors
minx,e
∥y −Ax− e∥22 + λ∥e∥0
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 11 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Modeling with Sparse Regularized AlgorithmsRepresentation of Non-Diffuse Corruption
Motivation: introduce a parameter to interpret corruption
New reflectance model:
I = ρnT l+ e
where
e ∈ R1xm is the additive corruption at this point in m images
Degree of freedom: m + 4 > m forever, unconstrained problem
Property: exhibiting dominant diffuse reflection while non-diffuseeffects emerge primarily in limited areas =⇒ e is sparse
Reformulating as following
minx,e
∥e∥0 s.t. y = Ax+ e
With relaxation for compensate more modeling errors
minx,e
∥y −Ax− e∥22 + λ∥e∥0
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 11 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Modeling with Sparse Regularized AlgorithmsRepresentation of Non-Diffuse Corruption
Motivation: introduce a parameter to interpret corruption
New reflectance model:
I = ρnT l+ e
where
e ∈ R1xm is the additive corruption at this point in m images
Degree of freedom: m + 4 > m forever, unconstrained problem
Property: exhibiting dominant diffuse reflection while non-diffuseeffects emerge primarily in limited areas =⇒ e is sparse
Reformulating as following
minx,e
∥e∥0 s.t. y = Ax+ e
With relaxation for compensate more modeling errors
minx,e
∥y −Ax− e∥22 + λ∥e∥0
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 11 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Modeling with Sparse Regularized AlgorithmsLasso Sparse Regression (L1) Model
Model restatementminx,e
∥y −Ax− e∥22 + λ∥e∥0where
λ is a nonnegative trade-off parameter∥ · ∥0 represents the ℓ0-norm
However, ℓ0-norm is discontinuous and non-convex
Replacing ℓ0-norm with ℓ1-norm
minx,e
∥y −Ax− e∥22 + λ∥e∥1ℓ1-norm is convex, easy to solve
Sequentially added sign constraints algorithmNon-negative variables methodIterated ridge regression (IRR), hybrid of lasso and ridge regression
Limitation: approximation from ℓ0-norm to ℓ1-norm, non-Lambertiandiffuse reflection
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 12 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Modeling with Sparse Regularized AlgorithmsLasso Sparse Regression (L1) Model
Model restatementminx,e
∥y −Ax− e∥22 + λ∥e∥0where
λ is a nonnegative trade-off parameter∥ · ∥0 represents the ℓ0-norm
However, ℓ0-norm is discontinuous and non-convex
Replacing ℓ0-norm with ℓ1-norm
minx,e
∥y −Ax− e∥22 + λ∥e∥1ℓ1-norm is convex, easy to solve
Sequentially added sign constraints algorithmNon-negative variables methodIterated ridge regression (IRR), hybrid of lasso and ridge regression
Limitation: approximation from ℓ0-norm to ℓ1-norm, non-Lambertiandiffuse reflection
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 12 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Modeling with Sparse Regularized AlgorithmsLasso Sparse Regression (L1) Model
Model restatementminx,e
∥y −Ax− e∥22 + λ∥e∥0where
λ is a nonnegative trade-off parameter∥ · ∥0 represents the ℓ0-norm
However, ℓ0-norm is discontinuous and non-convex
Replacing ℓ0-norm with ℓ1-norm
minx,e
∥y −Ax− e∥22 + λ∥e∥1ℓ1-norm is convex, easy to solve
Sequentially added sign constraints algorithmNon-negative variables methodIterated ridge regression (IRR), hybrid of lasso and ridge regression
Limitation: approximation from ℓ0-norm to ℓ1-norm, non-Lambertiandiffuse reflection
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 12 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Overview
1 Motivation
2 Basic Photometric Stereo: Woodham’s Method
3 Modeling with Sparse Regularized Algorithms
4 Representation of Sparse Bayesian Learning Model
5 Extension with Piecewise Linear Model
6 Summary
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 13 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Representation of Sparse Bayesian Learning (SBL) ModelBayesian Inference
Motivation: simple hierarchical Bayesian model to estimate x, e
Likelihood function
p(y|x, e) = N (y;Ax+ e, λI)
Conjugate priorp(x) = N (x; 0, σ2
xI)
p(e) = N (e; 0,Γ)Posterior
p(x, e|y) ∝ p(y|x, e)p(x)p(e)Marginal posterior
p(x|y) =∫
p(x, e|y)de = N (x;µ,Σ)
With mean and covariance calculated asµ = ΣAT (Γ+ λI)−1y
Σ = (AT (Γ+ λI)−1A+ σ−2x I)−1
Close-form solution, except Γ is unknown
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 14 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Representation of Sparse Bayesian Learning (SBL) ModelBayesian Inference
Motivation: simple hierarchical Bayesian model to estimate x, e
Likelihood function
p(y|x, e) = N (y;Ax+ e, λI)
Conjugate priorp(x) = N (x; 0, σ2
xI)
p(e) = N (e; 0,Γ)Posterior
p(x, e|y) ∝ p(y|x, e)p(x)p(e)Marginal posterior
p(x|y) =∫
p(x, e|y)de = N (x;µ,Σ)
With mean and covariance calculated asµ = ΣAT (Γ+ λI)−1y
Σ = (AT (Γ+ λI)−1A+ σ−2x I)−1
Close-form solution, except Γ is unknown
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 14 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Representation of Sparse Bayesian Learning (SBL) ModelBayesian Inference
Motivation: simple hierarchical Bayesian model to estimate x, e
Likelihood function
p(y|x, e) = N (y;Ax+ e, λI)
Conjugate priorp(x) = N (x; 0, σ2
xI)
p(e) = N (e; 0,Γ)Posterior
p(x, e|y) ∝ p(y|x, e)p(x)p(e)Marginal posterior
p(x|y) =∫
p(x, e|y)de = N (x;µ,Σ)
With mean and covariance calculated asµ = ΣAT (Γ+ λI)−1y
Σ = (AT (Γ+ λI)−1A+ σ−2x I)−1
Close-form solution, except Γ is unknown
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 14 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Representation of Sparse Bayesian Learning (SBL) ModelMajorization-Minimization Approach
Motivation: estimating Γ accurately based on data itself
Likelihood function of Γ:
Empirical Bayesian approach
L(Γ) ,∫
p(y|x, e)p(x)p(e)dedx = N (y; 0,Σy)
whereΣy = σ2
xAAT + Γ+ λI
Maximum likelihood estimation
Γ = argmaxL(Γ)
Equivalently, we should minimize
L(Γ) , −ln
∫p(y|x, e)p(x)p(e)dedx
= ln|Σy|+ yTΣ−1y y
However, L(Γ) is non-convex, is this computationally feasible?
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 15 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Representation of Sparse Bayesian Learning (SBL) ModelMajorization-Minimization Approach
Motivation: estimating Γ accurately based on data itself
Likelihood function of Γ:
Empirical Bayesian approach
L(Γ) ,∫
p(y|x, e)p(x)p(e)dedx = N (y; 0,Σy)
whereΣy = σ2
xAAT + Γ+ λI
Maximum likelihood estimation
Γ = argmaxL(Γ)
Equivalently, we should minimize
L(Γ) , −ln
∫p(y|x, e)p(x)p(e)dedx
= ln|Σy|+ yTΣ−1y y
However, L(Γ) is non-convex, is this computationally feasible?
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 15 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Representation of Sparse Bayesian Learning (SBL) ModelMajorization-Minimization Approach
Motivation: estimating Γ accurately based on data itself
Likelihood function of Γ:
Empirical Bayesian approach
L(Γ) ,∫
p(y|x, e)p(x)p(e)dedx = N (y; 0,Σy)
whereΣy = σ2
xAAT + Γ+ λI
Maximum likelihood estimation
Γ = argmaxL(Γ)
Equivalently, we should minimize
L(Γ) , −ln
∫p(y|x, e)p(x)p(e)dedx
= ln|Σy|+ yTΣ−1y y
However, L(Γ) is non-convex, is this computationally feasible?
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 15 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Representation of Sparse Bayesian Learning (SBL) ModelMajorization-Minimization Approach (cont.)
Problem restatement
minΓ
L(Γ) = minΓ
(ln|Σy|+ yTΣ−1y y)
Σy = σ2xAAT + Γ+ λI
Solution: constructing an upper bound and iterativelytightening it
For brevity, using results directly from thesis
Suppose Γ is fixed, partitioning problem into two parts
ln|Σy| ≤∑i
(uiγi
+ ln γi)− h∗(u)
yTΣ−1y y ≤
∑i
z2iγi
+ f(z)
Equalities are obtained if and only if
u = diag[(σ2xAAT + λI)−1 + Γ−1]−1
z = ΓΣ−1y y
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 16 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Representation of Sparse Bayesian Learning (SBL) ModelMajorization-Minimization Approach (cont.)
Problem restatement
minΓ
L(Γ) = minΓ
(ln|Σy|+ yTΣ−1y y)
Σy = σ2xAAT + Γ+ λI
Solution: constructing an upper bound and iterativelytightening it
For brevity, using results directly from thesis
Suppose Γ is fixed, partitioning problem into two parts
ln|Σy| ≤∑i
(uiγi
+ ln γi)− h∗(u)
yTΣ−1y y ≤
∑i
z2iγi
+ f(z)
Equalities are obtained if and only if
u = diag[(σ2xAAT + λI)−1 + Γ−1]−1
z = ΓΣ−1y y
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 16 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Representation of Sparse Bayesian Learning (SBL) ModelMajorization-Minimization Approach (cont.)
Problem restatement
minΓ
L(Γ) = minΓ
(ln|Σy|+ yTΣ−1y y)
Σy = σ2xAAT + Γ+ λI
Solution: constructing an upper bound and iterativelytightening it
For brevity, using results directly from thesis
Suppose u, z are fixed
L(Γ) ≤∑i
(ui + z2i
γi+ ln γi) + Constant
Equality is obtained if and only if
γi = ui + z2i
Γ = diag[γ]
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 17 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Representation of Sparse Bayesian Learning (SBL) ModelMajorization-Minimization Approach (cont.)
Problem restatementminΓ
L(Γ) = minΓ
(ln|Σy|+ yTΣ−1y y)
Σy = σ2xAAT + Γ+ λI
Solution: constructing an upper bound and iterativelytightening it
For brevity, using results directly from thesis
Majorization-minimization approach
1. initialize Γ0
2. while ∥Γi − Γi−1∥ > ϵ do
3. update u = diag[(σ2xAAT + λI)−1 + Γ−1]−1
4. update z = ΓΣ−1y y
5. update γi = ui + z2i6. end while
Limitation: non-Lambertian diffuse reflection
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 18 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Representation of Sparse Bayesian Learning (SBL) ModelMajorization-Minimization Approach (cont.)
Problem restatementminΓ
L(Γ) = minΓ
(ln|Σy|+ yTΣ−1y y)
Σy = σ2xAAT + Γ+ λI
Solution: constructing an upper bound and iterativelytightening it
For brevity, using results directly from thesis
Majorization-minimization approach
1. initialize Γ0
2. while ∥Γi − Γi−1∥ > ϵ do
3. update u = diag[(σ2xAAT + λI)−1 + Γ−1]−1
4. update z = ΓΣ−1y y
5. update γi = ui + z2i6. end while
Limitation: non-Lambertian diffuse reflection
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 18 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Representation of Sparse Bayesian Learning (SBL) ModelExperimental Result and Analysis
Recovery of surface normal and error mapNoise is 10 percentShadows and specularities are rendered (same below)
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 19 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Overview
1 Motivation
2 Basic Photometric Stereo: Woodham’s Method
3 Modeling with Sparse Regularized Algorithms
4 Representation of Sparse Bayesian Learning Model
5 Extension with Piecewise Linear Model
6 Summary
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 20 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Extension with Piecewise Linear ModelReexamination of Lambertian Diffuse Model
Model RestatementI = ρnT l+ e
minx,e
∥e∥0 s.t. y = Ax+ e
p(y|x, e) = N (y;Ax+ e, λI)
All above are based on Lambertian diffuse modelWhat if more mixed and complicated materials?
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 21 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Extension with Piecewise Linear ModelReexamination of Lambertian Diffuse Model
Model RestatementI = ρnT l+ e
minx,e
∥e∥0 s.t. y = Ax+ e
p(y|x, e) = N (y;Ax+ e, λI)
All above are based on Lambertian diffuse modelWhat if more mixed and complicated materials?
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 21 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Extension with Piecewise Linear ModelLinear Combination Model
Extension Lambertian model to general function
I = f(nT l)
Monotonicity
nT li > nT lj ↔ f(nT li) > f(nT lj)
Inverse diffuse reflectance model
f−1(I) = g(I) = nT l
Given the linearity on right-hand-side, representing g(·) as
g(I) =
p∑k=1
akgk(I)
a is an unknown weight vector, p is set as the number of segments
gk(I) is non-linear basis function, multiple choices for gk(I):
polynomial, Gaussian, logistic, piecewise linear and spline function
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 22 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Extension with Piecewise Linear ModelLinear Combination Model
Extension Lambertian model to general function
I = f(nT l)
Monotonicity
nT li > nT lj ↔ f(nT li) > f(nT lj)
Inverse diffuse reflectance model
f−1(I) = g(I) = nT l
Given the linearity on right-hand-side, representing g(·) as
g(I) =
p∑k=1
akgk(I)
a is an unknown weight vector, p is set as the number of segments
gk(I) is non-linear basis function, multiple choices for gk(I):
polynomial, Gaussian, logistic, piecewise linear and spline function
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 22 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Extension with Piecewise Linear ModelPiecewise Linear Function
For modest computational cost, representing gk(I) with piecewiselinear function
gk(Ij) =
0 0 ≤ Ij < bk−1
Ij − bk−1 bk−1 ≤ Ij < bk
bk − bk−1 bk ≤ Ij
bk are segment points, equally separate the range of I and b0 = 0
g(I) is continuous and monotonically increasing, intersect the origin
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 23 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Extension with Piecewise Linear ModelPiecewise Linear Model
With linear combination model and piecewise linear function
nT l−p∑
k=1
akgk(I) = 0
For avoiding the degenerate x = 0 solution by constrainingp∑
k=1
ak = 1
Representing above as
A∗x = y∗
x is an unknown vector, x , [nx, ny, nz, a1, a2, . . . , ap]T ∈ Rp+3
Piecewise linear least square (PL-LS) model, piecewise linearsparse Bayesian learning (PL-SBL) model
Results closely match ground truth!
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 24 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Extension with Piecewise Linear ModelPiecewise Linear Model
With linear combination model and piecewise linear function
nT l−p∑
k=1
akgk(I) = 0
For avoiding the degenerate x = 0 solution by constrainingp∑
k=1
ak = 1
Representing above as
A∗x = y∗
x is an unknown vector, x , [nx, ny, nz, a1, a2, . . . , ap]T ∈ Rp+3
Piecewise linear least square (PL-LS) model, piecewise linearsparse Bayesian learning (PL-SBL) model
Results closely match ground truth!
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 24 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Extension with Piecewise Linear ModelExperimental Results and Analysis
Recovery of surface normal and error map
Noise is 10 percent
Best p is set for PL-LS (p = 2) and PL-SBL (p = 4)
Best σ2a is set for PL-SBL
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 25 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Extension with Piecewise Linear ModelExperimental Results and Analysis (cont.)
Mean error of recovery of surface normal with varying number ofimages for each algorithmNoise is 50 percent
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 26 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Extension with Piecewise Linear ModelExperimental Results and Analysis (cont.)
Mean error of recovery of surface normal with varying amount ofadditive Gaussian noises for each algorithmNumber of images is fixed to 40
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 27 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Extension with Piecewise Linear ModelParameter Selection
Mean error of recovery of surface normal with varying number ofimages for PL-LS algorithmNoise is 10 percent
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 28 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Extension with Piecewise Linear ModelParameter Selection (cont.)
Mean error of recovery of surface normal with varying σ2a and p for
PL-SBL algorithm
Number of images is fixed to 40
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 29 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Overview
1 Motivation
2 Basic Photometric Stereo: Woodham’s Method
3 Modeling with Sparse Regularized Algorithms
4 Representation of Sparse Bayesian Learning Model
5 Extension with Piecewise Linear Model
6 Summary
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 30 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Summary
We explored the basic method of photometric stereo reconstruction
We improved the performance by introducing sparse regularizedalgorithms as it could interpret the non-diffuse corruption
From a different perspective, we implemented a sparse Bayesianlearning model to represent the non-diffuse corruption and showstate-of-the-art performance
We extended the sparse Bayesian learning model with piecewise linearmodel, which could also interpret the non-Lambertian diffusereflectance and modestly improve the performance
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 31 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
References I
P. Woodham (1980)
Photometric Method for Determining Surface Orientation from Multiple ImagesOptical Eng., vol. 19, no. 1, pp. 139-144.
S. Barsky and M. Petrou (2003)
The 4-Source Photometric Stereo Technique for Three-Dimensional Surfaces in the Presence of Highlights and ShadowsIEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 10, pp. 1239-1252.
V. Argyriou and M. Petrou (2008)
Recursive Photometric Stereo When Multiple Shadows and Highlights Are PresentProc. IEEE Conf. Computer Vision and Pattern Recognition.
J. Ackermann, F. Langguth, S. Fuhrmann, and M. Goesele (2012)
Photometric Stereo for Outdoor WebcamsProc. IEEE Conf. Computer Vision and Pattern Recognition.
M. Chandraker, S. Agarwal, and D. Kriegman (2007)
Shadowcuts: Photometric Stereo with ShadowsProc. IEEE Conf. Computer Vision and Pattern Recognition.
S.P. Mallick, T.E. Zickler, D.J. Kriegman, and P.N. Belhumeur (2005)
Beyond Lambert: Reconstructing Specular Surfaces Using ColorProc. IEEE Conf. Computer Vision and Pattern Recognition.
K.E. Torrance and E.M. Sparrowr (1967)
Theory for Off-Specular Reflection from Roughened SurfacesJ. Optical Soc. Am., vol. 57, no. 9, pp. 1105-1112.
G. Ward (1992)
Measuring and Modeling Anisotropic ReflectionACM SIGGRAPH Computer Graphics, vol. 26, no. 2, pp. 265-272.
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 32 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
References II
A.S. Georghiades (2003)
Incorporating the Torrance and Sparrow Model of Reflectance in Uncalibrated Photometric StereoProc. IEEE Ninth Intl Conf. Computer Vision.
H. Chung and J. Jia (2008)
Efficient Photometric Stereo on Glossy Surfaces with Wide Specular LobesProc. IEEE Conf. Computer Vision and Pattern Recognition.
N. Alldrin, T. Zickler, and D. Kriegman (2008)
Photometric Stereo with Non-parametric and Spatially-Varying ReflectanceProc. IEEE Conf. Computer Vision and Pattern Recognition.
P. Tan, S.P. Mallick, L. Quan, D. Kriegman, and T. Zickler (2007)
Isotropy, Reciprocity and the Generalized Bas-Relief AmbiguityProc. IEEE Conf. Computer Vision and Pattern Recognition.
N.G. Alldrin and D.J. Kriegman (2007)
Toward Reconstructing Surfaces with Arbitrary Isotropic Reflectance: A Stratified Photometric Stereo ApproachProc. IEEE 11th Intl Conf. Computer Vision.
T. Higo, Y. Matsushita, and K. Ikeuchi (2010)
Consensus Photometric StereoProc. IEEE Conf. Computer Vision and Pattern Recognition.
P. Tan, L. Quan, and T. Zickler (2011)
The Geometry of Reflectance SymmetriesIEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 12, pp. 2506-2520.
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 33 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
References III
M. Chandraker and R. Ramamoorthi (2011)
What an Image Reveals about Material ReflectanceProc. IEEE Intl Conf. Computer Vision.
B. Shi, P. Tan, Y. Matsushita, and K. Ikeuchi (2012)
Elevation Angle from Reflectance Monotonicity: Photometric Stereo for General Isotropic ReflectancesProc. 12th European Conf. Computer Vision.
Y. Mukaigawa, Y. Ishii, and T. Shakunaga (2007)
Analysis of Photometric Factors Based on Photometric LinearizationJ. Optical Soc. Am, vol. 24, no. 10, pp. 3326-3334.
C. Yu, Y. Seo, and S.W. Lee (2010)
Photometric Stereo from Maximum Feasible Lambertian ReflectionsProc. 11th European Conf. Computer Vision.
S. Boyd and L. Vandenberghe (2004)
Convex OptimizationCambridge Univ. Press.
C.M. Bishop (2006)
Pattern Recognition and Machine LearningSpringer.
R. Tibshirani (1994)
Regression Shrinkage and Selection via the LassoJournal of the Royal Statistical Society, vol. 58, pp. 267-288, 1994.
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 34 / 35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
References IV
J. Fan and R. Li (2001)
Variable Selection via Non-Concave Penalized Likelihood and its Oracle PropertiesJournal of the American Statistical Association, Theory and Methods, vol. 96, no. 456.
M. Tipping (2001)
Sparse Bayesian Learning and the Relevance Vector MachineJ. Machine Learning Research, vol. 1, pp. 211-244.
K.P. Murphy (2007)
Conjugate Bayesian Analysis of the Gaussian Distribution
D. Wipf and S. Nagarajan (2010)
Iterative Reweighted ℓ1ℓ2 Methods for Finding Sparse SolutionsIEEE J. Selected Topics in Signal Processing, vol. 4, no. 2, pp. 317-329.
E.J. Candes and T. Tao (2005)
Decoding by Linear ProgrammingIEEE Trans. Information Theory, vol. 51, no. 12, pp. 4203-4215.
D.P. Wipf, B.D. Rao, and S. Nagarajan (2011)
Latent Variable Bayesian Models for Promoting SparsityIEEE Trans. Information Theory, vol. 57, no. 9.
S. Ikehata, D. Wipf (2014)
Photometric Stereo Using Sparse Bayesian Regression for General Diffuse SurfacesIEEE Trans. Pattern Analysis and Machine Learning, vol. 38, no. 9.
Xinshuo Weng (Wuhan University) Sparse Photometric Stereo May 17, 2016 35 / 35
top related