lena gorelick joint work with o. veksler i. ben ayed a. delong y. boykov
TRANSCRIPT
Submodularization for Binary Pairwise Energies
Lena Gorelick
joint work with
O. Veksler
I. Ben Ayed
A. Delong
Y. Boykov
3
Binary Pairwise EnergyQuadratic Form
Submodular Energy global optimum with graphcut (Boros &
Hammer, 2002)
qp
qppqp
pp xxvxuE,
(x) 1,0x
pqpq vv 0
Potts Model
4
0, pqvpq
Non-Submodular Energy NP-hard
Binary Pairwise Energy Quadratic Form
const.)(,
qp
qppqp
pp xxvxuE x
Middlebury
Image credit: Carlos Hernandes
5
Standard Optimization Methods
General energy - NP-hardApproximate methods:
Global Linearization: QPBO, TRWS, SRMP (Kolmogorov et al. 2006, 2014)
Local Linearization: parallel ICM, IPFP (Leordeanu, 2009)
Message Passing: BP (Pearl 1989)
6
Related WorkGlobal Linearization
)(xE QPBO, TRWS, SRMP (Kolmogorov et al. 2006,
2014)
)(~
min..
yyE
Cts
Linearize introducing large number ofvariables and constraints
Solve relaxed LPor its dual
Integrality Gap
*relaxedyRounding
*integer x
Related WorkIterative Local Linearization
7
parallel ICM (Leordeanu, 2009) large steps weak min
IPFP (Leordeanu, 2009) controls step size by relaxation Integrality Gap
)(xE
x
tx
Et(x)~
1tx
N}1,0{Bounded domain of discrete configurations
8
Local Submodular Approximation LSA
Local Submodular Approximation model
Non-linear
Two ways to control step size
Et(x)~
)(xE
x
tx
1tx
N}1,0{Bounded domain of discrete configurations
9
Trust Region Local submodular approximation
Auxiliary Functions = Surrogate Functions = Upper Bounds = Majorize-Minimize Local submodular upper bound
Never leave the discrete domain
Iterative Optimization Framework
LSA-AUX
LSA-TR
10
Iterative Optimization Framework
Trust Region: Discrete High Order Energies Relaxed Quadratic Binary Energies Levenberg Marquardt
Auxiliary Functions=Surrogate Functions =Upper Bounds = Majorize-Minimize Discrete High Order Energies
Gorelick et al. 2012,2013
Ben Ayed et al. 2013
Olsson et al. 2008
Narasimhan & Bilmes 2005
Rother et al. 2006
Hartley & Zisserman 2004
13
Approximate around
Local Submodular ApproximationLSA
)()()( sup xxx EEE sub
)(xE tx
)(~
x(x) subt EE
Et(x)~
x
tx)(xE
14
Submodular functionLSA
Approximate around
Local Submodular ApproximationLSA
)()()( sup xxx EEE sub
)(xE tx
)()(~
xx(x) approxt
subt EEE
Linear Approximation
Et(x)~
x
tx)(xE
20
1
constyx 0
0
1
Linear Approximation of the Supermodular Term
x
y
1,0
0,0
1,1
Linear (Unary)
approximation
LSA-TRTrust Region Sub-Problem
24
)(xE
x
tx
Et(x)~
Trust Region
)()(~
xx(x) approxt
subt EEE
Trust Region Sub-Problem
td ||||s.t. txx
24
NP-hard!Constrained Submodular Optimization
25
fixed in each iteration inversely related to trust region size adjusted based on quality of approximation
LSA-TR: Approximate TR sub-problem
||||
)()(
tt
approxt
subt EEL
xx
xx(x)
Unary TermsBoykov et al. 2006
Gorelick et al. 2013
t
Submodular
27
Experiments & Results:Deconvolution
Binary De-convolution All pairwise terms supermodular
Original Img Convolved Convolved+Noise
?
Experiments & Results:Deconvolution
28
Noise:N(0,0.05)
SRMP
QPBOI
TRWS FTR-L
LBP
LSA-TR (0.3 sec.)E=21.13
LSA-AUX (0.04 sec)E=34.70
TRWS:5000 iter.E=65.07
LBP5000 iter.E=40.15
QPBO(0.1 sec.)
QPBO-I (0.2 sec.)E=66.44
IPFP(0.4 sec.)E=32.90
SRMP:5000 iter.E=39.06
Experiments & Results:Segmentation of Thin Structures
29
QPBO
QPBO-IE= -77.08
LBPE= -84.54
IPFPE= 163.25
Image
SRMP
Potts, v<0(submodula
r)
with edge repulsion, v>0(non-submodular)
TRWSE= -67.21
LSA-TRE= -175.05
LSA-AUXE= -120.03
SRMPE= -101.61
Repulsion = Reward different labels across high contrast edges
30
Experiments & Results:Inpainting
dtf-chinesechar database
LSA-TRInput ImgGround Truth
Kappes et al., 2013
32
Curvature Regularization
Efficient Squared Curvature model – (Nieuwenhuis et al. 2014, poster on Friday)
Potts Model Elastica
90-degree curvature
Heber et al. 2012
El-Zehiry&Grady, 2010
Our curvature Using LSA-TR
33
Summary
Two novel discrete optimization methods Simple, efficient, state-of-art results The code is available online -
http://vision.csd.uwo.ca/code/
Extensions: Find new applications▪ Convexity Shape Prior (in ECCV14)
Alternative optimization framework with LSA▪ Pseudo-Bounds (in ECCV14)
Please come by our poster