an analysis of convex relaxations for map estimation
DESCRIPTION
An Analysis of Convex Relaxations for MAP Estimation. Aim: To analyze Maximum a Posteriori (MAP) estimation methods based on convex relaxations. Comparing Relaxations. Two New SOCP Relaxations. Domination. For all ( u , P ). For at least one ( u , P ). All LP-S constraints. SOCP-C. a. - PowerPoint PPT PresentationTRANSCRIPT
An Analysis of Convex Relaxations for MAP EstimationM. Pawan Kumar Vladimir Kolmogorov Philip H.S. TorrM. Pawan Kumar Vladimir Kolmogorov Philip H.S. Torr
http://cms.brookes.ac.uk/research/visiongroup http://www.adastral.ucl.ac.uk/~vladkolm
Aim: To analyze Maximum a Posteriori (MAP) estimation methods based on convex relaxations
MAP Estimation - Integer Programming Formulation
Va
Label Vector x [ -1, 1 ; 1, -1]
Unary Vector u [ 5, 2 ; 2, 4]
Random Field ExamplePairwise Matrix P - - 0 3- - 1 00 1 - -3 0 - -
MAP x*
Linear Programming (LP) Relaxation
#variables n = 2#labels h = 2
arg min xT (4u + 2P1) + P X, subject to ∑i xa;i = 2 - h
x {-1,1}nh
X = x xT
5
21 3
Comparing Relaxations
LP-S vs. SOCP Relaxations over Trees and Cycles
A B = ∑ Aij Bij
Schlesinger, 1976
Second Order Cone Programming (SOCP) RelaxationsKim and Kojima, 2000
Domination
2
4
Non-convexConstraint
ConvexRelaxation
x {-1,1}nh
x [-1,1]nh
X = x xT
j Xab;ij = (2-h) xa;i
LP-S
Non-convexConstraint
X = x xT ConvexRelaxation
||UTx||2≤ C X
C = U UT
…
1+xa;i+xb;j+Xab;ij ≥ 0
A B
≥
dominates A B
>strictly
dominates
For all (u,P) For at least one (u,P)
Equivalent relaxations: A dominates B, B dominates A.Vb
Va Vb
Vd Vc
a b
d c
a b
d c
Random Field C Matrix
2 1 1 01 2 1 11 1 2 10 1 1 2
V(Constrained Variables)
E(Constrained Pairs)
G = (V,E)
Tree (SOCP-T) Even Cycle (SOCP-E) Odd Cycle (SOCP-O)
•LP-S dominates SOCP-T •Pab;ij ≥ 0
SOCP-Q All cycle inequalities
a b
c d
a b
c Clique ‘G’
Dominates linear cycle inequalities
Future Work 50 random fields4 neighbourhood 8 neighbourhood
Two New SOCP Relaxations
Open Questions
SOCP-C All LP-S constraints
a b
c d
Dominated by linear cycle inequalities?
a b
c Cycle ‘G’
•Special Case: Edge
•SOCP-MS/QP-RL
•Pab;ij ≤ 0
•LP-S dominates SOCP-E
•Pab;ij ≥ 0 for one (a,b)
•Pab;ij ≤ 0 for one/all (a,b)
•LP-S dominates SOCP-O
Comparing Existing SOCP and QP Relaxations
SOCP-MS
xa;i xb;jXab;ij
(xa;i+ xb;j)2 ≤ 2 + 2Xab;ij
(xa;i- xb;j)2 ≤ 2 - 2Xab;ij
Muramatsu and Suzuki, 2003
• Pab;ij ≥ 0 Xab;ij = infimum
• Pab;ij < 0 Xab;ij = supremum
SOCP-MS is QP-RLRavikumar and Lafferty, 2006
min P X
• Cycle inequalities vs. SOCP/QP?
• Best ‘C’ for special cases?
• Efficient solutions for SOCP?
a b b c c a
Unary Cost = 0 LP-S = 0 SOCP-C = 0.75
= 0
= 1
a b b c c d= -1/3
= 1/3d a a c b d
xa;0 = -1/3 xa;1 = -1/3
0
0