rounding-based moves for metric labeling m. pawan kumar center for visual computing ecole centrale...
TRANSCRIPT
Rounding-based Movesfor Metric Labeling
M. Pawan Kumar
Center for Visual Computing
Ecole Centrale Paris
PostMetric LabelingRandom variables V = {v1, v2, …, vn}
Label set L = {l1, l2, …, lh}
Labelings quantatively distinguished by energy E(y)
Labeling y L∈ n
Unary potential of variable va V∈
∑a θa(ya)
PostMetric LabelingRandom variables V = {v1, v2, …, vn}
Label set L = {l1, l2, …, lh}
Labelings quantatively distinguished by energy E(y)
Labeling y L∈ n
Pairwise potential of variables (va,vb)
∑a θa(ya) + ∑(a,b) wab d(ya,yb)
wab is non-negative d(.,.) is a metric distance function
miny
Post• Existing Work
– Move-Making Algorithms (Efficient)– Linear Programming Relaxation (Accurate)
• Rounding-based Moves– Equivalence– Complete Rounding and Complete Move– Interval Rounding and Interval Move– Hierarchical Rounding and Hierarchical Move
Outline
PostExpansion Algorithm
Sky
House
Tree
GroundInitialize with TreeExpand GroundExpand HouseExpand Sky
Variables take label lα or retain current label
Boykov, Veksler and Zabih, ICCV 1999
PostMove-Making AlgorithmsIteration t
Define St L⊆ n containing current labeling yt
∑a θa(ya) + ∑(a,b) wab d(ya,yb)argminy
s.t. y S∈ t
Sometimes it can even be solved exactly
Above problem is easier than original problem
yt+1 =
Start with an initial labeling y0
Post• Existing Work
– Move-Making Algorithms (Efficient)– Linear Programming Relaxation (Accurate)
• Rounding-based Moves– Equivalence– Complete Rounding and Complete Move– Interval Rounding and Interval Move– Hierarchical Rounding and Hierarchical Move
Outline
PostLinear Programming Relaxation
Chekuri, Khanna, Naor and Zosin, SODA 2001
Binary indicator xa(i) {0,1} ∈
If variable ‘a’ takes the label ‘i’ then xa(i) = 1
∑i xa(i) = 1Each variable ‘a’ takes one label
Similarly, binary indicator xab(i,k) {0,1} ∈
PostLinear Programming Relaxation
Minimize a linear function over feasible x
Indicators xa(i), xab(i,k) {0,1} Relaxed xa(i), xab(i,k) [0,1]
Rounding
Chekuri, Khanna, Naor and Zosin, SODA 2001
Post• Existing Work
– Move-Making Algorithms (Efficient)– Linear Programming Relaxation (Accurate)
• Rounding-based Moves– Equivalence– Complete Rounding and Complete Move– Interval Rounding and Interval Move– Hierarchical Rounding and Hierarchical Move
Outline
PostMove-Making Bound
y*: Optimal Labeling y: Estimated Labeling
Σa θa(ya) + Σ(a,b) wabd(ya,yb)
Σa θa(y*a) + Σ(a,b) wabd(y*a,y*b)
≥
PostMove-Making Bound
y*: Optimal Labeling y: Estimated Labeling
Σa θa(ya) + Σ(a,b) wabd(ya,yb)
Σa θa(y*a) + Σ(a,b) wabd(y*a,y*b)B
≤
For all possible values of θa(i) and wab
PostRounding Approximation
x*: LP Optimal Solution x: Rounded Solution
Σa Σi θa(i)xa(i) + Σ(a,b) Σ(i,k) wabd(i,k)xab(i,k)
≥
Σa Σi θa(i)x*a(i) + Σ(a,b) Σ(i,k) wabd(i,k)x*ab(i,k)
PostRounding Approximation
x*: LP Optimal Solution x: Rounded Solution
Σa Σi θa(i)xa(i) + Σ(a,b) Σ(i,k) wabd(i,k)xab(i,k)
≤
Σa Σi θa(i)x*a(i) + Σ(a,b) Σ(i,k) wabd(i,k)x*ab(i,k)A
For all possible values of θa(i) and wab
PostEquivalence
For any known rounding with approximation A
there exists a move-making algorithm
such that the move-making bound B = A
We know how to design such an algorithm
Post• Existing Work
– Move-Making Algorithms (Efficient)– Linear Programming Relaxation (Accurate)
• Rounding-based Moves– Equivalence– Complete Rounding and Complete Move– Interval Rounding and Interval Move– Hierarchical Rounding and Hierarchical Move
Outline
PostComplete Rounding
Treat x*a(i) [0,1] as probability that ya = li
Cumulative probability za(i) = Σj≤i x*a(j)
0 za(1) za(2) za(h) = 1za(k)za(i)
Generate a random number r (0,1]
Assign the label next to r
r
PostComplete Rounding - Example
0 za(1) za(4)za(3)za(2)
0.25 0.5 0.75 1.0
0 zb(1) zb(4)zb(3)zb(2)
0.7 0.8 0.9 1.0
0 zc(1) zc(4)zc(3)zc(2)
0.1 0.2 0.3 1.0
r
r
r
PostEquivalent Move
Complete Move !!
PostComplete MoveIteration t
Define St ⊆ Ln
∑a θa(ya) + ∑(a,b) wab d(ya,yb)argminy
s.t. y S∈ t
yt+1 =
Start with an initial labeling y0
PostComplete MoveIteration t
Define St = Ln
∑a θa(ya) + ∑(a,b) wab d(ya,yb)argminy
s.t. y S∈ t
How do we solve this problem?
Above problem is the same as the original problem
yt+1 =
Start with an initial labeling y0
PostComplete Move
Define St = Ln
∑a θa(ya) + ∑(a,b) wab d’(ya,yb)argminy
s.t. y S∈ t
How do we solve this problem?
Above problem is the same as the original problem
yt+1 =
PostComplete Move
Define St = Ln
∑a θa(ya) + ∑(a,b) wab d’(ya,yb)argminy
s.t. y S∈ t
Obtained by solving a small LP
Submodular overestimation d’ of d
yt+1 =
PostSubmodular Overestimation
maxi,k d’(li,lk)/d(li,lk)mind’
d’(li,lk) ≥ d(li,lk)s.t.
d’(li,lk+1) + d’(li+1,lk) ≥ d(li,lk) + d(li+1,lk+1)
PostSubmodular Overestimation
bmind’
d’(li,lk) ≥ d(li,lk)s.t.
d’(li,lk+1) + d’(li+1,lk) ≥ d(li,lk) + d(li+1,lk+1)
bd(li,lk) ≥ d’(li,lk)
Dual provides worst-case instance for complete rounding
Post• Existing Work
– Move-Making Algorithms (Efficient)– Linear Programming Relaxation (Accurate)
• Rounding-based Moves– Equivalence– Complete Rounding and Complete Move– Interval Rounding and Interval Move– Hierarchical Rounding and Hierarchical Move
Outline
PostInterval Rounding
Treat x*a(i) [0,1] as probability that ya = li
Cumulative probability za(i) = Σj≤i x*a(j)
0 za(1) za(2) za(h) = 1za(k)za(i)
Choose an interval of length h’
PostInterval Rounding
Treat x*a(i) [0,1] as probability that ya = li
Cumulative probability za(i) = Σj≤i x*a(j)
r
Generate a random number r (0,1]
Assign the label next to r if it is within the interval
za(k)-za(i)0
Choose an interval of length h’ REPEAT
PostInterval Rounding - Example
0 za(1) za(4)za(3)za(2)
0.25 0.5 0.75 1.0
0 zb(1) zb(4)zb(3)zb(2)
0.7 0.8 0.9 1.0
0 zc(1) zc(4)zc(3)zc(2)
0.1 0.2 0.3 1.0
PostInterval Rounding - Example
0 za(1) za(2)
0.25 0.5
0 zb(1) zb(2)
0.7 0.8
0 zc(1) zc(2)
0.1 0.2
r
r
r
PostInterval Rounding - Example
0 za(1) za(4)za(3)za(2)
0.25 0.5 0.75 1.0
0 zb(1) zb(4)zb(3)zb(2)
0.7 0.8 0.9 1.0
0 zc(1) zc(4)zc(3)zc(2)
0.1 0.2 0.3 1.0
PostInterval Rounding - Example
0 zc(1) zc(4)zc(3)zc(2)
0.1 0.2 0.3 1.0
PostInterval Rounding - Example
0 zc(3)zc(2)
0.1 0.2r
-zc(1) -zc(1)
PostInterval Rounding - Example
0 za(1) za(4)za(3)za(2)
0.25 0.5 0.75 1.0
0 zb(1) zb(4)zb(3)zb(2)
0.7 0.8 0.9 1.0
0 zc(1) zc(4)zc(3)zc(2)
0.1 0.2 0.3 1.0
PostEquivalent Move
Interval Move !!
PostInterval MoveIteration t
y S∈ t iff ya = yta or ya interval of labels ∈
∑a θa(ya) + ∑(a,b) wab d(ya,yb)argminy
s.t. y S∈ t
yt+1 =
Start with an initial labeling y0
Choose an interval of labels of length h’
How do we solve this problem?
PostInterval MoveIteration t
y S∈ t iff ya = yta or ya interval of labels ∈
∑a θa(ya) + ∑(a,b) wab d’(ya,yb)argminy
s.t. y S∈ t
yt+1 =
Start with an initial labeling y0
Choose an interval of labels of length h’
Submodular overestimation d’ of d
Post• Existing Work
– Move-Making Algorithms (Efficient)– Linear Programming Relaxation (Accurate)
• Rounding-based Moves– Equivalence– Complete Rounding and Complete Move– Interval Rounding and Interval Move– Hierarchical Rounding and Hierarchical Move
Outline
PostHierarchical Rounding
L1 L2
l1 l2 l3 l4 l5 l6 l7 l8 l9
L3
Hierarchical clustering of labels (e.g. r-HST metrics)
PostHierarchical Rounding
L1 L2
l1 l2 l3 l4 l5 l6 l7 l8 l9
L3
Assign variables to labels L1, L2 or L3
Move down the hierarchy until the leaf level
PostHierarchical Rounding
L1 L2
l1 l2 l3 l4 l5 l6 l7 l8 l9
L3
Assign variables to labels l1, l2 or l3
PostHierarchical Rounding
L1 L2
l1 l2 l3 l4 l5 l6 l7 l8 l9
L3
Assign variables to labels l4, l5 or l6
PostHierarchical Rounding
L1 L2
l1 l2 l3 l4 l5 l6 l7 l8 l9
L3
Assign variables to labels l7, l8 or l9
PostEquivalent Move
Hierarchical Move !!
PostHierarchical Move
L1 L2
l1 l2 l3 l4 l5 l6 l7 l8 l9
L3
Hierarchical clustering of labels (e.g. r-HST metrics)
PostHierarchical Move
L1 L2
l1 l2 l3 l4 l5 l6 l7 l8 l9
L3
Obtain labeling y1 restricted to labels {l1,l2,l3}
PostHierarchical Move
L1 L2
l1 l2 l3 l4 l5 l6 l7 l8 l9
L3
Obtain labeling y2 restricted to labels {l4,l5,l6}
PostHierarchical Move
L1 L2
l1 l2 l3 l4 l5 l6 l7 l8 l9
L3
Obtain labeling y3 restricted to labels {l7,l8,l9}
PostHierarchical Move
L1 L2 L3
Va Vb
y1(a)
y2(a)
y3(a)
Move up the hierarchy until we reach the root
y1(b)
y2(b)
y3(b)
PostSimple Example - Rounding
θa(1)xa(1) + θa(2)xa(2) + θb(1)xb(1) + θb(2)xb(2)minx≥0
+ d(1,1)xab(1,1) + d(1,2)xab(1,2)+ d(2,1)xab(2,1) + d(2,2)xab(2,2)
xa(1) + xa(2) = 1s.t.
xb(1) + xb(2) = 1
xab(1,1) + xab(1,2) = xa(1)
xab(2,1) + xab(2,2) = xa(2)
xab(1,1) + xab(2,1) = xb(1)
xab(1,2) + xab(2,2) = xb(2)
PostSimple Example - Rounding
x*a(1) + x*a(2) = 1x*a(1)0
x*b(1) + x*b(2) = 1x*b(1)0
Generate a uniform random number r (0,1]
Assign the label next to r
r
r
Probability that Va is assigned label l1? x*a(1)
Probability that Va is assigned label l2? x*a(2)
PostSimple Example - Rounding
x*a(1) + x*a(2) = 1x*a(1)0
x*b(1) + x*b(2) = 1x*b(1)0
Generate a uniform random number r (0,1]
Assign the label next to r
r
r
Probability that Va and Vb are assigned l1 and l1?
min{x*a(1), x*b(1)}
PostSimple Example - Rounding
x*a(1) + x*a(2) = 1x*a(1)0
x*b(1) + x*b(2) = 1x*b(1)0
Generate a uniform random number r (0,1]
Assign the label next to r
r
r
Probability that Va and Vb are assigned l1 and l1?
min{x*ab(1,1)+x*ab(1,2), x*ab(1,1) + x*ab(2,1)}
x*ab(1,1) + min{x*ab(1,2), x*ab(2,1)}
PostSimple Example - Rounding
x*a(1) + x*a(2) = 1x*a(1)0
x*b(1) + x*b(2) = 1x*b(1)0
Generate a uniform random number r (0,1]
Assign the label next to r
r
r
Probability that Va and Vb are assigned l1 and l2?
max{0,x*a(1) - x*b(1)}
PostSimple Example - Rounding
x*a(1) + x*a(2) = 1x*a(1)0
x*b(1) + x*b(2) = 1x*b(1)0
Generate a uniform random number r (0,1]
Assign the label next to r
r
r
Probability that Va and Vb are assigned l1 and l2?
x*ab(1,2) - min{x*ab(1,2), x*ab(2,1)}
max{0,x*ab(1,2) - x*ab(2,1)}
PostSimple Example - Rounding
x*a(1) + x*a(2) = 1x*a(1)0
x*b(1) + x*b(2) = 1x*b(1)0
Generate a uniform random number r (0,1]
Assign the label next to r
r
r
Probability that Va and Vb are assigned l2 and l1?
max{0,x*b(1) - x*a(1)}
PostSimple Example - Rounding
x*a(1) + x*a(2) = 1x*a(1)0
x*b(1) + x*b(2) = 1x*b(1)0
Generate a uniform random number r (0,1]
Assign the label next to r
r
r
Probability that Va and Vb are assigned l2 and l1?
x*ab(2,1) - min{x*ab(1,2), x*ab(2,1)}
max{0,x*ab(2,1) - x*ab(1,2)}
PostSimple Example - Rounding
x*a(1) + x*a(2) = 1x*a(1)0
x*b(1) + x*b(2) = 1x*b(1)0
Generate a uniform random number r (0,1]
Assign the label next to r
r
r
Probability that Va and Vb are assigned l2 and l2?
1-max{x*a(1), x*b(1)}
PostSimple Example - Rounding
x*a(1) + x*a(2) = 1x*a(1)0
x*b(1) + x*b(2) = 1x*b(1)0
Generate a uniform random number r (0,1]
Assign the label next to r
r
r
Probability that Va and Vb are assigned l2 and l2?
min{x*a(2), x*b(2)}
PostSimple Example - Rounding
x*a(1) + x*a(2) = 1x*a(1)0
x*b(1) + x*b(2) = 1x*b(1)0
Generate a uniform random number r (0,1]
Assign the label next to r
r
r
Probability that Va and Vb are assigned l2 and l2?
min{x*ab(2,2)+x*ab(1,2), x*ab(2,2) + x*ab(2,1)}
x*ab(2,2) + min{x*ab(1,2), x*ab(2,1)}
PostSimple Example - Move
θa(ya) + θb(yb)miny + d(ya,yb)
ya ,yb {1,2}∈
If d is submodular, solve using graph cuts
Otherwise
PostSimple Example - Move
θa(ya) + θb(yb)miny + d’(ya,yb)
ya ,yb {0,1}∈
If d is submodular, solve using graph cuts
Otherwise use submodular overestimation d’
Estimate d’ by minimizing distortion
PostSimple Example - Move
bmind'
d’(1,1) ≤ b d(1,1)s.t. d’(1,2) ≤ b d(1,2)
d’(2,1) ≤ b d(2,1) d’(2,2) ≤ b d(2,2)
d(1,1) ≤ d’(1,1) d(1,2) ≤ d’(1,2)
d(2,1) ≤ d’(2,1) d(2,2) ≤ d’(2,2)
d’(1,1) + d’(2,2) ≤ d’(2,1) + d’(2,2)
Dual LP provides worst-case rounding example
LP in the variables d’(i,k)
PostSimple Example - Moved(1,1)β(1,1)+d(1,2)β(1,2)+d(2,1)β(2,1)+d(2,2)β(2,2)minα,β,γ≥0
s.t. d(1,1)α(1,1)+d(1,2)α(1,2)+d(2,1)α(2,1)+d(2,2)α(2,2) = 1
β(1,1) = α(1,1) + γ
β(1,2) = α(1,2) - γ
β(2,1) = α(2,1) - γ
β(2,2) = α(2,2) + γ
Set xab*(i,k) = α(i,k) Set γ = min{xab*(1,2), xab*(2,1)}