![Page 1: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/1.jpg)
ICCV Tutorial 2007ICCV Tutorial 2007
Philip TorrPapers, presentations and videos on web.....
![Page 2: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/2.jpg)
OverviewOverview
Recycling Solutions
Illustrate on 2D graph cuts, generalization to 3D easy
![Page 3: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/3.jpg)
MRF for Image & Video MRF for Image & Video SegmentationSegmentation
EnergyMRF
Pair-wise Terms MAP SolutionUnary likelihoodData (D)
Unary likelihood Contrast Term Uniform Prior(Potts Model)
Seminal Work of Boykov and Jolly [ICCV 2001]
Maximum-a-posteriori (MAP) solution x* = arg min E(x)x
=
![Page 4: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/4.jpg)
Dynamic Graph CutsDynamic Graph Cuts
PB SB
cheaperoperation
computationally
expensive operation
Simplerproblem
PB*
differencesbetweenA and B
A and Bsimilar
PA SA
solve
![Page 5: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/5.jpg)
First segmentation problem MAP solution
Ga
Our AlgorithmOur Algorithm
Gb
second segmentation problem
Maximum flow
residual graph (Gr)
G`
differencebetween
Ga and Gbupdated residual
graph
![Page 6: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/6.jpg)
Energy Minimization using Energy Minimization using Graph Graph cutscuts
EMRF(a1,a2)
Sink (0)
Source (1)
a1 a2
What really happens? Building the graph
![Page 7: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/7.jpg)
Energy Minimization using Energy Minimization using Graph Graph cutscuts
If we cut a link to source or sink this indicates label
Sink (0)
Source (1)
a1 a2
What really happens? Building the graph
![Page 8: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/8.jpg)
Energy Minimization using Energy Minimization using Graph Graph cutscuts
Sink (0)
Source (1)
a1 a2
EMRF(a1,a2) = 2a1
2
What really happens? Building the graph
![Page 9: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/9.jpg)
Energy Minimization using Energy Minimization using Graph Graph cutscuts
What really happens? Building the graph
EMRF(a1,a2) = 2a1 + 5ā1
Sink (0)
Source (1)
a1 a2
2
5
![Page 10: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/10.jpg)
Energy Minimization using Energy Minimization using Graph Graph cutscuts
What really happens? Building the graph
EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2
Sink (0)
Source (1)
a1 a2
2
5
9
4
![Page 11: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/11.jpg)
Energy Minimization using Energy Minimization using Graph Graph cutscuts
What really happens? Building the graph
EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2
Sink (0)
Source (1)
a1 a2
2
5
9
4
2
![Page 12: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/12.jpg)
Energy Minimization using Energy Minimization using Graph Graph cutscuts
What really happens? Building the graph
EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 + ā1a2
Sink (0)
Source (1)
a1 a2
2
5
9
4
2
1
![Page 13: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/13.jpg)
Energy Minimization using Energy Minimization using Graph Graph cutscuts
What really happens? Building the graph
EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 + ā1a2
Sink (0)
Source (1)
a1 a2
2
5
9
4
2
1
n-edges(pair-wise term)
t-edges(unary terms)
![Page 14: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/14.jpg)
Energy Minimization using Energy Minimization using Graph Graph cutscuts
What really happens? Building the graph
EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 + ā1a2
Sink (0)
Source (1)
a1 a2
2
5
9
4
2
1a1 = 1 a2 = 1
EMRF(1,1) = 11
Cost of st-cut = 11
![Page 15: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/15.jpg)
Energy Minimization using Energy Minimization using Graph Graph cutscuts
What really happens? Building the graph
EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 + ā1a2
Sink (0)
Source (1)
a1 a2
2
5
9
4
2
1a1 = 1 a2 = 0
EMRF(1,0) = 8
Cost of st-cut = 8
![Page 16: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/16.jpg)
Energy Minimization using Energy Minimization using Graph Graph cutscuts
What really happens? Building the graph
EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 + ā1a2
Sink (0)
Source (1)
a1 a2
2
5
9
4
2
1a1 = 1 a2 = 0
EMRF(1,0) = 8
Cost of st-cut = 8
+ CONSTANT TERM K
![Page 17: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/17.jpg)
Energy Minimization using Energy Minimization using Graph Graph cutscuts
Posiform
EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 + ā1a2
+ CONSTANT TERM K
Posiform is a multilinear polynomials in binaryvariables with positive coefficients
![Page 18: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/18.jpg)
• The Max-flow Problem- Edge capacity and flow balance constraints
Computing the st-mincut from Computing the st-mincut from Max-flow Max-flow algorithmsalgorithms
• Notation- Residual capacity (edge capacity – current flow)- Augmenting path
• Simple Augmenting Path based Algorithms- Repeatedly find augmenting paths and push flow.- Saturated edges constitute the st-mincut. [Ford-Fulkerson Theorem]
Sink (1)
Source (0)
a1 a2
2
5
9
42
1
![Page 19: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/19.jpg)
9 + α
4 + α
Adding a constant to both thet-edges of a node does not change the edges constituting the st-mincut.
Key Observation
Sink (1)
Source (0)
a1 a2
2
5
2
1
E (a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 + ā1a2
E*(a1,a2 ) = E(a1,a2) + α(a2+ā2)
= E(a1,a2) + α [a2+ā2 =1]
ReparametrizationReparametrization
![Page 20: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/20.jpg)
9 + α
4
All reparametrizations of the graph are sums of these two types.
Other type of reparametrization
Sink (1)
Source (0)
a1 a2
2
5 + α
2 + α
1 - α
Reparametrization, second typeReparametrization, second type
Both maintain the solution and add a constant α to the energy.
![Page 21: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/21.jpg)
ReparametrizationReparametrization
• Nice result (easy to prove)
• All other reparametrizations can be viewed in terms of these two basic operations.
• Proof in Hammer, and also in one of Vlad’s recent papers.
![Page 22: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/22.jpg)
s
Gt
original graph
0/9
0/7
0/5
0/2 0/4
0/1
xi xj
flow/residual capacity
Graph Re-parameterizationGraph Re-parameterization
![Page 23: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/23.jpg)
s
Gt
original graph
0/9
0/7
0/5
0/2 0/4
0/1
xi xj
flow/residual capacity
Graph Re-parameterizationGraph Re-parameterization
t residual graph
xi xj0/12
5/2
3/2
1/0
2/0 4/0st-mincut
ComputeMaxflow
Gr
Edges cut
![Page 24: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/24.jpg)
Update Update t-edget-edge CapacitiesCapacities
s
Gr
t residual graph
xi xj0/12
5/2
3/2
1/0
2/0 4/0
![Page 25: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/25.jpg)
Update Update t-edget-edge CapacitiesCapacities
s
Gr
t residual graph
xi xj0/12
5/2
3/2
1/0
2/0 4/0
capacitychanges from
7 to 4
![Page 26: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/26.jpg)
Update Update t-edget-edge CapacitiesCapacities
s
G`t
updated residual graph
xi xj0/12
5/-1
3/2
1/0
2/0 4/0
capacitychanges from
7 to 4
edge capacityconstraint violated!(flow > capacity)
= 5 – 4 = 1
excess flow (e) = flow – new capacity
![Page 27: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/27.jpg)
add e to both t-edges connected to node i
Update Update t-edget-edge CapacitiesCapacities
s
G`t
updated residual graph
xi xj0/12
3/2
1/0
2/0 4/0
capacitychanges from
7 to 4
edge capacityconstraint violated!(flow > capacity)
= 5 – 4 = 1
excess flow (e) = flow – new capacity
5/-1
![Page 28: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/28.jpg)
Update Update t-edget-edge CapacitiesCapacities
s
G`t
updated residual graph
xi xj0/12
3/2
1/0
4/0
capacitychanges from
7 to 4
excess flow (e) = flow – new capacity
add e to both t-edges connected to node i
= 5 – 4 = 1
5/0
2/1
edge capacityconstraint violated!(flow > capacity)
![Page 29: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/29.jpg)
Update Update n-edgen-edge CapacitiesCapacities
s
Gr
t
residual graph
xi xj0/12
5/2
3/2
1/0
2/0 4/0
• Capacity changes from 5 to 2
![Page 30: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/30.jpg)
Update Update n-edgen-edge CapacitiesCapacities
s
t
Updated residual graph
xi xj0/12
5/2
3/-1
1/0
2/0 4/0
G`
• Capacity changes from 5 to 2- edge capacity constraint violated!
![Page 31: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/31.jpg)
Update Update n-edgen-edge CapacitiesCapacities
s
t
Updated residual graph
xi xj0/12
5/2
3/-1
1/0
2/0 4/0
G`
• Capacity changes from 5 to 2- edge capacity constraint violated!
• Reduce flow to satisfy constraint
![Page 32: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/32.jpg)
Update Update n-edgen-edge CapacitiesCapacities
s
t
Updated residual graph
xi xj0/11
5/2
2/0
1/0
2/0 4/0
excess
deficiency
G`
• Capacity changes from 5 to 2- edge capacity constraint violated!
• Reduce flow to satisfy constraint- causes flow imbalance!
![Page 33: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/33.jpg)
Update Update n-edgen-edge CapacitiesCapacities
s
t
Updated residual graph
xi xj0/11
5/2
2/0
1/0
2/0 4/0
deficiency
excess
G`
• Capacity changes from 5 to 2- edge capacity constraint violated!
• Reduce flow to satisfy constraint- causes flow imbalance!
• Push excess flow to/from the terminals
• Create capacity by adding α = excess to both t-edges.
![Page 34: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/34.jpg)
Update Update n-edgen-edge CapacitiesCapacities
Updated residual graph
• Capacity changes from 5 to 2- edge capacity constraint violated!
• Reduce flow to satisfy constraint- causes flow imbalance!
• Push excess flow to the terminals
• Create capacity by adding α = excess to both t-edges.
G`
xi xj0/11
5/3
2/0
2/0
3/0 4/1
s
t
![Page 35: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/35.jpg)
Update Update n-edgen-edge CapacitiesCapacities
Updated residual graph
• Capacity changes from 5 to 2- edge capacity constraint violated!
• Reduce flow to satisfy constraint- causes flow imbalance!
• Push excess flow to the terminals
• Create capacity by adding α = excess to both t-edges.
G`
xi xj0/11
5/3
2/0
2/0
3/0 4/1
s
t
![Page 36: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/36.jpg)
Complexity analysisComplexity analysis of MRF Update of MRF Update OperationsOperations
MRF Energy Operation
Graph Operation Complexity
modifying a unary term
modifying a pair-wise term
adding a latent variable
delete a latent variable
Updating a t-edge capacity
Updating a n-edge capacity
adding a node
set the capacities of all edges of a node zero
O(1)
O(1)
O(1)
O(k)*
*requires k edge update operations where k is degree of the node
![Page 37: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/37.jpg)
• Finding augmenting paths is time consuming.
• Dual-tree maxflow algorithm [Boykov & Kolmogorov PAMI 2004]
- Reuses search trees after each augmentation.
- Empirically shown to be substantially faster.
• Our Idea
– Reuse search trees from previous graph cut computation
– Saves us search tree creation tree time [O(#edges)]
– Search trees have to be modified to make them consistent with new graphs
– Constrain the search of augmenting paths
• New paths must contain at least one updated edge
Improving the AlgorithmImproving the Algorithm
![Page 38: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/38.jpg)
Reusing Search TreesReusing Search Trees
c’ = measure of change in the energy
– Running time
• Dynamic algorithm (c’ + re-create search tree )
• Improved dynamic algorithm (c’)
• Video Segmentation Example
- Duplicate image frames (No time is needed)
![Page 39: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/39.jpg)
Experimental AnalysisExperimental Analysis
additional segmentation
cues
user segmentation cuesstatic: 175 msecdynamic : 80 msecdynamic (optimized): 15 msec
static : 175 msec
Interactive Image segmentation (update unary terms)
EnergyMRF =
![Page 40: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/40.jpg)
ExperimentalExperimental AnalysisAnalysis
Image resolution: 720x576 static: 220 msec dynamic (optimized): 50 msec
Image segmentation in videos (unary & pairwise terms)
Graph CutsDynamic Graph Cuts
EnergyMRF =
![Page 41: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/41.jpg)
ExperimentalExperimental AnalysisAnalysis
Image resolution: 720x576 static: 177 msec dynamic (optimized): 60 msec
Image segmentation in videos (unary & pairwise terms)
Graph CutsDynamic Graph Cuts
EnergyMRF =
![Page 42: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/42.jpg)
ExperimentalExperimental AnalysisAnalysis
MRF consisting of 2x105 latent variables connected in a 4-neighborhood.
Running time of the dynamic algorithm
![Page 43: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/43.jpg)
Some ConclusionsSome Conclusions
• An exact dynamic algorithm for computing st-mincut- handles arbitrary changes in the graph
• Rapidly solves dynamic sub-modular energy functions
• Running time roughly proportional to the number of changes in the energy terms.
![Page 44: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/44.jpg)
ObjCut and PoseCut:ObjCut and PoseCut:
• Combination of Top Down and Bottom Up Cues..
• Classical Vision Problem.
• Combine theory of MRF’s with Object Recognition
![Page 45: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/45.jpg)
ObjectiveObjective
Image Segmentation Pose Estimate??
Aim to get a clean segmentation of a human…
![Page 46: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/46.jpg)
MRF for Interactive Image MRF for Interactive Image Segmentation, Segmentation,
Boykov and Jolly [ICCV 2001]Boykov and Jolly [ICCV 2001]
EnergyMR
F
Pair-wise Terms MAP SolutionUnary likelihoodData (D)
Unary likelihood Contrast Term Uniform Prior(Potts Model)
Maximum-a-posteriori (MAP) solution x* = arg min E(x)x
=
![Page 47: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/47.jpg)
However…However…
• This energy formulation rarely provides realistic (target-
like) results.
![Page 48: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/48.jpg)
Shape-Priors and SegmentationShape-Priors and Segmentation
• Combine object detection with segmentation
– Obj-Cut, Kumar et al., CVPR ’05
– Zhao and Davis, ICCV ’05
• Obj-Cut
– Shape-Prior: Layered Pictorial Structure (LPS)
– Learned exemplars for parts of the LPS model
– Obtained impressive results
+
Layer 1 Layer 2
=
LPS model
![Page 49: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/49.jpg)
LPS for DetectionLPS for Detection• Learning
– Learnt automatically using a set of examples
• DetectionTree of chamfers to detect parts, assemble with pictorial
structure and belief propogation.
![Page 50: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/50.jpg)
Solve via Integer ProgrammingSolve via Integer Programming
• SDP formulation (Torr 2001, AI stats)
• SOCP formulation (Kumar, Torr & Zisserman this conference)
• LBP (Huttenlocher, many)
![Page 51: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/51.jpg)
Obj-CutObj-CutImage Likelihood Ratio (Colour)
Shape Prior Distance from
Likelihood + Distance from
![Page 52: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/52.jpg)
Integrating Shape-Prior in MRFsIntegrating Shape-Prior in MRFs
Unary potential
Pairwise potential
Labels
Pixels
Prior Potts model
MRF for segmentation
![Page 53: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/53.jpg)
Integrating Shape-Prior in MRFsIntegrating Shape-Prior in MRFs
Unary potential
Pairwise potential
Pose parameters
Labels
Pixels
Prior Potts model
Pose-specific MRF
![Page 54: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/54.jpg)
SegmentationImage
ResultsResultsUsing LPS Model for Cow
![Page 55: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/55.jpg)
Layer 2
Layer 1
Transformations
Θ1
P(Θ1) = 0.9
Cow Instance
Do we really need accurate models?Do we really need accurate models?
![Page 56: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/56.jpg)
Do we really need accurate Do we really need accurate models?models?
• Segmentation boundary can be extracted from edges
• Rough 3D Shape-prior enough for region disambiguation
![Page 57: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/57.jpg)
Energy of the Pose-specific MRFEnergy of the Pose-specific MRFEnergy to be
minimizedUnary term
Shape prior
Pairwise potential
Potts model
But what should be the value of θ?
![Page 58: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/58.jpg)
The different terms of the MRFThe different terms of the MRF
Original image
Likelihood of being foreground given a
foreground histogram
Grimson-Stauffer
segmentation
Shape prior model
Shape prior (distance transform)
Likelihood of being foreground
given all the terms
Resulting Graph-Cuts
segmentation
![Page 59: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/59.jpg)
Can segment multiple views Can segment multiple views simultaneouslysimultaneously
![Page 60: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/60.jpg)
Solve via gradient descentSolve via gradient descent
• Comparable to level set methods
• Could use other approaches (e.g. Objcut)
• Need a graph cut per function evaluation
![Page 61: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/61.jpg)
Formulating the Pose Inference Formulating the Pose Inference ProblemProblem
![Page 62: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/62.jpg)
But…But…
… to compute the MAP of E(x) w.r.t the pose, it means that the unary terms will be changed at EACHEACH iteration and the maxflow recomputed!
However…However… Kohli and Torr showed how dynamic graph cuts can
be used to efficiently find MAP solutions for MRFs that change minimally from one time instant to the next: Dynamic Graph Cuts (ICCV05).
![Page 63: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/63.jpg)
Dynamic Graph CutsDynamic Graph Cuts
PB SB
cheaperoperation
computationally
expensive operation
Simplerproblem
PB*
differencesbetweenA and B
A and Bsimilar
PA SA
solve
![Page 64: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/64.jpg)
Segmentation ComparisonSegmentation Comparison
Gri
mson
-G
rim
son
-S
tau
ffer
Sta
uff
er
Bath
ia0
Bath
ia0
44O
ur
Ou
r m
eth
od
meth
od
![Page 65: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/65.jpg)
SegmentationSegmentation
![Page 66: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/66.jpg)
Combine Recognition and Combine Recognition and SegmentationSegmentation
• Using all the information should help…– E.g. segmentation result can be used to
eliminate false positives.
• Face/Head and shoulders segmentation particularly useful for applications such as Skype or Windows Messenger.
![Page 67: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/67.jpg)
ObjCut Face SegmentationObjCut Face Segmentation
![Page 68: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/68.jpg)
Face Detector and ObjCutFace Detector and ObjCut
![Page 69: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/69.jpg)
Segmentation + RecognitionSegmentation + Recognitionto prune false positivesto prune false positives
![Page 70: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/70.jpg)
InitialisationInitialisation
• Use off-the-shelf face detector to find location of face in an image
![Page 71: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/71.jpg)
InitialisationInitialisation
• Place shape prior relative to face detection
• Define region over which to perform segmentation
![Page 72: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/72.jpg)
AdjustmentAdjustment
• Vary parameters of shape prior to find lowest segmentation energy
![Page 73: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/73.jpg)
Other Applications of Dynamic Other Applications of Dynamic Graph CutsGraph Cuts
![Page 74: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/74.jpg)
Computing Max-marginalsComputing Max-marginals
Graph Cuts
Dynamic Graph Cuts
Image (MSRC) Segmentation Max-Marginal (Fg)
: 0.002 msec per MM (1.2 secs)
: 300 msec per MM (51.2 days)
1
0
0.5
![Page 75: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/75.jpg)
Parameter Learning in CRFsParameter Learning in CRFs
• Maximum Likelihood Parameter Learning– Kumar and Herbert – NIPS 2004, EMMCVPR 2005
![Page 76: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/76.jpg)
PP33 && BeyondBeyondSolving Energies with Higher Order Solving Energies with Higher Order
CliquesCliques
Pushmeet Kohli Pawan Kumar Philip H. S. TorrOxford Brookes University
![Page 77: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/77.jpg)
Energy FunctionsEnergy Functions
• Pairwise Energy Functions
Unary Pairwise
Markov Random Field
Observed Variables
Hidden Variables
![Page 78: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/78.jpg)
Energy FunctionsEnergy Functions
• Pairwise Energy Functions
Unary Pairwise
Markov Random Field
![Page 79: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/79.jpg)
Energy FunctionsEnergy Functions
• Pairwise Energy Functions
Unary Pairwise
Markov Random Field
![Page 80: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/80.jpg)
Energy FunctionsEnergy Functions
• Pairwise Energy Functions
Unary Pairwise
Markov Random Field
![Page 81: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/81.jpg)
Energy FunctionsEnergy Functions
• Pairwise Energy Functions
Unary Pairwise
Markov Random Field
• Efficient Algorithms for Minimization
• Restricted Expressive Power!
![Page 82: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/82.jpg)
Energy FunctionsEnergy Functions
• Higher Order Energy Functions
Unary Pairwise Higher order
Markov Random Field
More expressive than pairwise
FOE: Field of Experts(Roth & Black CVPR05)
![Page 83: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/83.jpg)
Energy FunctionsEnergy Functions
• Higher Order Energy Functions
Unary Pairwise Higher order
Original Pairwise MRF Higher order MRF
Images Courtesy: Lan et al. ECCV06
MRF for Image Denoising
![Page 84: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/84.jpg)
Energy FunctionsEnergy Functions
• Higher Order Energy Functions
Unary Pairwise Higher order
• Computationally expensive to minimize!• Exponential Complexity O(LN)
– L = Number of Labels
– N = Size of Clique
![Page 85: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/85.jpg)
Move Making AlgorithmsMove Making Algorithms
Solution Space
En
erg
y
![Page 86: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/86.jpg)
Move Making AlgorithmsMove Making Algorithms
Solution Space
En
erg
ySearch Neighbourhood
Current Solution
Optimal Move
![Page 87: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/87.jpg)
Move Making AlgorithmsMove Making Algorithms• Moves using graph cuts
[Boykov, Veksler, Zabih] PAMI 2001
• Algorithm– Encode move by vector t
– Transformation function T(x,t) → x`– Move Energy
– Optimal move t*
Submodular
![Page 88: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/88.jpg)
Expansion MoveExpansion MoveCharacteristics• Move
• Variables take label or retain current label
• Algorithm– Make a move for all in L
![Page 89: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/89.jpg)
Expansion MoveExpansion Move
Sky
House
Tree
GroundInitialize with Tree
Status:
Expand GroundExpand HouseExpand Sky
![Page 90: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/90.jpg)
Expansion MoveExpansion MoveCharacteristics• Neighbourhood Size
• 2N where N is the number of variables
• Guarantee• Move energy is submodular for all
metric energy functions. [Boykov, Veksler, Zabih] PAMI 2001
![Page 91: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/91.jpg)
Swap MoveSwap MoveCharacteristics• Move
• Variables labeled can swap their labels
• Algorithm– Make a move for all in L
![Page 92: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/92.jpg)
Swap MoveSwap Move
Sky
House
Tree
GroundSwap Sky, House
![Page 93: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/93.jpg)
Swap MoveSwap MoveCharacteristics• Neighbourhood Size
• 2N where N is the number of variables
• Guarantee• Move energy is submodular for all
semi-metric energy functions. [Boykov, Veksler, Zabih] PAMI 2001
![Page 94: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/94.jpg)
• Form of the Higher Order Potentials
• Sum Form
• Max Form
Moves for Higher Order CliquesMoves for Higher Order Cliques
![Page 95: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/95.jpg)
Theoretical Results: SwapTheoretical Results: Swap• Move energy is always submodular if
non-decreasing concave.
See paper for proofs
![Page 96: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/96.jpg)
Theoretical Results: ExpansionTheoretical Results: Expansion• Move energy is always submodular if
increasing linear
See paper for proofs
![Page 97: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/97.jpg)
PPNN Potts Model Potts Model
• Generalization of the Potts Model:
![Page 98: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/98.jpg)
Solving the Solving the PPNN Potts Model Potts Model• Computing the optimal swap move
v1 v2 vn
Ms
Mtti = 0 vi Source
tj = 1 vj Sink
Source
Sink
![Page 99: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/99.jpg)
Solving the Solving the PPNN Potts Model Potts Model• Computing the optimal swap move
Case 1: all ti = 0 (xi = )
Cost:
Source
Sink
v1 v2 vn
Ms
Mt
![Page 100: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/100.jpg)
Solving the Solving the PPNN Potts Model Potts Model• Computing the optimal swap move
v1 v2 vn
Ms
MtCost:
Source
Sink
Case 2: all ti = 1 (xi = )
![Page 101: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/101.jpg)
Solving the Solving the PPNN Potts Model Potts Model• Computing the optimal swap move
Cost:
v1 v2 vn
Ms
Mt
Source
Sink
Case 3: ti = 0,1 (xi = )
![Page 102: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/102.jpg)
Solving the Solving the PPNN Potts Model Potts Model• Computing the optimal expansion move
v1 v2 vn
Ms
Mt
Source
Sink
![Page 103: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/103.jpg)
Solving the Solving the PPNN Potts Model Potts Model• Computing the optimal expansion move
Cost:
Source
Sink
v1 v2 vn
Ms
Mt
Case 1: all ti = 0 (xi = xi )
![Page 104: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/104.jpg)
Solving the Solving the PPNN Potts Model Potts Model• Computing the optimal expansion move
Cost:
v1 v2 vn
Ms
Mt
Source
Sink
Case 2: all ti = 1 (xi = )
![Page 105: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/105.jpg)
Solving the Solving the PPNN Potts Model Potts Model• Computing the optimal expansion move
Cost:
v1 v2 vn
Ms
Mt
Source
Sink
Case 3: ti = 0,1 (xi = xi , )
![Page 106: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/106.jpg)
Experimental ResultsExperimental Results
• Exemplar based Texture Segmentation
Unary(Colour)
Pairwise(Smoothness)
Higher Order(Texture)
Original Image
![Page 107: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/107.jpg)
Experimental ResultsExperimental Results
• Exemplar based Texture Segmentation
Unary(Colour)
Pairwise(Smoothness)
Higher Order(Texture)
Colour Histograms Unary Cost: Tree
![Page 108: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/108.jpg)
Experimental ResultsExperimental Results
• Exemplar based Texture Segmentation
Unary(Colour)
Pairwise(Smoothness)
Higher Order(Texture)
Edge Sensitive Smoothness Cost
![Page 109: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/109.jpg)
Experimental ResultsExperimental Results
• Exemplar based Texture Segmentation
Unary(Colour)
Pairwise(Smoothness)
Higher Order(Texture)
MAP Solution
![Page 110: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/110.jpg)
Experimental ResultsExperimental Results
• Exemplar based Texture Segmentation
Unary(Colour)
Pairwise(Smoothness)
Higher Order(Texture)
Patch Dictionary Higher Order Cost: Tree
![Page 111: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/111.jpg)
Experimental ResultsExperimental Results
• Exemplar based Texture Segmentation
Unary(Colour)
Pairwise(Smoothness)
Higher Order(Texture)
Higher Order Cost: Tree Unary Cost: Tree
![Page 112: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/112.jpg)
Experimental ResultsExperimental Results
• Exemplar based Texture Segmentation
Unary(Colour)
Pairwise(Smoothness)
Higher Order(Texture)
Original Pairwise Higher order
![Page 113: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/113.jpg)
Experimental ResultsExperimental Results
Original
Ground Truth
Swap (3.2 sec)
Expansion (2.5 sec)
Pairwise Higher Order
Swap (4.2 sec)
Expansion (3.0 sec)
![Page 114: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/114.jpg)
Experimental ResultsExperimental Results
Original
Ground Truth
Pairwise Higher Order
Swap (4.7 sec)
Expansion (3.7sec)
Swap (5.0 sec)
Expansion (4.4 sec)
![Page 115: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/115.jpg)
Conclusions & Future WorkConclusions & Future Work
• Efficient minimization of certain higher order energies
• Can handle very large cliques
• Useful for many Computer Vision problems
• Explore other interesting family of potential functions
![Page 116: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/116.jpg)
Other RelaxationsOther Relaxations
• New work NIPS 2007
• All previous work based on LP relaxation, we demonstrate can get better approximation using SOCP relaxation..
![Page 117: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/117.jpg)
PreliminariesPreliminariesConvex Optimization Program
min f(x)
subject to gi(x) ≤ 0 i=1, … , N
Objective Function
Constraints
• Objective function f(.) should be convex
• Constraints gi(.) should define a convex set
Feasibility region F(P) should be a convex set
P
![Page 118: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/118.jpg)
PreliminariesPreliminariesLinear Program (LP)
min cTx
subject to Ax ≤ b
• Linear objective function
• Linear constraints
![Page 119: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/119.jpg)
PreliminariesPreliminariesQuadratic Program (QP)
min xTQx + cTx
subject to Ax ≤ b
• Convex quadratic objective function
• Linear constraints
Q 0
LP < QP
![Page 120: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/120.jpg)
PreliminariesPreliminariesSecond Order Cone Program (SOCP)
min cTx
subject to xTQix + xTai ≤ bi
• Linear objective function
• Convex quadratic constraints Qi 0
LP < QP < SOCP
![Page 121: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/121.jpg)
ConclusionsConclusions
• Large class of SOCP/QP dominated by LP-S
• New SOCP relaxations dominate LP-S
• Experiments conform with analysis
• Determining “best” SOCP constraints ??
![Page 122: ICCV Tutorial 2007 Philip Torr Papers, presentations and videos on web](https://reader037.vdocuments.site/reader037/viewer/2022110207/56649d405503460f94a1ac72/html5/thumbnails/122.jpg)
SOCP Relaxations on Cycles SOCP Relaxations on Cycles
• True for odd cycles with Pij ≤ 0
• True for odd cycles with Pij ≤ 0 for only one edge
• True for odd cycles with Pij ≥ 0 for only one edge
• True for all combinations of above cases