high-order inference, ranking, and ... - puneet dokania€¦ · puneet k. dokania 10. parsimonious...

126
High-Order Inference, Ranking, and Regularization Path for Structured SVM Puneet Kumar Dokania Supervisors: Prof. M. Pawan Kumar & Prof. Nikos Paragios CentraleSup´ elec and INRIA Saclay May 30, 2016 Puneet K. Dokania 1

Upload: others

Post on 03-Aug-2020

8 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

High-Order Inference, Ranking, andRegularization Path for Structured SVM

Puneet Kumar DokaniaSupervisors: Prof. M. Pawan Kumar & Prof. Nikos Paragios

CentraleSupelec and INRIA Saclay

May 30, 2016

Puneet K. Dokania 1

Page 2: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Thesis Overview

Presentation Outline

1 Thesis Overview

2 Parsimonious Labeling

3 Learning to Rank Using High-Order Information

4 Regularization Path for SSVM

5 Future Work

6 Publications

Puneet K. Dokania 2

Page 3: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Thesis Overview

ick Overview

High-Order Inference: Parsimonious Labeling

E(x, y;w) =∑i∈V

θ(xi, yi;w) +∑c∈C

θc(xc, yc;w)︸ ︷︷ ︸diversity

HOAP-SVM: w very high-dimensional→ exhaustive search

min

w

λ

2

‖w‖2 + L(x, y;w)︸ ︷︷ ︸AP-Based

Regularization path for SSVM: Eiciently explore the entire space of

λ ∈ [0,∞]

Puneet K. Dokania 3

Page 4: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Thesis Overview

ick Overview

High-Order Inference: Parsimonious Labeling

E(x, y;w) =∑i∈V

θ(xi, yi;w) +∑c∈C

θc(xc, yc;w)︸ ︷︷ ︸diversity

HOAP-SVM: w very high-dimensional→ exhaustive search

min

w

λ

2

‖w‖2 + L(x, y;w)︸ ︷︷ ︸AP-Based

Regularization path for SSVM: Eiciently explore the entire space of

λ ∈ [0,∞]

Puneet K. Dokania 3

Page 5: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Thesis Overview

ick Overview

High-Order Inference: Parsimonious Labeling

E(x, y;w) =∑i∈V

θ(xi, yi;w) +∑c∈C

θc(xc, yc;w)︸ ︷︷ ︸diversity

HOAP-SVM: w very high-dimensional→ exhaustive search

min

w

λ

2

‖w‖2 + L(x, y;w)︸ ︷︷ ︸AP-Based

Regularization path for SSVM: Eiciently explore the entire space of

λ ∈ [0,∞]

Puneet K. Dokania 3

Page 6: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling

Presentation Outline

1 Thesis Overview

2 Parsimonious Labeling

3 Learning to Rank Using High-Order Information

4 Regularization Path for SSVM

5 Future Work

6 Publications

Puneet K. Dokania 4

Page 7: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Background

The Labeling Problem

Input

Laice V = 1, · · · ,N, Random variables y = y1, · · · , yNA discrete label set L = l1, · · · , lHEnergy functional to assess the quality of each labeling y:

E(y) =∑i∈V

θi(yi) +∑c∈C

θc(yc). (1)

Output

Labeling corresponding to the minimum energy

y∗ = argmin

yE(y). (2)

HN possible labelings

Puneet K. Dokania 5

Page 8: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Background

The Labeling Problem

Input

Laice V = 1, · · · ,N, Random variables y = y1, · · · , yNA discrete label set L = l1, · · · , lHEnergy functional to assess the quality of each labeling y:

E(y) =∑i∈V

θi(yi) +∑c∈C

θc(yc). (1)

Output

Labeling corresponding to the minimum energy

y∗ = argmin

yE(y). (2)

HN possible labelings

Puneet K. Dokania 5

Page 9: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Background

Special case – Metric Labeling (Pairwise)

Pairwise Potentials θ(yi, yj)→Metric over the labels

Recall, distance function θ(yi, yj) : L × L → R+ is metric if:

Non Negative

Symmetric

Triangular Inequality

α−expansion1

– Very Eicient – Approximate solution

1Boykov et al., Fast Approximate Energy Minimization via Graph Cuts, 2001.

Puneet K. Dokania 6

Page 10: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Background

Special case – PnPos Model

2(High-Order)

Pn Pos Model

θc(yc) ∝

γk , if yi = lk ,∀i ∈ c,

γmax , otherwise,

Very eicient α-expansion algorithm – Approximate solution

2Kohli et al., P3 & Beyond: Solving Energies with Higher Order Cliques, 2007.

Puneet K. Dokania 7

Page 11: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Background

Special case – PnPos Model

2(High-Order)

Pn Pos Model

θc(yc) ∝

γk , if yi = lk ,∀i ∈ c,

γmax , otherwise,

Very eicient α-expansion algorithm – Approximate solution

2Kohli et al., P3 & Beyond: Solving Energies with Higher Order Cliques, 2007.

Puneet K. Dokania 7

Page 12: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Background

Special case – PnPos Model

2(High-Order)

Pn Pos Model

θc(yc) ∝

γk , if yi = lk , ∀i ∈ c,

γmax , otherwise,

Very eicient α-expansion algorithm – Approximate solution

2Kohli et al., P3 & Beyond: Solving Energies with Higher Order Cliques, 2007.

Puneet K. Dokania 7

Page 13: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling The Energy Function

Parsimonious Labeling: Energy Function

E(y) =∑i∈V

θi(yi) +∑c∈C

θc(yc).

Unary potentials: Arbitrary

Clique potentials: Diversity

θc(yc) ∝ δ(Γ(yc))︸ ︷︷ ︸Diversity

where, Γ(yc) is the set of unique

labels δc(l1, l2, l3)Energy function for Parsimonious Labeling

E(y) =∑i∈V

θi(yi) +∑c∈C

wcδ(Γ(yc))︸ ︷︷ ︸Diversity

Puneet K. Dokania 8

Page 14: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling The Energy Function

Parsimonious Labeling: Energy Function

E(y) =∑i∈V

θi(yi) +∑c∈C

θc(yc).

Unary potentials: Arbitrary

Clique potentials: Diversity

θc(yc) ∝ δ(Γ(yc))︸ ︷︷ ︸Diversity

where, Γ(yc) is the set of unique

labels

δc(l1, l2, l3)Energy function for Parsimonious Labeling

E(y) =∑i∈V

θi(yi) +∑c∈C

wcδ(Γ(yc))︸ ︷︷ ︸Diversity

Puneet K. Dokania 8

Page 15: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling The Energy Function

Parsimonious Labeling: Energy Function

E(y) =∑i∈V

θi(yi) +∑c∈C

θc(yc).

Unary potentials: Arbitrary

Clique potentials: Diversity

θc(yc) ∝ δ(Γ(yc))︸ ︷︷ ︸Diversity

where, Γ(yc) is the set of unique

labels δc(l1, l2, l3)

Energy function for Parsimonious Labeling

E(y) =∑i∈V

θi(yi) +∑c∈C

wcδ(Γ(yc))︸ ︷︷ ︸Diversity

Puneet K. Dokania 8

Page 16: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling The Energy Function

Parsimonious Labeling: Energy Function

E(y) =∑i∈V

θi(yi) +∑c∈C

θc(yc).

Unary potentials: Arbitrary

Clique potentials: Diversity

θc(yc) ∝ δ(Γ(yc))︸ ︷︷ ︸Diversity

where, Γ(yc) is the set of unique

labels δc(l1, l2, l3)Energy function for Parsimonious Labeling

E(y) =∑i∈V

θi(yi) +∑c∈C

wcδ(Γ(yc))︸ ︷︷ ︸Diversity

Puneet K. Dokania 8

Page 17: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling The Energy Function

Diversity3: Metric over sets

θc(yc) ∝ δ(Γ(yc))︸ ︷︷ ︸Diversity

Metric over sets δ : L → R,∀L ⊆ L, satisfying

Non Negativity

Triangular Inequality

Monotonicity: L1 ⊆ L2 implies δ(L1) ≤ δ(L2)→ Parsimony

Induced Metric: Every diversity induces a metric:

d(li, lj) = δ(li, lj)

Diameter Diversity: δdia(L) = maxli ,lj∈L d(li, lj)

3Bryant and Tupper, Advances in Mathematics, 2012.

Puneet K. Dokania 9

Page 18: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling The Energy Function

Diversity3: Metric over sets

θc(yc) ∝ δ(Γ(yc))︸ ︷︷ ︸Diversity

Metric over sets δ : L → R,∀L ⊆ L, satisfying

Non Negativity

Triangular Inequality

Monotonicity: L1 ⊆ L2 implies δ(L1) ≤ δ(L2)→ Parsimony

Induced Metric: Every diversity induces a metric:

d(li, lj) = δ(li, lj)

Diameter Diversity: δdia(L) = maxli ,lj∈L d(li, lj)

3Bryant and Tupper, Advances in Mathematics, 2012.

Puneet K. Dokania 9

Page 19: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling The Energy Function

Diversity3: Metric over sets

θc(yc) ∝ δ(Γ(yc))︸ ︷︷ ︸Diversity

Metric over sets δ : L → R,∀L ⊆ L, satisfying

Non Negativity

Triangular Inequality

Monotonicity: L1 ⊆ L2 implies δ(L1) ≤ δ(L2)→ Parsimony

Induced Metric: Every diversity induces a metric:

d(li, lj) = δ(li, lj)

Diameter Diversity: δdia(L) = maxli ,lj∈L d(li, lj)

3Bryant and Tupper, Advances in Mathematics, 2012.

Puneet K. Dokania 9

Page 20: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling The Energy Function

Diversity3: Metric over sets

θc(yc) ∝ δ(Γ(yc))︸ ︷︷ ︸Diversity

Metric over sets δ : L → R,∀L ⊆ L, satisfying

Non Negativity

Triangular Inequality

Monotonicity: L1 ⊆ L2 implies δ(L1) ≤ δ(L2)→ Parsimony

Induced Metric: Every diversity induces a metric:

d(li, lj) = δ(li, lj)

Diameter Diversity: δdia(L) = maxli ,lj∈L d(li, lj)

3Bryant and Tupper, Advances in Mathematics, 2012.

Puneet K. Dokania 9

Page 21: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Special cases of Parsimonious Labeling

Special Case 1: Metric Labeling

If cliques are of size 2→ diversity→ metric

Parsimonious Labeling→Metric Labeling4

Many applications in low level vision tasks: Stereo matching,

Inpainting, Denoising, Image stitching.

4Boykov et al., Fast Approximate Energy Minimization via Graph Cuts, 2001.

Puneet K. Dokania 10

Page 22: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Special cases of Parsimonious Labeling

Special Case 1: Metric Labeling

If cliques are of size 2→ diversity→ metric

Parsimonious Labeling→Metric Labeling4

Many applications in low level vision tasks: Stereo matching,

Inpainting, Denoising, Image stitching.

4Boykov et al., Fast Approximate Energy Minimization via Graph Cuts, 2001.

Puneet K. Dokania 10

Page 23: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Special cases of Parsimonious Labeling

Special Case 2: Pn-Pos Model

5

Uniform Metric

d(li, lj) = min(|li − lj|, 1),∀li, lj ∈ L

Diversity→ Diameter diversity over uniform metric

Parsimonious Labeling→ Pn-Pos Model

Labels l1 l2 l3l1 0 1 1

l2 1 0 1

l3 1 1 0

Table: Uniform Metric

θc(l1, l2, l3) = max(d(l1, l2), d(l1, l3), d(l2, l3))

= 1

θc(yc) ∝

0, if yi = lk ,∀i ∈ c,

1, otherwise,

5Kohli et al., P3 “& Beyond: Solving Energies with Higher Order Cliques, 2007.

Puneet K. Dokania 11

Page 24: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Special cases of Parsimonious Labeling

Special Case 2: Pn-Pos Model

5

Uniform Metric

d(li, lj) = min(|li − lj|, 1),∀li, lj ∈ L

Diversity→ Diameter diversity over uniform metric

Parsimonious Labeling→ Pn-Pos Model

Labels l1 l2 l3l1 0 1 1

l2 1 0 1

l3 1 1 0

Table: Uniform Metric

θc(l1, l2, l3) = max(d(l1, l2), d(l1, l3), d(l2, l3))

= 1

θc(yc) ∝

0, if yi = lk ,∀i ∈ c,

1, otherwise,

5Kohli et al., P3 “& Beyond: Solving Energies with Higher Order Cliques, 2007.

Puneet K. Dokania 11

Page 25: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Special cases of Parsimonious Labeling

Special Case 2: Pn-Pos Model

5

Uniform Metric

d(li, lj) = min(|li − lj|, 1),∀li, lj ∈ L

Diversity→ Diameter diversity over uniform metric

Parsimonious Labeling→ Pn-Pos Model

Labels l1 l2 l3l1 0 1 1

l2 1 0 1

l3 1 1 0

Table: Uniform Metric

θc(l1, l2, l3) = max(d(l1, l2), d(l1, l3), d(l2, l3))

= 1

θc(yc) ∝

0, if yi = lk ,∀i ∈ c,

1, otherwise,

5Kohli et al., P3 “& Beyond: Solving Energies with Higher Order Cliques, 2007.

Puneet K. Dokania 11

Page 26: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Special cases of Parsimonious Labeling

Special Case 2: Pn-Pos Model

5

Uniform Metric

d(li, lj) = min(|li − lj|, 1),∀li, lj ∈ L

Diversity→ Diameter diversity over uniform metric

Parsimonious Labeling→ Pn-Pos Model

Labels l1 l2 l3l1 0 1 1

l2 1 0 1

l3 1 1 0

Table: Uniform Metric

θc(l1, l2, l3) = max(d(l1, l2), d(l1, l3), d(l2, l3))

= 1

θc(yc) ∝

0, if yi = lk , ∀i ∈ c,

1, otherwise,

5Kohli et al., P3 “& Beyond: Solving Energies with Higher Order Cliques, 2007.

Puneet K. Dokania 11

Page 27: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Special cases of Parsimonious Labeling

So far …

E(y) =∑i∈V

θi(yi) +∑c∈C

wcδ(Γ(yc))︸ ︷︷ ︸Diversity

Puneet K. Dokania 12

Page 28: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Hierarchical Pn Pos Model

Hierarchical PnPos Model

Given tree metric

d t(l1, l2) = 14, d t(l1, l3) = 4, d t(l1, l1) = 0

Hierarchical PnPos Model→ diameter diversity over tree metric

Diameter diversity at cluster p is maxli ,lj dt(li, lj) = 14.

Puneet K. Dokania 13

Page 29: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Hierarchical Pn Pos Model

Hierarchical PnPos Model

Given tree metric

d t(l1, l2) = 14, d t(l1, l3) = 4, d t(l1, l1) = 0

Hierarchical PnPos Model→ diameter diversity over tree metric

Diameter diversity at cluster p is maxli ,lj dt(li, lj) = 14.

Puneet K. Dokania 13

Page 30: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Hierarchical Pn Pos Model

Hierarchical PnPos Model

Given tree metric

d t(l1, l2) = 14, d t(l1, l3) = 4, d t(l1, l1) = 0

Hierarchical PnPos Model→ diameter diversity over tree metric

Diameter diversity at cluster p is maxli ,lj dt(li, lj) = 14.

Puneet K. Dokania 13

Page 31: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Hierarchical Pn Pos Model

Hierarchical PnPos Model

Given tree metric

d t(l1, l2) = 14, d t(l1, l3) = 4, d t(l1, l1) = 0

Hierarchical PnPos Model→ diameter diversity over tree metric

Diameter diversity at cluster p is maxli ,lj dt(li, lj) = 14.

Puneet K. Dokania 13

Page 32: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Hierarchical Pn Pos Model

Move Making Algorithm for Hierarchical PnPos Model

Optimizing directly at the root node is non-trivial

We propose divide and conquer based boom-up approach

Puneet K. Dokania 14

Page 33: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Hierarchical Pn Pos Model

Move Making Algorithm for Hierarchical PnPos Model

Optimizing directly at the root node is non-trivial

We propose divide and conquer based boom-up approach

Puneet K. Dokania 14

Page 34: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Hierarchical Pn Pos Model

Move Making Algorithm for Hierarchical PnPos Model

Optimizing directly at the root node is non-trivial

We propose divide and conquer based boom-up approach

Puneet K. Dokania 14

Page 35: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Hierarchical Pn Pos Model

Move Making Algorithm for Hierarchical PnPos Model

Optimizing directly at the root node is non-trivial

We propose divide and conquer based boom-up approach

Puneet K. Dokania 14

Page 36: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Hierarchical Pn Pos Model

Move Making Algorithm for Hierarchical PnPos Model

Optimizing directly at the root node is non-trivial

We propose divide and conquer based boom-up approach

Puneet K. Dokania 14

Page 37: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Hierarchical Pn Pos Model

Move Making Algorithm for Hierarchical PnPos Model

Optimizing directly at the root node is non-trivial

We propose divide and conquer based boom-up approach

Puneet K. Dokania 14

Page 38: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Hierarchical Pn Pos Model

Move Making Algorithm for Hierarchical PnPos Model

Optimizing directly at the root node is non-trivial

We propose divide and conquer based boom-up approach

Puneet K. Dokania 14

Page 39: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Hierarchical Pn Pos Model

Move Making Algorithm for Hierarchical PnPos Model

Solving the problem at leaf node→ Trivial

Fusing at non-leaf node→ Pn-Pos Model

Puneet K. Dokania 15

Page 40: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Hierarchical Pn Pos Model

Move Making Algorithm for Hierarchical PnPos Model

Solving the problem at leaf node→ Trivial

Fusing at non-leaf node→ Pn-Pos Model

Puneet K. Dokania 15

Page 41: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Move Making Algorithm for Parsimonious Labeling

Move Making for Parsimonious Labeling

Given any general diversity→ Get the induced metric

Induced Metric→Mixture of tree metrics (r-hst)6

Hierarchical Pn-Pos model over each tree metric→ diameter diversity

over each tree metric (r-hst)

Optimize each Hierarchical Pn-Pos model using proposed move

making algorithm

Fuse solutions or choose the one with minimum energy

6Fakcharoenphol et al., In STOC 2003.

Puneet K. Dokania 16

Page 42: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Move Making Algorithm for Parsimonious Labeling

Move Making for Parsimonious Labeling

Given any general diversity→ Get the induced metric

Induced Metric→Mixture of tree metrics (r-hst)6

Hierarchical Pn-Pos model over each tree metric→ diameter diversity

over each tree metric (r-hst)

Optimize each Hierarchical Pn-Pos model using proposed move

making algorithm

Fuse solutions or choose the one with minimum energy

6Fakcharoenphol et al., In STOC 2003.

Puneet K. Dokania 16

Page 43: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Move Making Algorithm for Parsimonious Labeling

Move Making for Parsimonious Labeling

Given any general diversity→ Get the induced metric

Induced Metric→Mixture of tree metrics (r-hst)6

Hierarchical Pn-Pos model over each tree metric→ diameter diversity

over each tree metric (r-hst)

Optimize each Hierarchical Pn-Pos model using proposed move

making algorithm

Fuse solutions or choose the one with minimum energy

6Fakcharoenphol et al., In STOC 2003.

Puneet K. Dokania 16

Page 44: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Move Making Algorithm for Parsimonious Labeling

Move Making for Parsimonious Labeling

Given any general diversity→ Get the induced metric

Induced Metric→Mixture of tree metrics (r-hst)6

Hierarchical Pn-Pos model over each tree metric→ diameter diversity

over each tree metric (r-hst)

Optimize each Hierarchical Pn-Pos model using proposed move

making algorithm

Fuse solutions or choose the one with minimum energy

6Fakcharoenphol et al., In STOC 2003.

Puneet K. Dokania 16

Page 45: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Move Making Algorithm for Parsimonious Labeling

Move Making for Parsimonious Labeling

Given any general diversity→ Get the induced metric

Induced Metric→Mixture of tree metrics (r-hst)6

Hierarchical Pn-Pos model over each tree metric→ diameter diversity

over each tree metric (r-hst)

Optimize each Hierarchical Pn-Pos model using proposed move

making algorithm

Fuse solutions or choose the one with minimum energy

6Fakcharoenphol et al., In STOC 2003.

Puneet K. Dokania 16

Page 46: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Comparison

Comparison

Co-oc7:

Clique potentials→Monotonic

Very fast optimization algorithm

No theoretical guarantees

SoSPD8:

Clique potentials→ Arbitrary→ Upperbound as SoS functions

Slow. Practically, can not go beyond the clique of size 9

Loose multiplicative bound

Parsimonious Labeling9:

Clique potentials→ Diversities

Very fast. We experimented with cliques of size ≈ 1200.

Can be parallelized over the trees and over the levels.

Very tight multiplicative bound.

7Ladicky, Russell, Kohli, and Torr, ECCV 2010.

8Fix, Wang, and Zabih, CVPR 2014.

9Dokania and Kumar, ICCV 2015.

Puneet K. Dokania 17

Page 47: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Comparison

Comparison

Co-oc7:

Clique potentials→Monotonic

Very fast optimization algorithm

No theoretical guarantees

SoSPD8:

Clique potentials→ Arbitrary→ Upperbound as SoS functions

Slow. Practically, can not go beyond the clique of size 9

Loose multiplicative bound

Parsimonious Labeling9:

Clique potentials→ Diversities

Very fast. We experimented with cliques of size ≈ 1200.

Can be parallelized over the trees and over the levels.

Very tight multiplicative bound.

7Ladicky, Russell, Kohli, and Torr, ECCV 2010.

8Fix, Wang, and Zabih, CVPR 2014.

9Dokania and Kumar, ICCV 2015.

Puneet K. Dokania 17

Page 48: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Comparison

Comparison

Co-oc7:

Clique potentials→Monotonic

Very fast optimization algorithm

No theoretical guarantees

SoSPD8:

Clique potentials→ Arbitrary→ Upperbound as SoS functions

Slow. Practically, can not go beyond the clique of size 9

Loose multiplicative bound

Parsimonious Labeling9:

Clique potentials→ Diversities

Very fast. We experimented with cliques of size ≈ 1200.

Can be parallelized over the trees and over the levels.

Very tight multiplicative bound.

7Ladicky, Russell, Kohli, and Torr, ECCV 2010.

8Fix, Wang, and Zabih, CVPR 2014.

9Dokania and Kumar, ICCV 2015.

Puneet K. Dokania 17

Page 49: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Experimental Results

Experimental Seing

Energy Function:

E(y) =∑i∈V

θi(yi) +∑c∈C

wcδ(Γ(yc))︸ ︷︷ ︸Diversity

(3)

Clique Potential: Diameter diversity over truncated Linear Metric:

θi,j(la, lb) = λmin(|la − lb|,M),∀la, lb ∈ L

Cliques: Superpixels generate using Mean Shi.

Clique Weights:

wc = exp(−ρ(yc)

σ2

)where, ρ(yc) is the variance of intensities of pixels in clique yc .

Puneet K. Dokania 18

Page 50: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Experimental Results

Stereo Matching Results – Visually

(a) Ground Truth (b) Our

(c) α−Exp (d) Co-oc

Puneet K. Dokania 19

Page 51: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Experimental Results

Stereo Matching Results – Energy and Time

(a) Our

(E = 1.4× 106, 773 sec)(b) Co-oc

(E = 2.1× 106, 306 sec)

Puneet K. Dokania 20

Page 52: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Experimental Results

Image denoising and Inpainting Results – Visually

(a) Original (b) Our

(c) α−Exp (d) Co-oc

Puneet K. Dokania 21

Page 53: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Parsimonious Labeling Experimental Results

Image denoising and Inpainting Results – Energy and Time

(a) Our

(E = 1.2× 107, 1964 sec)(b) Co-occ

(E = 1.4× 107, 358 sec)

Puneet K. Dokania 22

Page 54: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information

Presentation Outline

1 Thesis Overview

2 Parsimonious Labeling

3 Learning to Rank Using High-Order Information

4 Regularization Path for SSVM

5 Future Work

6 Publications

Puneet K. Dokania 23

Page 55: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information AP-SVM

Ranking?

Get the feature vector φ(xi)

Learn wSort using si(w) = w>φ(xi)

SVM→ Optimizes accuracy

Accuracy 6= Average Precision

Puneet K. Dokania 24

Page 56: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information AP-SVM

Ranking?

Get the feature vector φ(xi)

Learn wSort using si(w) = w>φ(xi)

SVM→ Optimizes accuracy

Accuracy 6= Average Precision

Puneet K. Dokania 24

Page 57: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information AP-SVM

Ranking?

Get the feature vector φ(xi)

Learn wSort using si(w) = w>φ(xi)

SVM→ Optimizes accuracy

Accuracy 6= Average Precision

Puneet K. Dokania 24

Page 58: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information AP-SVM

Ranking?

Get the feature vector φ(xi)

Learn wSort using si(w) = w>φ(xi)

SVM→ Optimizes accuracy

Accuracy 6= Average Precision

Puneet K. Dokania 24

Page 59: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information AP-SVM

AP-SVM10

: Problem Formulation

Single input x, Positive Set P , Negative Set Nφ(xi),∀i ∈ P , φ(xj),∀j ∈ NRank Matrix

Rij =

+1, if i is beer ranked than j

−1, if j is beer ranked than i

Define Joint Score:

S(x,R;w) =1

|P||N |∑i∈P

∑j∈N

Rij(si(w)− sj(w))

Encodes Ranking

10Yue et al., A support vector method for optimizing average precision, 2007.

Puneet K. Dokania 25

Page 60: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information AP-SVM

AP-SVM10

: Problem Formulation

Single input x, Positive Set P , Negative Set Nφ(xi), ∀i ∈ P , φ(xj),∀j ∈ N

Rank Matrix

Rij =

+1, if i is beer ranked than j

−1, if j is beer ranked than i

Define Joint Score:

S(x,R;w) =1

|P||N |∑i∈P

∑j∈N

Rij(si(w)− sj(w))

Encodes Ranking

10Yue et al., A support vector method for optimizing average precision, 2007.

Puneet K. Dokania 25

Page 61: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information AP-SVM

AP-SVM10

: Problem Formulation

Single input x, Positive Set P , Negative Set Nφ(xi), ∀i ∈ P , φ(xj),∀j ∈ NRank Matrix

Rij =

+1, if i is beer ranked than j

−1, if j is beer ranked than i

Define Joint Score:

S(x,R;w) =1

|P||N |∑i∈P

∑j∈N

Rij(si(w)− sj(w))

Encodes Ranking

10Yue et al., A support vector method for optimizing average precision, 2007.

Puneet K. Dokania 25

Page 62: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information AP-SVM

AP-SVM: Objective Function

Loss function ∆(R,R∗) = 1− AP(R,R∗)Objective Function

min

w, ξ

λ

2

‖w‖2 + ξ (4)

s.t. S(x,R∗;w) ≥ S(x,R;w) + ∆(R,R∗)− ξ, ∀R. (5)

Loss augmented inference: R = argmaxRS(x, R;w) + ∆(R,R∗),greedy algorithm O(|P||N |) by Yue et.al.

Puneet K. Dokania 26

Page 63: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information AP-SVM

AP-SVM: Joint Score

Joint Score:

S(x,R;w) =1

|P||N |∑i∈P

∑j∈N

Rij(si(w)− sj(w))

Encodes Ranking

Sample Score:

si(w) = w>φ(xi)

No High-Order Information

Puneet K. Dokania 27

Page 64: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information High-Order Information

Why High-Order Information?

Puneet K. Dokania 28

Page 65: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information High-Order Information

Why High-Order Information?

Puneet K. Dokania 28

Page 66: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information High-Order Information

Why High-Order Information?

Puneet K. Dokania 28

Page 67: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information High-Order Information

Encoding High-Order Information

Puneet K. Dokania 29

Page 68: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information High-Order Information

Encoding High-Order Information

Puneet K. Dokania 29

Page 69: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information High-Order Information

Encoding High-Order Information

Define Joint Feature Map (encodes the

structure)

Φ(x, y) =

( ∑i Φ1(xi, yi)∑

i,j Φ2(xi, yi, xj, yj)

)

Φ1 - first-order information

Φ2 - high-order information

Joint labeling: y ∈ −1,+1n

Define Score S(x, y;w) = w>Φ(x, y)

Puneet K. Dokania 30

Page 70: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information High-Order Information

Encoding High-Order Information

Define Joint Feature Map (encodes the

structure)

Φ(x, y) =

( ∑i Φ1(xi, yi)∑

i,j Φ2(xi, yi, xj, yj)

)

Φ1 - first-order information

Φ2 - high-order information

Joint labeling: y ∈ −1,+1n

Define Score S(x, y;w) = w>Φ(x, y)

Puneet K. Dokania 30

Page 71: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information High-Order Information

Joint Score: Closer look

w>Φ(x, y) =

(w1

w2

)>( ∑i Φ1(xi, yi)∑

i,j Φ2(xi, yi, xj, yj)

)=

∑i

w>1

Φ1(xi, yi) +∑i,j

w>2

Φ2(xi, yi, xj, yj)︸ ︷︷ ︸Encodes High-Order Information

(6)

Single score for the entire dataset→ Ranking?

Puneet K. Dokania 31

Page 72: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information High-Order Information

Joint Score: Closer look

w>Φ(x, y) =

(w1

w2

)>( ∑i Φ1(xi, yi)∑

i,j Φ2(xi, yi, xj, yj)

)=

∑i

w>1

Φ1(xi, yi) +∑i,j

w>2

Φ2(xi, yi, xj, yj)︸ ︷︷ ︸Encodes High-Order Information

(6)

Single score for the entire dataset→ Ranking?

Puneet K. Dokania 31

Page 73: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information High-Order Information

Ranking Using Max-Marginals

We propose to use dierence of max-marginals

s(xi;w) = m+i (w)−m−i (w), where, m+

i (w) is the max-marginal score

such that sample xi takes label of +1.

m+i (w) = argmaxy,yi=+1w>Φ(x, y)

Dynamic Graph Cuts11

— Very Eicient

11Kohli et al., In PAMI 2007.

Puneet K. Dokania 32

Page 74: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information High-Order Information

Ranking Using Max-Marginals

We propose to use dierence of max-marginals

s(xi;w) = m+i (w)−m−i (w), where, m+

i (w) is the max-marginal score

such that sample xi takes label of +1.

m+i (w) = argmaxy,yi=+1w>Φ(x, y)

Dynamic Graph Cuts11

— Very Eicient

11Kohli et al., In PAMI 2007.

Puneet K. Dokania 32

Page 75: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information High-Order AP-SVM

HOAP-SVM: Score

Score that can encode ranking and high-order information

Joint Score for the given ranking

S(x,R;w) =1

|P||N |∑i∈P

∑j∈N

Rij(si(w)− sj(w))

Encodes Ranking

Sample score si as dierence of max-marginals

si(w) = m+i (w)−m−i (w)

Encodes High-Order Information

Puneet K. Dokania 33

Page 76: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information High-Order AP-SVM

HOAP-SVM: Score

Score that can encode ranking and high-order information

Joint Score for the given ranking

S(x,R;w) =1

|P||N |∑i∈P

∑j∈N

Rij(si(w)− sj(w))

Encodes Ranking

Sample score si as dierence of max-marginals

si(w) = m+i (w)−m−i (w)

Encodes High-Order Information

Puneet K. Dokania 33

Page 77: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information High-Order AP-SVM

HOAP-SVM: Score

Score that can encode ranking and high-order information

Joint Score for the given ranking

S(x,R;w) =1

|P||N |∑i∈P

∑j∈N

Rij(si(w)− sj(w))

Encodes Ranking

Sample score si as dierence of max-marginals

si(w) = m+i (w)−m−i (w)

Encodes High-Order Information

Puneet K. Dokania 33

Page 78: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information High-Order AP-SVM

HOAP-SVM: Objective Function

Objective Function

min

w, ξ

λ

2

‖w‖2 + ξ (7)

s.t. S(x,R∗;w) ≥ S(x,R;w) + ∆(R,R∗)− ξ, ∀R, (8)

w2 ≤ 0, ξ ≥ 0.

Each max-marginal is a convex function (max over aine functions)

m+i (w) = argmaxy,yi=+1w>Φ(x, y)

The objective function is a dierence of convex program

Puneet K. Dokania 34

Page 79: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information High-Order AP-SVM

HOAP-SVM: Objective Function

Objective Function

min

w, ξ

λ

2

‖w‖2 + ξ (7)

s.t. S(x,R∗;w) ≥ S(x,R;w) + ∆(R,R∗)− ξ, ∀R, (8)

w2 ≤ 0, ξ ≥ 0.

Each max-marginal is a convex function (max over aine functions)

m+i (w) = argmaxy,yi=+1w>Φ(x, y)

The objective function is a dierence of convex program

Puneet K. Dokania 34

Page 80: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information High-Order AP-SVM

HOAP-SVM: Objective Function

Objective Function

min

w, ξ

λ

2

‖w‖2 + ξ (7)

s.t. S(x,R∗;w) ≥ S(x,R;w) + ∆(R,R∗)− ξ, ∀R, (8)

w2 ≤ 0, ξ ≥ 0.

Each max-marginal is a convex function (max over aine functions)

m+i (w) = argmaxy,yi=+1w>Φ(x, y)

The objective function is a dierence of convex program

Puneet K. Dokania 34

Page 81: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information High-Order AP-SVM

CCCP12

Dierence of convex functions can be optimized using CCCP algorithm

12Yuille et al., The concave-convex procedure, 2003.

Puneet K. Dokania 35

Page 82: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information High-Order AP-SVM

CCCP12

Dierence of convex functions can be optimized using CCCP algorithm

12Yuille et al., The concave-convex procedure, 2003.

Puneet K. Dokania 35

Page 83: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information High-Order AP-SVM

CCCP12

Dierence of convex functions can be optimized using CCCP algorithm

12Yuille et al., The concave-convex procedure, 2003.

Puneet K. Dokania 35

Page 84: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information High-Order AP-SVM

CCCP12

Dierence of convex functions can be optimized using CCCP algorithm

12Yuille et al., The concave-convex procedure, 2003.

Puneet K. Dokania 35

Page 85: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information Results

Action Recognition

PASCAL VOC 2011 Dataset

10 Action Classes

Unary Feature - POSELET and GIST concatenated

High-Order Feature -POSELET

High-Order Information

Hypothesis: Persons in the same image are more likely to perform same

action

Connected bounding boxes coming from the same image

Puneet K. Dokania 36

Page 86: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information Results

PASCAL VOC Results - Average ap over all 10 action

classes

Method Trainval Test

svm 54.7/+4.2 48.82/+4.93

ap-svm 56.2/+2.7 51.42/+2.33

hoap-svm 58.9 53.75

Puneet K. Dokania 37

Page 87: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Learning to Rank Using High-Order Information Results

Visualization - Reading top 4

Puneet K. Dokania 38

Page 88: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM

Presentation Outline

1 Thesis Overview

2 Parsimonious Labeling

3 Learning to Rank Using High-Order Information

4 Regularization Path for SSVM

5 Future Work

6 Publications

Puneet K. Dokania 39

Page 89: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Definition and Motivation

Regularization Path: What and Why

Optimize SSVM objective function

min

w, ξ

λ

2

‖w‖2 +1

n

n∑i=1

ξi

s.t. set of constraints

λ→ important for good generalization→ cross validate

λ ∈ [0,∞]→ cross validation over subset→ poor generalization

ε-optimal regularization path algorithm

Puneet K. Dokania 40

Page 90: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Definition and Motivation

Regularization Path: What and Why

Optimize SSVM objective function

min

w, ξ

λ

2

‖w‖2 +1

n

n∑i=1

ξi

s.t. set of constraints

λ→ important for good generalization→ cross validate

λ ∈ [0,∞]→ cross validation over subset→ poor generalization

ε-optimal regularization path algorithm

Puneet K. Dokania 40

Page 91: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Definition and Motivation

Regularization Path: What and Why

Optimize SSVM objective function

min

w, ξ

λ

2

‖w‖2 +1

n

n∑i=1

ξi

s.t. set of constraints

λ→ important for good generalization→ cross validate

λ ∈ [0,∞]→ cross validation over subset→ poor generalization

ε-optimal regularization path algorithm

Puneet K. Dokania 40

Page 92: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Definition and Motivation

Regularization Path: What and Why

Optimize SSVM objective function

min

w, ξ

λ

2

‖w‖2 +1

n

n∑i=1

ξi

s.t. set of constraints

λ→ important for good generalization→ cross validate

λ ∈ [0,∞]→ cross validation over subset→ poor generalization

ε-optimal regularization path algorithm

Puneet K. Dokania 40

Page 93: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Definition and Motivation

Dual Objective and Duality Gap

SSVM dual objective function

min

αf (α)→ smooth convex

s.t.∑y∈Yi

αi(y) = 1,∀i ∈ [n],

αi(y) ≥ 0, ∀i ∈ [n],∀y ∈ Yi.

where, α = (α1, · · · , αn) ∈ R|Y1| × · · ·R|Yn|.

Duality Gap

g(α;λ) =1

n

∑i

(max

y∈YiHi(y;w)−

∑y∈Yi

αi(y)Hi(y;w))

where, Hi(y;w) is the hinge loss.

Puneet K. Dokania 41

Page 94: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Definition and Motivation

Dual Objective and Duality Gap

SSVM dual objective function

min

αf (α)→ smooth convex

s.t.∑y∈Yi

αi(y) = 1,∀i ∈ [n],

αi(y) ≥ 0, ∀i ∈ [n],∀y ∈ Yi.

where, α = (α1, · · · , αn) ∈ R|Y1| × · · ·R|Yn|.Duality Gap

g(α;λ) =1

n

∑i

(max

y∈YiHi(y;w)−

∑y∈Yi

αi(y)Hi(y;w))

where, Hi(y;w) is the hinge loss.

Puneet K. Dokania 41

Page 95: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions

Key Idea: ε-Optimal Regularization Path

Puneet K. Dokania 42

Page 96: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions

Key Idea: ε-Optimal Regularization Path

Puneet K. Dokania 42

Page 97: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions

Key Idea: ε-Optimal Regularization Path

Puneet K. Dokania 42

Page 98: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions

Key Idea: ε-Optimal Regularization Path

Puneet K. Dokania 42

Page 99: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions

Key Idea: ε-Optimal Regularization Path

Puneet K. Dokania 42

Page 100: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions

Challenge 1: How do we start?

Let Yi = argmaxy∈Yi ∆(y, yi) be the loss-maximizer and yi ∈ Yi,∀i.Let Ψ = 1

n

∑i Ψi(yi), where Ψi(y) = Φ(xi, yi)− Φ(xi, y).

Then, wk = Ψλ is guaranteed to be ε1 optimal for any λ satisfying the

condition:

λ ≥

∥∥∥Ψ∥∥∥2

+ 1

n

∑i max

y∈Yi

(− Ψ>Ψ(y)

)︸ ︷︷ ︸

Inference

ε1

(9)

Puneet K. Dokania 43

Page 101: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions

Challenge 1: How do we start?

Let Yi = argmaxy∈Yi ∆(y, yi) be the loss-maximizer and yi ∈ Yi, ∀i.

Let Ψ = 1

n

∑i Ψi(yi), where Ψi(y) = Φ(xi, yi)− Φ(xi, y).

Then, wk = Ψλ is guaranteed to be ε1 optimal for any λ satisfying the

condition:

λ ≥

∥∥∥Ψ∥∥∥2

+ 1

n

∑i max

y∈Yi

(− Ψ>Ψ(y)

)︸ ︷︷ ︸

Inference

ε1

(9)

Puneet K. Dokania 43

Page 102: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions

Challenge 1: How do we start?

Let Yi = argmaxy∈Yi ∆(y, yi) be the loss-maximizer and yi ∈ Yi, ∀i.Let Ψ = 1

n

∑i Ψi(yi), where Ψi(y) = Φ(xi, yi)− Φ(xi, y).

Then, wk = Ψλ is guaranteed to be ε1 optimal for any λ satisfying the

condition:

λ ≥

∥∥∥Ψ∥∥∥2

+ 1

n

∑i max

y∈Yi

(− Ψ>Ψ(y)

)︸ ︷︷ ︸

Inference

ε1

(9)

Puneet K. Dokania 43

Page 103: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions

Challenge 1: How do we start?

Let Yi = argmaxy∈Yi ∆(y, yi) be the loss-maximizer and yi ∈ Yi, ∀i.Let Ψ = 1

n

∑i Ψi(yi), where Ψi(y) = Φ(xi, yi)− Φ(xi, y).

Then, wk = Ψλ is guaranteed to be ε1 optimal for any λ satisfying the

condition:

λ ≥

∥∥∥Ψ∥∥∥2

+ 1

n

∑i max

y∈Yi

(− Ψ>Ψ(y)

)︸ ︷︷ ︸

Inference

ε1

(9)

Puneet K. Dokania 43

Page 104: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions

Challenge 2: How to find the breakpoints?

Let λk+1 = ηλk , 0 ≤ η ≤ 1.

wk → εopt , for all λk+1 obtained using η satisfying the condition:.

1− ε− g(αk ;λk)

Ω(αk , λk)≤ η ≤ 1 (10)

where, Ω(αk , λk) := `αk − λkw>k wk

Puneet K. Dokania 44

Page 105: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions

Challenge 2: How to find the breakpoints?

Let λk+1 = ηλk , 0 ≤ η ≤ 1.

wk → εopt , for all λk+1 obtained using η satisfying the condition:.

1− ε− g(αk ;λk)

Ω(αk , λk)≤ η ≤ 1 (10)

where, Ω(αk , λk) := `αk − λkw>k wk

Puneet K. Dokania 44

Page 106: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions

Challenge 2: How to find the breakpoints?

Let λk+1 = ηλk , 0 ≤ η ≤ 1.

wk → εopt , for all λk+1 obtained using η satisfying the condition:.

1− ε− g(αk ;λk)

Ω(αk , λk)≤ η ≤ 1 (10)

where, Ω(αk , λk) := `αk − λkw>k wk

Puneet K. Dokania 44

Page 107: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions

Challenge 2: Proof Sketch

Keeping wk constant – from kkt condition

wk =1

n

∑i∈[n], y∈Yi

αki (y)

λkΨ(xi, y).

Therefore, using

αk+1

i (y)

λk+1

=αki (y)

λk, ∀y 6= yi;

∑y∈Yi

αi(y) = 1,∀i ∈ [n]; λk+1 = ηλk

New duality gap

g(αk+1;λk+1) = g(αk ;λk)︸ ︷︷ ︸Old gap

+(1− η

)Ω(αk , λk)

≤ ε

Puneet K. Dokania 45

Page 108: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions

Challenge 2: Proof Sketch

Keeping wk constant – from kkt condition

wk =1

n

∑i∈[n], y∈Yi

αki (y)

λkΨ(xi, y).

Therefore, using

αk+1

i (y)

λk+1

=αki (y)

λk, ∀y 6= yi;

∑y∈Yi

αi(y) = 1,∀i ∈ [n]; λk+1 = ηλk

New duality gap

g(αk+1;λk+1) = g(αk ;λk)︸ ︷︷ ︸Old gap

+(1− η

)Ω(αk , λk)

≤ ε

Puneet K. Dokania 45

Page 109: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions

Challenge 2: Proof Sketch

Keeping wk constant – from kkt condition

wk =1

n

∑i∈[n], y∈Yi

αki (y)

λkΨ(xi, y).

Therefore, using

αk+1

i (y)

λk+1

=αki (y)

λk, ∀y 6= yi;

∑y∈Yi

αi(y) = 1,∀i ∈ [n]; λk+1 = ηλk

New duality gap

g(αk+1;λk+1) = g(αk ;λk)︸ ︷︷ ︸Old gap

+(1− η

)Ω(αk , λk)

≤ ε

Puneet K. Dokania 45

Page 110: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions

Challenge 3: How to optimize eiciently?

Notice that, wk is already ε-optimal at λk+1

Warm starting with wk requires us to reduce the duality gap only by

(ε− ε1)→ very fast convergence

We use Block-Coordinate Frank-Wolfe algorithm13

for the

optimization.

Lagrange duality gap is the by product

13Lacoste-Julien et al., In ICML 2013.

Puneet K. Dokania 46

Page 111: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions

Eects of ε1

Decrease ε1:

(ε− ε1) increases — More passes through the data to get (ε1)opt solution.

η decreases — big jumps — number of breakpoints decreases (see below)

λk+1 = ηλk ; 1− ε− g(αk ;λk)

Ω(αk , λk)≤ η ≤ 1

Increase ε1 — Similar arguments

Puneet K. Dokania 47

Page 112: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions

Eects of ε1

Decrease ε1:

(ε− ε1) increases — More passes through the data to get (ε1)opt solution.

η decreases — big jumps — number of breakpoints decreases (see below)

λk+1 = ηλk ; 1− ε− g(αk ;λk)

Ω(αk , λk)≤ η ≤ 1

Increase ε1 — Similar arguments

Puneet K. Dokania 47

Page 113: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Experiments and Analysis

Dataset and bcfw Variants

OCR dataset14

with 6251 train and 626 test samples.

ε = 0.1

20 dierent values of λ equally spaced between [10−4, 10

3]

BCFW variants

BCFW-HEU-G: Heuristic convergence with gap based sampling

BCFW-STD-G: Exact convergence with gap based sampling

RP-BCFW-HEU-G: Regularization Path with BCFW-HEU-G.

14Taskar et al., Max-margin Markov networks, NIPS 2003.

Puneet K. Dokania 48

Page 114: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Experiments and Analysis

Dataset and bcfw Variants

OCR dataset14

with 6251 train and 626 test samples.

ε = 0.1

20 dierent values of λ equally spaced between [10−4, 10

3]

BCFW variants

BCFW-HEU-G: Heuristic convergence with gap based sampling

BCFW-STD-G: Exact convergence with gap based sampling

RP-BCFW-HEU-G: Regularization Path with BCFW-HEU-G.

14Taskar et al., Max-margin Markov networks, NIPS 2003.

Puneet K. Dokania 48

Page 115: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Experiments and Analysis

Dataset and bcfw Variants

OCR dataset14

with 6251 train and 626 test samples.

ε = 0.1

20 dierent values of λ equally spaced between [10−4, 10

3]

BCFW variants

BCFW-HEU-G: Heuristic convergence with gap based sampling

BCFW-STD-G: Exact convergence with gap based sampling

RP-BCFW-HEU-G: Regularization Path with BCFW-HEU-G.

14Taskar et al., Max-margin Markov networks, NIPS 2003.

Puneet K. Dokania 48

Page 116: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Experiments and Analysis

Eect of ε1 for ε = 0.1

Number of breakpoints in the regularization path

ε1 RP-BCFW-HEU-G RP-BCFW-STD-G0.01 142 133

0.05 225 153

0.09 1060 349

Number of passes through the data for optimization

ε1 RP-BCFW-HEU-G RP-BCFW-STD-G BCFW-STD-G0.01 2711.946 4405.881 1138.872

0.05 1301.869 2120.969 1138.872

0.09 1076.005 2100.304 1138.872

Puneet K. Dokania 49

Page 117: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Experiments and Analysis

Eect of ε1 for ε = 0.1

Number of breakpoints in the regularization path

ε1 RP-BCFW-HEU-G RP-BCFW-STD-G0.01 142 133

0.05 225 153

0.09 1060 349

Number of passes through the data for optimization

ε1 RP-BCFW-HEU-G RP-BCFW-STD-G BCFW-STD-G0.01 2711.946 4405.881 1138.872

0.05 1301.869 2120.969 1138.872

0.09 1076.005 2100.304 1138.872

Puneet K. Dokania 49

Page 118: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Experiments and Analysis

Duality gap for ε1 = 0.01

Puneet K. Dokania 50

Page 119: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Experiments and Analysis

Duality gap for ε1 = 0.09

Puneet K. Dokania 51

Page 120: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Experiments and Analysis

Test loss for ε1 = 0.01

Puneet K. Dokania 52

Page 121: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Regularization Path for SSVM Experiments and Analysis

Test loss for ε1 = 0.09

Puneet K. Dokania 53

Page 122: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Future Work

Presentation Outline

1 Thesis Overview

2 Parsimonious Labeling

3 Learning to Rank Using High-Order Information

4 Regularization Path for SSVM

5 Future Work

6 Publications

Puneet K. Dokania 54

Page 123: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Future Work

Possible future directions…

High-Order

Parsimonious labeling for semantic labels

SSVM

Latent HOAP-SVM

Discovering label dependence structure

Latent SSVM: Interaction between latent variables?

Regularization path

min

w

λ

2

‖w‖2 + L(x, y;w)

Puneet K. Dokania 55

Page 124: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Publications

Presentation Outline

1 Thesis Overview

2 Parsimonious Labeling

3 Learning to Rank Using High-Order Information

4 Regularization Path for SSVM

5 Future Work

6 Publications

Puneet K. Dokania 56

Page 125: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Publications

List of publications

1 Discriminative parameter estimation for random walks segmentation, In

MICCAI 2013.

2 Learning to Rank using High-Order Information, In ECCV 2014.

3 Parsimonious Labeling, In ICCV 2015.

4 Minding the Gaps for Block Frank-Wolfe Optimization of StructuredSVM, In ICML 2016.

5 Rounding-based Combinatorial Algorithms for Metric Labeling, In JMLR

2016.

6 Deformable Registration through Learning of Context-Specific MetricAggregation, Under submission, ECCV 2016.

7 Partial Linearization based Optimization for Multi-class SVM, Under

submission, ECCV 2016.

Puneet K. Dokania 57

Page 126: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling

Thank you

Puneet K. Dokania 58