self-paced learning for semantic segmentation m. pawan kumar
TRANSCRIPT
![Page 1: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/1.jpg)
Self-Paced Learning forSemantic Segmentation
M. Pawan Kumar
![Page 2: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/2.jpg)
![Page 3: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/3.jpg)
Self-Paced Learning forLatent Structural SVM
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Daphne Koller
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Benjamin Packer
M. Pawan Kumar
![Page 4: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/4.jpg)
AimTo learn accurate parameters for latent structural SVM
Input x
Output y Y
“Deer”
Hidden Variableh H
Y = {“Bison”, “Deer”, ”Elephant”, “Giraffe”, “Llama”, “Rhino” }
![Page 5: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/5.jpg)
AimTo learn accurate parameters for latent structural SVM
Feature (x,y,h)(HOG, BoW)
(y*,h*) = maxyY,hH wT(x,y,h)
Parameters w
![Page 6: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/6.jpg)
Motivation
Real Numbers
Imaginary Numbers
eiπ+1 = 0
Math is forlosers !!
FAILURE … BAD LOCAL MINIMUM
![Page 7: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/7.jpg)
Motivation
Real Numbers
Imaginary Numbers
eiπ+1 = 0
Euler wasa Genius!!
SUCCESS … GOOD LOCAL MINIMUM
![Page 8: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/8.jpg)
Motivation
Start with “easy” examples, then consider “hard” ones
Easy vs. Hard
Expensive
Easy for human Easy for machine
Simultaneously estimate easiness and parametersEasiness is property of data sets, not single instances
![Page 9: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/9.jpg)
Outline
• Latent Structural SVM
• Concave-Convex Procedure
• Self-Paced Learning
• Experiments
![Page 10: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/10.jpg)
Latent Structural SVM
Training samples xi
Ground-truth label yi
Loss Function(yi, yi(w), hi(w))
Felzenszwalb et al, 2008, Yu and Joachims, 2009
![Page 11: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/11.jpg)
Latent Structural SVM
(yi(w),hi(w)) = maxyY,hH wT(x,y,h)
min ||w||2 + C∑i(yi, yi(w), hi(w))
Non-convex Objective
Minimize an upper bound
![Page 12: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/12.jpg)
Latent Structural SVM
min ||w||2 + C∑i i
maxhiwT(xi,yi,hi) - wT(xi,y,h)
≥ (yi, y, h) - i
Still non-convex Difference of convex
CCCP Algorithm - converges to a local minimum
(yi(w),hi(w)) = maxyY,hH wT(x,y,h)
![Page 13: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/13.jpg)
Outline
• Latent Structural SVM
• Concave-Convex Procedure
• Self-Paced Learning
• Experiments
![Page 14: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/14.jpg)
Concave-Convex Procedure
Start with an initial estimate w0
Update
Update wt+1 by solving a convex problem
min ||w||2 + C∑i i
wT(xi,yi,hi) - wT(xi,y,h)≥ (yi, y, h) - i
hi = maxhH wtT(xi,yi,h)
![Page 15: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/15.jpg)
Concave-Convex Procedure
Looks at all samples simultaneously
“Hard” samples will cause confusion
Start with “easy” samples, then consider “hard” ones
![Page 16: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/16.jpg)
Outline
• Latent Structural SVM
• Concave-Convex Procedure
• Self-Paced Learning
• Experiments
![Page 17: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/17.jpg)
Self-Paced Learning
REMINDER
Simultaneously estimate easiness and parametersEasiness is property of data sets, not single instances
![Page 18: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/18.jpg)
Self-Paced Learning
Start with an initial estimate w0
Update
Update wt+1 by solving a convex problem
min ||w||2 + C∑i i
wT(xi,yi,hi) - wT(xi,y,h)≥ (yi, y, h) - i
hi = maxhH wtT(xi,yi,h)
![Page 19: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/19.jpg)
Self-Paced Learning
min ||w||2 + C∑i i
wT(xi,yi,hi) - wT(xi,y,h)≥ (yi, y, h) - i
![Page 20: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/20.jpg)
Self-Paced Learning
min ||w||2 + C∑i vii
wT(xi,yi,hi) - wT(xi,y,h)≥ (yi, y, h) - i
vi {0,1}
Trivial Solution
![Page 21: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/21.jpg)
Self-Paced Learning
vi {0,1}
Large K Medium K Small K
min ||w||2 + C∑i vii - ∑ivi/K
wT(xi,yi,hi) - wT(xi,y,h)≥ (yi, y, h) - i
![Page 22: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/22.jpg)
Self-Paced Learning
vi [0,1]
min ||w||2 + C∑i vii - ∑ivi/K
wT(xi,yi,hi) - wT(xi,y,h)≥ (yi, y, h) - i
Large K Medium K Small K
BiconvexProblem
AlternatingConvex Search
![Page 23: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/23.jpg)
Self-Paced LearningStart with an initial estimate w0
Update
Update wt+1 by solving a convex problem
min ||w||2 + C∑i vii - ∑i vi/K
wT(xi,yi,hi) - wT(xi,y,h)≥ (yi, y, h) - i
hi = maxhH wtT(xi,yi,h)
Decrease K K/
![Page 24: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/24.jpg)
Outline
• Latent Structural SVM
• Concave-Convex Procedure
• Self-Paced Learning
• Experiments
![Page 25: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/25.jpg)
Object Detection
Feature (x,y,h) - HOG
Input x - Image
Output y Y
Latent h - Box
- 0/1 Loss
Y = {“Bison”, “Deer”, ”Elephant”, “Giraffe”, “Llama”, “Rhino” }
![Page 26: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/26.jpg)
Object Detection
271 images, 6 classes
90/10 train/test split
4 folds
Mammals Dataset
![Page 27: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/27.jpg)
Object DetectionCCCP Self-Paced
![Page 28: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/28.jpg)
Object DetectionCCCP Self-Paced
![Page 29: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/29.jpg)
Object DetectionCCCP Self-Paced
![Page 30: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/30.jpg)
Object DetectionCCCP Self-Paced
![Page 31: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/31.jpg)
Objective value Test error
Object Detection
4
4.2
4.4
4.6
4.8
5
Fold1 Fold2 Fold3 Fold4
CCCP
SPL
0
5
10
15
20
25
Fold1 Fold2 Fold3 Fold4
CCCP
SPL
![Page 32: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/32.jpg)
Handwritten Digit Recognition
Feature (x,y,h) - PCA + Projection
Input x - Image
Output y Y
Y = {0, 1, … , 9}
Latent h - Rotation
MNIST Dataset
- 0/1 Loss
![Page 33: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/33.jpg)
Handwritten Digit Recognition
- Significant Difference
C
C
C
SPL
![Page 34: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/34.jpg)
Handwritten Digit Recognition
- Significant Difference
C
C
C
SPL
![Page 35: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/35.jpg)
Handwritten Digit Recognition
- Significant Difference
C
C
C
SPL
![Page 36: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/36.jpg)
Handwritten Digit Recognition
- Significant Difference
C
C
C
SPL
![Page 37: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/37.jpg)
Motif Finding
Feature (x,y,h) - Ng and Cardie, ACL 2002
Input x - DNA Sequence
Output y Y
Y = {0, 1}
Latent h - Motif Location
- 0/1 Loss
![Page 38: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/38.jpg)
Motif Finding
40,000 sequences
50/50 train/test split
5 folds
UniProbe Dataset
![Page 39: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/39.jpg)
Motif FindingAverage Hamming Distance of Inferred Motifs
SPL SPL
SPLSPL
![Page 40: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/40.jpg)
Motif Finding
020406080
100120140160
Fold 1 Fold 2 Fold 3 Fold 4 Fold 5
CCCPCurr
Objective Value
SPL
![Page 41: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/41.jpg)
Motif Finding
0
10
20
30
40
50
Fold1
Fold2
Fold3
Fold4
Fold5
CCCPCurr
Test Error
SPL
![Page 42: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/42.jpg)
Noun Phrase Coreference
Feature (x,y,h) - Yu and Joachims, ICML 2009
Input x - Nouns Output y - Clustering
Latent h - Spanning Forest over Nouns
![Page 43: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/43.jpg)
Noun Phrase Coreference60 documents
50/50 train/test split 1 predefined fold
MUC6 Dataset
![Page 44: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/44.jpg)
Noun Phrase Coreference
- Significant Improvement
- Significant Decrement
MITRELoss
PairwiseLoss
![Page 45: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/45.jpg)
Noun Phrase Coreference
MITRELoss
PairwiseLoss
SPL
SPL
![Page 46: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/46.jpg)
Noun Phrase Coreference
MITRELoss
PairwiseLoss
SPL
SPL
![Page 47: Self-Paced Learning for Semantic Segmentation M. Pawan Kumar](https://reader033.vdocuments.site/reader033/viewer/2022061306/55147771550346b2598b45f2/html5/thumbnails/47.jpg)
Summary
• Automatic Self-Paced Learning
• Concave-Biconvex Procedure
• Generalization to other Latent models– Expectation-Maximization– E-step remains the same– M-step includes indicator variables vi
Kumar, Packer and Koller, NIPS 2010