online multiple classifier boosting for object tracking tae-kyun kim 1 thomas woodley 1 björn...
TRANSCRIPT
![Page 1: Online Multiple Classifier Boosting for Object Tracking Tae-Kyun Kim 1 Thomas Woodley 1 Björn Stenger 2 Roberto Cipolla 1 1 Dept. of Engineering, University](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649cf85503460f949c8fc9/html5/thumbnails/1.jpg)
Online Multiple Classifier Boosting for Object Tracking
Tae-Kyun Kim1 Thomas Woodley1 Björn Stenger2 Roberto Cipolla1
1Dept. of Engineering, University of Cambridge2Computer Vision Group, Toshiba Research Europe
![Page 2: Online Multiple Classifier Boosting for Object Tracking Tae-Kyun Kim 1 Thomas Woodley 1 Björn Stenger 2 Roberto Cipolla 1 1 Dept. of Engineering, University](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649cf85503460f949c8fc9/html5/thumbnails/2.jpg)
The Task: Object TrackingExample sequence 1
Target appearance changes due to changes in- pose - illumination- object deformation
Example sequence 2
![Page 3: Online Multiple Classifier Boosting for Object Tracking Tae-Kyun Kim 1 Thomas Woodley 1 Björn Stenger 2 Roberto Cipolla 1 1 Dept. of Engineering, University](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649cf85503460f949c8fc9/html5/thumbnails/3.jpg)
Learning Multi-Modal Representations
- Multi-view face detection [Rowley et al. 98, Schneiderman et al. 00, Jones Viola 03]- Multi-category detection, Sharing features [Torralba et al. 04]
Positive examples
Negative examples
![Page 4: Online Multiple Classifier Boosting for Object Tracking Tae-Kyun Kim 1 Thomas Woodley 1 Björn Stenger 2 Roberto Cipolla 1 1 Dept. of Engineering, University](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649cf85503460f949c8fc9/html5/thumbnails/4.jpg)
Joint Clustering and Training
K-means clustering
Face cluster 1
Face cluster 2
Positive examples Negative examplesFeature pool
[Kim and Cipolla 08, Babenko et al. 08]
![Page 5: Online Multiple Classifier Boosting for Object Tracking Tae-Kyun Kim 1 Thomas Woodley 1 Björn Stenger 2 Roberto Cipolla 1 1 Dept. of Engineering, University](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649cf85503460f949c8fc9/html5/thumbnails/5.jpg)
Given:
Set of n training samples with labels number of strong classifiers
Learn strong classifiers:
Combine classifier output with“Noisy OR” function
Map to probabilitieswith sigmoid function
MCBoost: Multiple Strong Classifier Boosting[Kim and Cipolla 08, Babenko et al. 08]
![Page 6: Online Multiple Classifier Boosting for Object Tracking Tae-Kyun Kim 1 Thomas Woodley 1 Björn Stenger 2 Roberto Cipolla 1 1 Dept. of Engineering, University](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649cf85503460f949c8fc9/html5/thumbnails/6.jpg)
• For given weights, find K weak-learners at t-th round of boosting to maximize
• Weak-learner weights found by a line search to maximize
where
• Sample weight update by AnyBoost method [Mason et al. 00]
MCBoost (continued)
![Page 7: Online Multiple Classifier Boosting for Object Tracking Tae-Kyun Kim 1 Thomas Woodley 1 Björn Stenger 2 Roberto Cipolla 1 1 Dept. of Engineering, University](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649cf85503460f949c8fc9/html5/thumbnails/7.jpg)
MCBoost: Toy Example 1
Input data MCBoost result (K=3)
![Page 8: Online Multiple Classifier Boosting for Object Tracking Tae-Kyun Kim 1 Thomas Woodley 1 Björn Stenger 2 Roberto Cipolla 1 1 Dept. of Engineering, University](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649cf85503460f949c8fc9/html5/thumbnails/8.jpg)
Toy Example 2
![Page 9: Online Multiple Classifier Boosting for Object Tracking Tae-Kyun Kim 1 Thomas Woodley 1 Björn Stenger 2 Roberto Cipolla 1 1 Dept. of Engineering, University](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649cf85503460f949c8fc9/html5/thumbnails/9.jpg)
Standard AdaBoost
![Page 10: Online Multiple Classifier Boosting for Object Tracking Tae-Kyun Kim 1 Thomas Woodley 1 Björn Stenger 2 Roberto Cipolla 1 1 Dept. of Engineering, University](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649cf85503460f949c8fc9/html5/thumbnails/10.jpg)
MCBoost [Kim and Cipolla 08]
![Page 11: Online Multiple Classifier Boosting for Object Tracking Tae-Kyun Kim 1 Thomas Woodley 1 Björn Stenger 2 Roberto Cipolla 1 1 Dept. of Engineering, University](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649cf85503460f949c8fc9/html5/thumbnails/11.jpg)
MC Boost with weighting function QMC Boost with weighting function QMCBQ
![Page 12: Online Multiple Classifier Boosting for Object Tracking Tae-Kyun Kim 1 Thomas Woodley 1 Björn Stenger 2 Roberto Cipolla 1 1 Dept. of Engineering, University](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649cf85503460f949c8fc9/html5/thumbnails/12.jpg)
Classifier Assignment
Make classifier assignment explicit using function
weight of strong classifier on sample
is updated at each round of boosting.
Here: K-component GMM in d-dim eigenspace, k-th mode is area of expertise of
![Page 13: Online Multiple Classifier Boosting for Object Tracking Tae-Kyun Kim 1 Thomas Woodley 1 Björn Stenger 2 Roberto Cipolla 1 1 Dept. of Engineering, University](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649cf85503460f949c8fc9/html5/thumbnails/13.jpg)
Joint Boosting and Clustering
MCBoost MCBQ
![Page 14: Online Multiple Classifier Boosting for Object Tracking Tae-Kyun Kim 1 Thomas Woodley 1 Björn Stenger 2 Roberto Cipolla 1 1 Dept. of Engineering, University](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649cf85503460f949c8fc9/html5/thumbnails/14.jpg)
Input: Data set , set of weak learnersOutput: Strong classifiers
for t=1,…,T // boosting roundsfor k=1,…,K // strong classifiers
Find weak learners and their weightsUpdate sample weights
endend
MCBQ Algorithm
Update sample weightsUpdate weighting function
Init with GMMInit weights to values of
, weighting function
![Page 15: Online Multiple Classifier Boosting for Object Tracking Tae-Kyun Kim 1 Thomas Woodley 1 Björn Stenger 2 Roberto Cipolla 1 1 Dept. of Engineering, University](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649cf85503460f949c8fc9/html5/thumbnails/15.jpg)
MCBQ for Object TrackingPrinciple: 1. (Short) supervised training phase
2. On-line updates
![Page 16: Online Multiple Classifier Boosting for Object Tracking Tae-Kyun Kim 1 Thomas Woodley 1 Björn Stenger 2 Roberto Cipolla 1 1 Dept. of Engineering, University](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649cf85503460f949c8fc9/html5/thumbnails/16.jpg)
Online Boosting
one sample
Init importance
Estimate errors
Select best weak classifier
Update weight
Estimate importance
Current strong classifier
[Oza, Russel 01, Grabner, Bischof 06]
Global classifier pool
Estimate errors
Select best weak classifier
Update weight
Estimate errors
Select best weak classifier
Update weight
Estimate importance
![Page 17: Online Multiple Classifier Boosting for Object Tracking Tae-Kyun Kim 1 Thomas Woodley 1 Björn Stenger 2 Roberto Cipolla 1 1 Dept. of Engineering, University](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649cf85503460f949c8fc9/html5/thumbnails/17.jpg)
Online MCBQClassifiers
Sample weight distribution
Selector Selector Selector
Update
Selector Selector Selector
Select weak classifiers, add to
Update weights, re-normalize
![Page 18: Online Multiple Classifier Boosting for Object Tracking Tae-Kyun Kim 1 Thomas Woodley 1 Björn Stenger 2 Roberto Cipolla 1 1 Dept. of Engineering, University](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649cf85503460f949c8fc9/html5/thumbnails/18.jpg)
Results
![Page 19: Online Multiple Classifier Boosting for Object Tracking Tae-Kyun Kim 1 Thomas Woodley 1 Björn Stenger 2 Roberto Cipolla 1 1 Dept. of Engineering, University](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649cf85503460f949c8fc9/html5/thumbnails/19.jpg)
Improved Pose Expertise
MCBoost
MCBQ
![Page 20: Online Multiple Classifier Boosting for Object Tracking Tae-Kyun Kim 1 Thomas Woodley 1 Björn Stenger 2 Roberto Cipolla 1 1 Dept. of Engineering, University](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649cf85503460f949c8fc9/html5/thumbnails/20.jpg)
Multi-pose Tracking with MCBQ
![Page 21: Online Multiple Classifier Boosting for Object Tracking Tae-Kyun Kim 1 Thomas Woodley 1 Björn Stenger 2 Roberto Cipolla 1 1 Dept. of Engineering, University](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649cf85503460f949c8fc9/html5/thumbnails/21.jpg)
Tracking Experiments
![Page 22: Online Multiple Classifier Boosting for Object Tracking Tae-Kyun Kim 1 Thomas Woodley 1 Björn Stenger 2 Roberto Cipolla 1 1 Dept. of Engineering, University](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649cf85503460f949c8fc9/html5/thumbnails/22.jpg)
Tracking “Cube” sequence
MCBQMILTrack SemiBoost
![Page 23: Online Multiple Classifier Boosting for Object Tracking Tae-Kyun Kim 1 Thomas Woodley 1 Björn Stenger 2 Roberto Cipolla 1 1 Dept. of Engineering, University](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649cf85503460f949c8fc9/html5/thumbnails/23.jpg)
Tracking Experiments
Tracking error
![Page 24: Online Multiple Classifier Boosting for Object Tracking Tae-Kyun Kim 1 Thomas Woodley 1 Björn Stenger 2 Roberto Cipolla 1 1 Dept. of Engineering, University](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649cf85503460f949c8fc9/html5/thumbnails/24.jpg)
Summary
Tracking: Build appearance model, then update online
No detector is required, i.e. not object specific.Handles rapid appearance changes.Simultaneous pose estimation and tracking is possible.
K is currently set by hand.Incorrect adaptation may still occur.
Extension of MCBoost to online settingExtension of MIL to multi-class
![Page 25: Online Multiple Classifier Boosting for Object Tracking Tae-Kyun Kim 1 Thomas Woodley 1 Björn Stenger 2 Roberto Cipolla 1 1 Dept. of Engineering, University](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649cf85503460f949c8fc9/html5/thumbnails/25.jpg)
Thank you