mean shift : a robust approach toward feature space analysis - bayesian background modeling...
TRANSCRIPT
![Page 1: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/1.jpg)
Mean Shift : A Robust Approach Toward Feature Space Analysis
- Bayesian Background Modeling
2009.10.9
23年 4月 19日 1
![Page 2: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/2.jpg)
23年 4月 19日 2
![Page 3: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/3.jpg)
Mean ShiftTheory and Applications
Yaron Ukrainitz & Bernard Sarel
23年 4月 19日 3
![Page 4: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/4.jpg)
Agenda
• Mean Shift Theory• What is Mean Shift ? • Density Estimation Methods• Nonparametric Kernel Density Estimation• Deriving the Mean Shift• Mean shift properties
• Applications• Clustering• Discontinuity Preserving Smoothing• Object Contour Detection• Segmentation• Object Tracking
23年 4月 19日 4
![Page 5: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/5.jpg)
Mean Shift Theory
23年 4月 19日 5
![Page 6: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/6.jpg)
Intuitive Description
Distribution of identical billiard balls
Region ofinterest
Center ofmass
Mean Shiftvector
Objective : Find the densest region23年 4月 19日 6
![Page 7: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/7.jpg)
Intuitive Description
Distribution of identical billiard balls
Region ofinterest
Center ofmass
Mean Shiftvector
Objective : Find the densest region23年 4月 19日 7
![Page 8: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/8.jpg)
Intuitive Description
Distribution of identical billiard balls
Region ofinterest
Center ofmass
Mean Shiftvector
Objective : Find the densest region23年 4月 19日 8
![Page 9: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/9.jpg)
Intuitive Description
Distribution of identical billiard balls
Region ofinterest
Center ofmass
Mean Shiftvector
Objective : Find the densest region23年 4月 19日 9
![Page 10: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/10.jpg)
Intuitive Description
Distribution of identical billiard balls
Region ofinterest
Center ofmass
Mean Shiftvector
Objective : Find the densest region23年 4月 19日 10
![Page 11: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/11.jpg)
Intuitive Description
Distribution of identical billiard balls
Region ofinterest
Center ofmass
Mean Shiftvector
Objective : Find the densest region23年 4月 19日 11
![Page 12: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/12.jpg)
Intuitive Description
Distribution of identical billiard balls
Region ofinterest
Center ofmass
Objective : Find the densest region23年 4月 19日 12
![Page 13: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/13.jpg)
What is Mean Shift ?
Non-parametricDensity Estimation
Non-parametricDensity GRADIENT Estimation
(Mean Shift)
Data
Discrete PDF Representation
PDF Analysis
PDF in feature space• Color space• Scale space• Actually any feature space you can conceive• …
A tool for:Finding modes in a set of data samples, manifesting an underlying probability density function (PDF) in RN
23年 4月 19日 13
![Page 14: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/14.jpg)
Non-Parametric Density Estimation
Assumption : The data points are sampled from an underlying PDF
Assumed Underlying PDF Real Data Samples
Data point density implies PDF value !
23年 4月 19日 14
![Page 15: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/15.jpg)
Assumed Underlying PDF Real Data Samples
Non-Parametric Density Estimation
23年 4月 19日 15
![Page 16: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/16.jpg)
Assumed Underlying PDF Real Data Samples
?Non-Parametric Density Estimation
23年 4月 19日 16
![Page 17: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/17.jpg)
Parametric Density Estimation
Assumption : The data points are sampled from an underlying PDF
Assumed Underlying PDF
2
2
( )
2
i
PDF( ) = i
iic e
x-μ
x
Estimate
Real Data Samples
23年 4月 19日 17
![Page 18: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/18.jpg)
1
1( ) ( )
n
ii
P Kn
x x - x A function of some finite number of data pointsx1…xn
DataIn practice one uses the forms:
1
( ) ( )d
ii
K c k x
x or ( )K ckx x
Same function on each dimension Function of vector length only
Kernel Density EstimationParzen Windows - General Framework
23年 4月 19日 19
![Page 19: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/19.jpg)
1
1( ) ( )
n
ii
P Kn
x x - x A function of some finite number of data pointsx1…xn
Examples:
• Epanechnikov Kernel
• Uniform Kernel
• Normal Kernel
21 1
( ) 0 otherwise
E
cK
x xx
1( )
0 otherwiseU
cK
xx
21( ) exp
2NK c
x x
Data
Kernel Density EstimationVarious Kernels
23年 4月 19日 20
![Page 20: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/20.jpg)
23年 4月 19日 21
Nonparametric Kernel Density Estimation
![Page 21: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/21.jpg)
Kernel Density EstimationGradient
1
1 ( ) ( )
n
ii
P Kn
x x - x Give up estimating the PDF !Estimate ONLY the gradient
2
( ) iiK ck
h
x - xx - x
Using theKernel form:
We get :
1
1 1
1
( )
n
i in ni
i i ni i
ii
gc c
P k gn n g
xx x
Size of window
g( ) ( )kx x23年 4月 19日 22
![Page 22: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/22.jpg)
Kernel Density EstimationGradient
1
1 1
1
( )
n
i in ni
i i ni i
ii
gc c
P k gn n g
xx x
Computing The Mean Shift
g( ) ( )kx x23年 4月 19日 23
![Page 23: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/23.jpg)
1
1 1
1
( )
n
i in ni
i i ni i
ii
gc c
P k gn n g
xx x
Computing The Mean Shift
Yet another Kernel density estimation !
Simple Mean Shift procedure:• Compute mean shift vector
•Translate the Kernel window by m(x)
2
1
2
1
( )
ni
ii
ni
i
gh
gh
x - xx
m x xx - x
g( ) ( )kx x23年 4月 19日 24
![Page 24: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/24.jpg)
23年 4月 19日 25
![Page 25: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/25.jpg)
Mean Shift Mode Detection
Updated Mean Shift Procedure:• Find all modes using the Simple Mean Shift Procedure• Prune modes by perturbing them (find saddle points and plateaus)• Prune nearby – take highest mode in the window
What happens if wereach a saddle point
?
Perturb the mode positionand check if we return back
23年 4月 19日 26
![Page 26: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/26.jpg)
AdaptiveGradient Ascent
Mean Shift Properties
• Automatic convergence speed – the mean shift vector size depends on the gradient itself.
• Near maxima, the steps are small and refined
• Convergence is guaranteed for infinitesimal steps only infinitely convergent, (therefore set a lower bound)
• For Uniform Kernel ( ), convergence is achieved in a finite number of steps
• Normal Kernel ( ) exhibits a smooth trajectory, but is slower than Uniform Kernel ( ).
23年 4月 19日 27
![Page 27: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/27.jpg)
Real Modality Analysis
Tessellate the space with windows
Run the procedure in parallel23年 4月 19日 28
![Page 28: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/28.jpg)
Real Modality Analysis
The blue data points were traversed by the windows towards the mode23年 4月 19日 29
![Page 29: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/29.jpg)
Real Modality Analysis
Window tracks signify the steepest ascent directions
An example
23年 4月 19日 30
![Page 30: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/30.jpg)
Mean Shift Strengths & Weaknesses
Strengths :
• Application independent tool
• Suitable for real data analysis
• Does not assume any prior shape (e.g. elliptical) on data clusters
• Can handle arbitrary feature spaces
• Only ONE parameter to choose
• h (window size) has a physical meaning, unlike K-Means
Weaknesses :
• The window size (bandwidth selection) is not trivial
• Inappropriate window size can cause modes to be merged, or generate additional “shallow” modes Use adaptive window size
23年 4月 19日 32
![Page 31: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/31.jpg)
Mean Shift Applications
23年 4月 19日 33
![Page 32: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/32.jpg)
Clustering
Attraction basin : the region for which all trajectories lead to the same mode
Cluster : All data points in the attraction basin of a mode
Mean Shift : A robust Approach Toward Feature Space Analysis, by Comaniciu, Meer
23年 4月 19日 34
![Page 33: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/33.jpg)
Clustering
Simple Modal Structures
Complex Modal Structures
Synthetic Examples
23年 4月 19日 35
![Page 34: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/34.jpg)
Clustering
Modes found Modes afterpruning
Final clusters
Feature space:L*u*v representation
Real Example Initial windowcenters
23年 4月 19日 36
![Page 35: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/35.jpg)
L*u*v space representation
ClusteringReal Example
23年 4月 19日 37
![Page 36: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/36.jpg)
Not all trajectoriesin the attraction basinreach the same mode
2D (L*u) space representation
Final clusters
ClusteringReal Example
23年 4月 19日 38
![Page 37: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/37.jpg)
Discontinuity Preserving SmoothingFeature space : Joint domain = spatial coordinates + color space
( )s r
s rs r
K C k kh h
x xx
Meaning : treat the image as data points in the spatial and gray level domain
Image Data(slice)
Mean Shiftvectors
Smoothingresult
Mean Shift : A robust Approach Toward Feature Space Analysis, by Comaniciu, Meer
23年 4月 19日 39
![Page 38: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/38.jpg)
Discontinuity Preserving Smoothing
y
zFlat regions induce the modes !
23年 4月 19日 41
![Page 39: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/39.jpg)
Discontinuity Preserving SmoothingThe effect of window sizein spatial andrange spaces
23年 4月 19日 42
![Page 40: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/40.jpg)
Discontinuity Preserving SmoothingExample
23年 4月 19日 43
![Page 41: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/41.jpg)
Discontinuity Preserving SmoothingExample
23年 4月 19日 44
![Page 42: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/42.jpg)
Object Contour DetectionRay Propagation
Vessel Detection by Mean Shift Based Ray Propagation, by Tek, Comaniciu, Williams
Accurately segment various objects (rounded in nature) in medical images
23年 4月 19日 45
![Page 43: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/43.jpg)
Object Contour DetectionRay Propagation
Use displacement data to guide ray propagation
Discontinuity preserving smoothing
Displacementvectors
Vessel Detection by Mean Shift Based Ray Propagation, by Tek, Comaniciu, Williams23年 4月 19日 46
![Page 44: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/44.jpg)
Object Contour DetectionRay Propagation
( , ) ( , )Ray
s t Speed x y Nt
Speed function
Normal to the contour
( , ) ( , ) ( , )Speed x y f disp x y x y
Curvature
23年 4月 19日 47
![Page 45: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/45.jpg)
Object Contour DetectionOriginal image
Gray levels along red line
Gray levels aftersmoothing
Displacement vectors Displacement vectors’derivative
( , ) ( , ) ( , )Speed x y f disp x y x y 23年 4月 19日 48
![Page 46: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/46.jpg)
Object Contour DetectionExample
23年 4月 19日 49
![Page 47: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/47.jpg)
Object Contour DetectionExample
Importance of smoothing by curvature
23年 4月 19日 50
![Page 48: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/48.jpg)
SegmentationSegment = Cluster, or Cluster of Clusters
Algorithm:• Run Filtering (discontinuity preserving smoothing)• Cluster the clusters which are closer than window size
Image Data(slice)
Mean Shiftvectors
Segmentationresult
Smoothingresult
Mean Shift : A robust Approach Toward Feature Space Analysis, by Comaniciu, Meerhttp://www.caip.rutgers.edu/~comanici
23年 4月 19日 51
![Page 49: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/49.jpg)
SegmentationExample
…when feature space is only gray levels…
23年 4月 19日 52
![Page 50: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/50.jpg)
SegmentationExample
23年 4月 19日 53
![Page 51: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/51.jpg)
SegmentationExample
23年 4月 19日 54
![Page 52: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/52.jpg)
SegmentationExample
23年 4月 19日 55
![Page 53: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/53.jpg)
SegmentationExample
23年 4月 19日 56
![Page 54: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/54.jpg)
SegmentationExample
23年 4月 19日 57
![Page 55: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/55.jpg)
SegmentationExample
23年 4月 19日 58
![Page 56: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/56.jpg)
Non-Rigid Object Tracking
… …
23年 4月 19日 59
![Page 57: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/57.jpg)
Current frame
… …
Mean-Shift Object TrackingGeneral Framework: Target Representation
Choose a feature space
Represent the model in the
chosen feature space
Choose a reference
model in the current frame
23年 4月 19日 61
![Page 58: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/58.jpg)
Mean-Shift Object TrackingGeneral Framework: Target Localization
Search in the model’s
neighborhood in next frame
Start from the position of the model in the current frame
Find best candidate by maximizing a similarity func.
Repeat the same process in the next pair
of frames
Current frame
… …Model Candidate
23年 4月 19日 62
![Page 59: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/59.jpg)
Mean-Shift Object TrackingTarget Representation
Choose a reference
target model
Quantized Color Space
Choose a feature space
Represent the model by its PDF in the
feature space
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
1 2 3 . . . m
color
Pro
bab
ility
Kernel Based Object Tracking, by Comaniniu, Ramesh, Meer23年 4月 19日 63
![Page 60: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/60.jpg)
Mean-Shift Object TrackingPDF Representation
,f y f q p y Similarity
Function:
Target Model(centered at 0)
Target Candidate(centered at y)
1..1
1m
u uu mu
q q q
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
1 2 3 . . . m
color
Pro
bab
ility
1..
1
1m
u uu mu
p y p y p
0
0.05
0.1
0.15
0.2
0.25
0.3
1 2 3 . . . m
color
Pro
bab
ility
23年 4月 19日 64
![Page 61: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/61.jpg)
Mean-Shift Object TrackingFinding the PDF of the target model
1..i i nx
Target pixel locations
( )k x A differentiable, isotropic, convex, monotonically decreasing kernel• Peripheral pixels are affected by occlusion and background interference
( )b x The color bin index (1..m) of pixel x
2
( )i
u ib x u
q C k x
Normalization factor
Pixel weight
Probability of feature u in model
0
0.05
0.1
0.15
0.2
0.25
0.3
1 2 3 . . . m
color
Pro
bab
ility
Probability of feature u in candidate
0
0.05
0.1
0.15
0.2
0.25
0.3
1 2 3 . . . m
color
Pro
bab
ility
2
( )i
iu h
b x u
y xp y C k
h
Normalization factor
Pixel weight
0
model
y
candidate
23年 4月 19日 66
![Page 62: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/62.jpg)
Mean-Shift Object TrackingSimilarity Function
1 , , mp y p y p y
1, , mq q q
Target model:
Target candidate:
Similarity function: , ?f y f p y q
1 , , mp y p y p y
1 , , mq q q
q
p y
y1
1
The Bhattacharyya Coefficient
1
cosT m
y u uu
p y qf y p y q
p y q
23年 4月 19日 67
![Page 63: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/63.jpg)
,f p y q
Mean-Shift Object TrackingTarget Localization Algorithm
Start from the position of the model in the current frame
q
Search in the model’s
neighborhood in next frame
p y
Find best candidate by maximizing a similarity func.
23年 4月 19日 68
![Page 64: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/64.jpg)
01 1 0
1 1
2 2
m mu
u u uu u u
qf y p y q p y
p y
Linear
approx.(around y0)
Mean-Shift Object TrackingApproximating the Similarity Function
0yModel location:yCandidate location:
2
12
nh i
ii
C y xw k
h
2
( )i
iu h
b x u
y xp y C k
h
Independent of y
Density estimate!
(as a function of y)
1
m
u uu
f y p y q
23年 4月 19日 69
![Page 65: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/65.jpg)
Mean-Shift Object TrackingMaximizing the Similarity Function
2
12
nh i
ii
C y xw k
h
The mode of = sought maximum
Important Assumption:
One mode in the searched neighborhood
The target representation
provides sufficient discrimination
23年 4月 19日 70
![Page 66: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/66.jpg)
Mean-Shift Object TrackingApplying Mean-Shift
Original Mean-Shift:
2
0
1
1 2
0
1
ni
ii
ni
i
y xx g
hy
y xg
h
Find mode of
2
1
ni
i
y xc k
h
using
2
12
nh i
ii
C y xw k
h
The mode of = sought maximum
Extended Mean-Shift:
2
0
1
1 2
0
1
ni
i ii
ni
ii
y xx w g
hy
y xw g
h
2
1
ni
ii
y xc w k
h
Find mode of using
23年 4月 19日 71
![Page 67: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/67.jpg)
Mean-Shift Object TrackingAbout Kernels and Profiles
A special class of radially symmetric kernels: 2
K x ck x
The profile of kernel K
Extended Mean-Shift:
2
0
1
1 2
0
1
ni
i ii
ni
ii
y xx w g
hy
y xw g
h
2
1
ni
ii
y xc w k
h
Find mode of using
k x g x
23年 4月 19日 72
![Page 68: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/68.jpg)
Mean-Shift Object TrackingChoosing the Kernel
Epanechnikov kernel:
1 if x 1
0 otherwise
xk x
A special class of radially symmetric kernels: 2
K x ck x
11
1
n
i iin
ii
x wy
w
2
0
1
1 2
0
1
ni
i ii
ni
ii
y xx w g
hy
y xw g
h
1 if x 1
0 otherwiseg x k x
Uniform kernel:
23年 4月 19日 73
![Page 69: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/69.jpg)
Mean-Shift Object TrackingAdaptive Scale
Problem:
The scale of the target
changes in time
The scale (h) of the
kernel must be adapted
Solution:
Run localization 3
times with different h
Choose h that achieves
maximum similarity
23年 4月 19日 74
![Page 70: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/70.jpg)
Mean-Shift Object TrackingResults
Feature space: 161616 quantized RGBTarget: manually selected on 1st frameAverage mean-shift iterations: 4
23年 4月 19日 75
![Page 71: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/71.jpg)
Mean-Shift Object TrackingResults
Partial occlusion Distraction Motion blur
23年 4月 19日 76
![Page 72: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/72.jpg)
Mean-Shift Object TrackingResults
23年 4月 19日 77
![Page 73: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/73.jpg)
Mean-Shift Object TrackingResults
Feature space: 128128 quantized RG23年 4月 19日 78
![Page 74: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/74.jpg)
Mean-Shift Object TrackingThe Scale Selection Problem
Kernel too big
Kernel too smallPoor
localization
h mustn’t get too big or too
small!
Problem:In uniformly
colored regions, similarity is
invariant to h
Smaller h may achieve better
similarity
Nothing keeps h from shrinking
too small!23年 4月 19日 79
![Page 75: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/75.jpg)
Tracking Through Scale SpaceMotivation
Spatial localization for several
scalesPrevious method
Simultaneous localization in
space and scale
This method
Mean-shift Blob Tracking through Scale Space, by R. Collins23年 4月 19日 80
![Page 76: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/76.jpg)
Lindeberg’s TheorySelecting the best scale for describing image features
Scale-space representation
Differential operator applied
50 strongest responses
x
y
σ
23年 4月 19日 81
![Page 77: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/77.jpg)
Scale-space representation
1;G x
2;G x
; kG x
Lindeberg’s TheoryThe Laplacian operator for selecting blob-like features
Laplacian of Gaussian (LOG)
f x
Best features are at (x,σ) that maximize L
1( ; )LOG x
2( ; )LOG x
( ; )kLOG x
2
2
22
26
2;
2
xxLOG x e
2D LOG filter with scale σ
1.., :
, ;kx f
L x LOG x f x
x
y
σ
3D scale-space representation
21;G x
22;G x
2 ; kG x
23年 4月 19日 82
![Page 78: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/78.jpg)
Lindeberg’s TheoryMulti-Scale Feature Selection Process
Original Image
f x
3D scale-space function
, ;L x LOG x f x
Convolve
250 strongest responses(Large circle = large scale)
Maximize
23年 4月 19日 83
![Page 79: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/79.jpg)
Tracking Through Scale SpaceApproximating LOG using DOG
Why DOG?
• Gaussian pyramids are created faster
• Gaussian can be used as a mean-shift kernel
; ; ; ;1.6LOG x DOG x G x G x
2D LOG filter with scale σ
2D DOG filter with scale σ
2D Gaussian with μ=0 and scale σ
2D Gaussian with μ=0 and scale 1.6σ
,K x
DOG filters at multiple scales3D spatial
kernel
2
1
k
Scale-space filter bank
23年 4月 19日 84
![Page 80: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/80.jpg)
Tracking Through Scale SpaceUsing Lindeberg’s Theory
Weight image
( )
( ) 0
b x
b x
qw x
p y
1 , , mp y p y p y
1, , mq q q
Model:
Candidate:
( )b xColor bin:
0yat
Pixel weight:
Recall:
The likelihood that each candidate
pixel belongs to the target
1D scale kernel (Epanechnikov)
3D spatial kernel (DOG)
Centered at current
location and scale
3D scale-space representation
,E x
Modes are blobs in the scale-space neighborhood
Need a mean-shift procedure that
finds local modes in E(x,σ)
23年 4月 19日 85
![Page 81: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/81.jpg)
Tracking Through Scale SpaceExample
Image of 3 blobs
A slice through the 3D scale-
space representation
23年 4月 19日 86
![Page 82: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/82.jpg)
Tracking Through Scale SpaceApplying Mean-Shift
Use interleaved spatial/scale mean-shift
Spatial stage:
Fix σ and look for the
best x
Scale stage:
Fix x and look for the
best σ
Iterate stages until
convergence of x and σx
σ
x0
σ0
xopt
σopt
23年 4月 19日 87
![Page 83: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/83.jpg)
Tracking Through Scale SpaceResults
Fixed-scale
Tracking through scale space
± 10% scale adaptation
23年 4月 19日 88
![Page 84: Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling 2009.10.9 lbg@dongseo.ac.kr 2015-09-121](https://reader035.vdocuments.site/reader035/viewer/2022062421/56649e395503460f94b2aed0/html5/thumbnails/84.jpg)
23年 4月 19日 89