background subtraction for temporally irregular dynamic textures gerald dalley, joshua migdal, and...
TRANSCRIPT
![Page 1: Background Subtraction for Temporally Irregular Dynamic Textures Gerald Dalley, Joshua Migdal, and W. Eric L. Grimson Workshop on Applications of Computer](https://reader030.vdocuments.site/reader030/viewer/2022032709/56649eb45503460f94bbc0d5/html5/thumbnails/1.jpg)
Background Subtraction for Temporally Irregular
Dynamic TexturesGerald Dalley, Joshua Migdal,
and W. Eric L. Grimson
Workshop on Applications of Computer Vision (WACV) 2008
![Page 2: Background Subtraction for Temporally Irregular Dynamic Textures Gerald Dalley, Joshua Migdal, and W. Eric L. Grimson Workshop on Applications of Computer](https://reader030.vdocuments.site/reader030/viewer/2022032709/56649eb45503460f94bbc0d5/html5/thumbnails/2.jpg)
8 Jan. 2008 2
Key Prior Approaches
MoG – learn modes for all pixels & track changes Works as long as dynamic textures are regular
Per-pixel kernels (Mittal & Paragios) Can pay attention to higher-level features like optical flow How many kernels to keep?
Spatial kernels (Sheikh & Shah) Handles temporal irregularity in dynamic textures How many kernels to keep?
![Page 3: Background Subtraction for Temporally Irregular Dynamic Textures Gerald Dalley, Joshua Migdal, and W. Eric L. Grimson Workshop on Applications of Computer](https://reader030.vdocuments.site/reader030/viewer/2022032709/56649eb45503460f94bbc0d5/html5/thumbnails/3.jpg)
8 Jan. 2008 3
Results with Mittal & Paragios Traffic Clip
Key: bboxes from
ground truth True
positive False
positive False
negative
No ground truth available
![Page 4: Background Subtraction for Temporally Irregular Dynamic Textures Gerald Dalley, Joshua Migdal, and W. Eric L. Grimson Workshop on Applications of Computer](https://reader030.vdocuments.site/reader030/viewer/2022032709/56649eb45503460f94bbc0d5/html5/thumbnails/4.jpg)
8 Jan. 2008 4
Our Goals
Compactness Speed Complexity growth
Grow with scene complexity, not with the amount of time to see the complexity If a leaf blows in front of a pixel once a minute, we don’t
want to have to keep 1800 kernels
Usable in MoG frameworks MoG variants are very widely used (original
Stauffer & Grimson paper has 1000+ citations)
![Page 5: Background Subtraction for Temporally Irregular Dynamic Textures Gerald Dalley, Joshua Migdal, and W. Eric L. Grimson Workshop on Applications of Computer](https://reader030.vdocuments.site/reader030/viewer/2022032709/56649eb45503460f94bbc0d5/html5/thumbnails/5.jpg)
8 Jan. 2008 5
Pixels Affected by a GivenMixture ComponentOur Model
Mixture of Gaussians (MoG)
Standard MoG
Ours
p(ci j©) /P
j 2N iwj N (ci ;¹ j ;§ j )
ci theobserved color at pixel i
© themodel fwj ;¹ j ;§ j gj
Ni neighborhood of pixel i
p(ci j©) /P
j 2N iwj N (ci ;¹ j ;§ j )
ci theobserved color at pixel i
© themodel fwj ;¹ j ;§ j gj
Ni neighborhood of pixel i
![Page 6: Background Subtraction for Temporally Irregular Dynamic Textures Gerald Dalley, Joshua Migdal, and W. Eric L. Grimson Workshop on Applications of Computer](https://reader030.vdocuments.site/reader030/viewer/2022032709/56649eb45503460f94bbc0d5/html5/thumbnails/6.jpg)
8 Jan. 2008 6
Foreground/Background Classification Find best matching background Gaussian, j
Use neighborhood Squared Mahalanobis Distance
-
=
input
d2i j
mixture model
©1 ©2 ©3
di j = (ci ¡ ¹ j )T § ¡ 1j (ci ¡ ¹ j )di j = (ci ¡ ¹ j )T § ¡ 1j (ci ¡ ¹ j )
![Page 7: Background Subtraction for Temporally Irregular Dynamic Textures Gerald Dalley, Joshua Migdal, and W. Eric L. Grimson Workshop on Applications of Computer](https://reader030.vdocuments.site/reader030/viewer/2022032709/56649eb45503460f94bbc0d5/html5/thumbnails/7.jpg)
8 Jan. 2008 7
hard Stauffer-Grimson
soft Stauffer-Grimson
best match only
update all matches
Model Update OptionsHard UpdatesSoft Updates
Loca
lR
egio
nal
![Page 8: Background Subtraction for Temporally Irregular Dynamic Textures Gerald Dalley, Joshua Migdal, and W. Eric L. Grimson Workshop on Applications of Computer](https://reader030.vdocuments.site/reader030/viewer/2022032709/56649eb45503460f94bbc0d5/html5/thumbnails/8.jpg)
8 Jan. 2008 8
Results on a Classic Dataset: Wallflower WavingTrees
Ground
Truth
Toyam
a et al.
Best
MoG
Ours
Final ResultAfter MRFMahalanobisDistances
![Page 9: Background Subtraction for Temporally Irregular Dynamic Textures Gerald Dalley, Joshua Migdal, and W. Eric L. Grimson Workshop on Applications of Computer](https://reader030.vdocuments.site/reader030/viewer/2022032709/56649eb45503460f94bbc0d5/html5/thumbnails/9.jpg)
8 Jan. 2008 9
ROC (Wallflower)
![Page 10: Background Subtraction for Temporally Irregular Dynamic Textures Gerald Dalley, Joshua Migdal, and W. Eric L. Grimson Workshop on Applications of Computer](https://reader030.vdocuments.site/reader030/viewer/2022032709/56649eb45503460f94bbc0d5/html5/thumbnails/10.jpg)
8 Jan. 2008 10
Traffic #410G
round T
ruthM
ittal &
Paragios
Best
MoG
Ours
Final ResultAfter MRFMahalanobisDistances
![Page 11: Background Subtraction for Temporally Irregular Dynamic Textures Gerald Dalley, Joshua Migdal, and W. Eric L. Grimson Workshop on Applications of Computer](https://reader030.vdocuments.site/reader030/viewer/2022032709/56649eb45503460f94bbc0d5/html5/thumbnails/11.jpg)
8 Jan. 2008 11
Traffic #150G
round T
ruthM
ittal &
Paragios
Best
MoG
Ours
Final ResultAfter MRFMahalanobisDistances
![Page 12: Background Subtraction for Temporally Irregular Dynamic Textures Gerald Dalley, Joshua Migdal, and W. Eric L. Grimson Workshop on Applications of Computer](https://reader030.vdocuments.site/reader030/viewer/2022032709/56649eb45503460f94bbc0d5/html5/thumbnails/12.jpg)
8 Jan. 2008 12
Pushing the Limits:Ducks
Ground
Truth
B
est M
oGO
urs
Final ResultAfter MRFMahalanobisDistances
![Page 13: Background Subtraction for Temporally Irregular Dynamic Textures Gerald Dalley, Joshua Migdal, and W. Eric L. Grimson Workshop on Applications of Computer](https://reader030.vdocuments.site/reader030/viewer/2022032709/56649eb45503460f94bbc0d5/html5/thumbnails/13.jpg)
8 Jan. 2008 13
Pushing the Limits:Jug
Ground
Truth
Zhong &
S
claroffB
est M
oGO
urs
Final ResultAfter MRFMahalanobisDistances
![Page 14: Background Subtraction for Temporally Irregular Dynamic Textures Gerald Dalley, Joshua Migdal, and W. Eric L. Grimson Workshop on Applications of Computer](https://reader030.vdocuments.site/reader030/viewer/2022032709/56649eb45503460f94bbc0d5/html5/thumbnails/14.jpg)
8 Jan. 2008 14
Parameter Sensitivity
0 1000 2000 3000 4000 5000 6000 70000
0.2
0.4
0.6
0.8
1
min number of mistakes
frequ
ency
ws=7ws=5ws=3ws=1
traffic
0 500 1000 1500 20000
0.05
0.1
0.15
min number of mistakes
frequ
ency
ws=7ws=5ws=3ws=1
wallflower
max max
![Page 15: Background Subtraction for Temporally Irregular Dynamic Textures Gerald Dalley, Joshua Migdal, and W. Eric L. Grimson Workshop on Applications of Computer](https://reader030.vdocuments.site/reader030/viewer/2022032709/56649eb45503460f94bbc0d5/html5/thumbnails/15.jpg)
8 Jan. 2008 15
Summary
Model representation Same as MoG
FG/BG Classifier Input Add local neighborhood loop
Update Several options, including standard MoG updates
Results Reduces foreground false positives Reduces need for exotic input features (e.g. optical flow) Cost window size∝
![Page 16: Background Subtraction for Temporally Irregular Dynamic Textures Gerald Dalley, Joshua Migdal, and W. Eric L. Grimson Workshop on Applications of Computer](https://reader030.vdocuments.site/reader030/viewer/2022032709/56649eb45503460f94bbc0d5/html5/thumbnails/16.jpg)
8 Jan. 2008 16
Also in the Paper…
Analysis Timing experiments More ROC analysis
Implementation Details MRF choice Update formulas
![Page 17: Background Subtraction for Temporally Irregular Dynamic Textures Gerald Dalley, Joshua Migdal, and W. Eric L. Grimson Workshop on Applications of Computer](https://reader030.vdocuments.site/reader030/viewer/2022032709/56649eb45503460f94bbc0d5/html5/thumbnails/17.jpg)
8 Jan. 2008 17
Thank You
Questions...
![Page 18: Background Subtraction for Temporally Irregular Dynamic Textures Gerald Dalley, Joshua Migdal, and W. Eric L. Grimson Workshop on Applications of Computer](https://reader030.vdocuments.site/reader030/viewer/2022032709/56649eb45503460f94bbc0d5/html5/thumbnails/18.jpg)
8 Jan. 2008 18
Backup Slides
![Page 19: Background Subtraction for Temporally Irregular Dynamic Textures Gerald Dalley, Joshua Migdal, and W. Eric L. Grimson Workshop on Applications of Computer](https://reader030.vdocuments.site/reader030/viewer/2022032709/56649eb45503460f94bbc0d5/html5/thumbnails/19.jpg)
8 Jan. 2008 19
PureSoft SoftLocal PureHard HardLocal0
10
20
30
40
50
60R
elat
ive
Tim
e C
ost
Model Update Computational Costs
W=12
W=32
W=52
W=72
![Page 20: Background Subtraction for Temporally Irregular Dynamic Textures Gerald Dalley, Joshua Migdal, and W. Eric L. Grimson Workshop on Applications of Computer](https://reader030.vdocuments.site/reader030/viewer/2022032709/56649eb45503460f94bbc0d5/html5/thumbnails/20.jpg)
8 Jan. 2008 208 Jan. 2008 20
PureSoft SoftLocal PureHard HardLocal0
20
40
60
80
100R
elat
ive
Tim
e C
ost
Mahalanobis Map Computational Costs
W=12
W=32
W=52
W=72
![Page 21: Background Subtraction for Temporally Irregular Dynamic Textures Gerald Dalley, Joshua Migdal, and W. Eric L. Grimson Workshop on Applications of Computer](https://reader030.vdocuments.site/reader030/viewer/2022032709/56649eb45503460f94bbc0d5/html5/thumbnails/21.jpg)
8 Jan. 2008 21
12345678
1 2 3 4 1 2 3 4 1 2 3 41 2 3 4
Background Model
Observed Image
9x9 Matchin
g
3x3 Matchin
g
Small Windows are Still Useful
![Page 22: Background Subtraction for Temporally Irregular Dynamic Textures Gerald Dalley, Joshua Migdal, and W. Eric L. Grimson Workshop on Applications of Computer](https://reader030.vdocuments.site/reader030/viewer/2022032709/56649eb45503460f94bbc0d5/html5/thumbnails/22.jpg)
8 Jan. 2008 22
c(t¡ 1)ic(t¡ 1)ic(t¡ 1)i c(t)
ic(t)ic(t)i
Á(t)jÁ(t)j
z(t¡ 1)iz(t¡ 1)i
l(t¡ 1)jl(t¡ 1)jl(t¡ 1)j l(t)jl
(t)jl(t)j
JJ
II
z(t)iz(t)i
Á(t¡ 1)jÁ(t¡ 1)j
Generative Model
Particle locations
Particle appearances
Pixel-to-particle assignments
Pixel appearance
![Page 23: Background Subtraction for Temporally Irregular Dynamic Textures Gerald Dalley, Joshua Migdal, and W. Eric L. Grimson Workshop on Applications of Computer](https://reader030.vdocuments.site/reader030/viewer/2022032709/56649eb45503460f94bbc0d5/html5/thumbnails/23.jpg)
8 Jan. 2008 23
Independent Assignments
l(t¡ 1)1
l(t¡ 1)2
l(t¡ 1)3
z(t)i = 2
l(t)1
l(t)2
l(t)3
l(t¡ 1)1l(t¡ 1)1l(t¡ 1)1
l(t¡ 1)2l(t¡ 1)2l(t¡ 1)2
l(t¡ 1)3l(t¡ 1)3l(t¡ 1)3
z(t)i = 2z(t)i = 2z(t)i = 2
l(t)1l(t)1l(t)1
l(t)2l(t)2l(t)2
l(t)3l(t)3l(t)3