![Page 1: Online Detection of Unusual Events in Videos via Dynamic Sparse Coding](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813b85550346895da4b0ae/html5/thumbnails/1.jpg)
Online Detection of Unusual Events in Videos via Dynamic Sparse Coding
![Page 2: Online Detection of Unusual Events in Videos via Dynamic Sparse Coding](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813b85550346895da4b0ae/html5/thumbnails/2.jpg)
Outline
• Unusual Event Detection• Video Representation• Dynamic Sparse Coding• Empirical Study• Conclusions
![Page 3: Online Detection of Unusual Events in Videos via Dynamic Sparse Coding](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813b85550346895da4b0ae/html5/thumbnails/3.jpg)
Outline
• Unusual Event Detection• Video Representation• Dynamic Sparse Coding• Empirical Study• Conclusions
![Page 4: Online Detection of Unusual Events in Videos via Dynamic Sparse Coding](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813b85550346895da4b0ae/html5/thumbnails/4.jpg)
Unusual events: Incidences that occur very rarely in the entire video
![Page 5: Online Detection of Unusual Events in Videos via Dynamic Sparse Coding](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813b85550346895da4b0ae/html5/thumbnails/5.jpg)
Unusual Event Detection
• Easy-to-verify– Given a frame, fairly easy to decide if unusual
events occur• Hard-to-describe– Cannot enumerate all possible unusual events – Cannot model unusual events directly
• Solution: Model usual events instead, and claim anything different as unusual
Easy to model usual events?
![Page 6: Online Detection of Unusual Events in Videos via Dynamic Sparse Coding](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813b85550346895da4b0ae/html5/thumbnails/6.jpg)
Challenges
• Unsupervised learning– Only input is video itself
• Online detecting– In most cases, cannot afford multiple runs through
the video• Concept drift– Usual events change
• Truly unusual event vs. Noisy usual event
![Page 7: Online Detection of Unusual Events in Videos via Dynamic Sparse Coding](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813b85550346895da4b0ae/html5/thumbnails/7.jpg)
Previous Works
• Clustering Based Method (CVPR 2004)
– Finding spatially isolated clusters• Reconstruction (IJCV 2007)
• Space-time Markov Random Field (CVPR 2009)
![Page 8: Online Detection of Unusual Events in Videos via Dynamic Sparse Coding](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813b85550346895da4b0ae/html5/thumbnails/8.jpg)
Outline
• Unusual Event Detection• Video Representation• Dynamic Sparse Coding• Empirical Study• Conclusions
![Page 9: Online Detection of Unusual Events in Videos via Dynamic Sparse Coding](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813b85550346895da4b0ae/html5/thumbnails/9.jpg)
Video Features
• Static features based on edges and object shapes– Image-level information
• Dynamic features based on optical flow measurements– Motion information
• Spatio-Temporal Interest Points– Obtained from local video patches– Shown to be useful in human action categorization
![Page 10: Online Detection of Unusual Events in Videos via Dynamic Sparse Coding](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813b85550346895da4b0ae/html5/thumbnails/10.jpg)
Spatio-Temporal Interest Point
• Detection– Basic idea: generalize interest point detector from
spatial domain to spatio-temporal domain– Spatial (image): Laplacian, Hessian, Harris corner
detector, etc.– Spatio-temporal (video): spatio-temporal corners,
Laplacian on spatial and temporal axis– Output: small video patches extracted from each
interest point
![Page 11: Online Detection of Unusual Events in Videos via Dynamic Sparse Coding](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813b85550346895da4b0ae/html5/thumbnails/11.jpg)
Spatio-Temporal Interest Point (Cont.)
• Description– Similar to detection, generalization of spatial
method to spatio-temporal domain– Spatial: histogram of directional gradients – SIFT,
HOG– Spatio-Temporal: gradients on x, y, and time
directions
![Page 12: Online Detection of Unusual Events in Videos via Dynamic Sparse Coding](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813b85550346895da4b0ae/html5/thumbnails/12.jpg)
Outline
• Unusual Event Detection• Video Representation• Dynamic Sparse Coding• Empirical Study• Conclusions
![Page 13: Online Detection of Unusual Events in Videos via Dynamic Sparse Coding](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813b85550346895da4b0ae/html5/thumbnails/13.jpg)
Motivation of the Approach
• Sparse Reconstruction– Reconstruct an event with other events in the
video• Usual events: multiple appearances could find a few
events to reconstruct it SPARSE• Unusual events: rare appearance need large amount
of events for reconstruction DENSE
• Concisely represent the knowledge of usual events
![Page 14: Online Detection of Unusual Events in Videos via Dynamic Sparse Coding](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813b85550346895da4b0ae/html5/thumbnails/14.jpg)
The Proposed Approach• Define events in the video
– Sliding window runs through the video– Spatio-temporal interest points within the same window define
an event
• Knowledge of what are usual events– Store in the learned dictionary D
• Abnormality of an event
• Update dictionary D
![Page 15: Online Detection of Unusual Events in Videos via Dynamic Sparse Coding](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813b85550346895da4b0ae/html5/thumbnails/15.jpg)
(Ab)Normality
• Reconstruction error
• Sparsity regularization
• Smoothness regularization
![Page 16: Online Detection of Unusual Events in Videos via Dynamic Sparse Coding](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813b85550346895da4b0ae/html5/thumbnails/16.jpg)
(Ab)Normality (Cont.)
• Empirical demonstration
![Page 17: Online Detection of Unusual Events in Videos via Dynamic Sparse Coding](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813b85550346895da4b0ae/html5/thumbnails/17.jpg)
Work-flow
![Page 18: Online Detection of Unusual Events in Videos via Dynamic Sparse Coding](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813b85550346895da4b0ae/html5/thumbnails/18.jpg)
Optimization
• Learning with Fixed 𝜶 D
• Learning D with Fixed 𝜶
• Online Dictionary Update
![Page 19: Online Detection of Unusual Events in Videos via Dynamic Sparse Coding](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813b85550346895da4b0ae/html5/thumbnails/19.jpg)
Outline
• Unusual Event Detection• Video Representation• Dynamic Sparse Coding• Empirical Study• Conclusions
![Page 20: Online Detection of Unusual Events in Videos via Dynamic Sparse Coding](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813b85550346895da4b0ae/html5/thumbnails/20.jpg)
Video Data• Subway Surveillance Videos
– Subway exit: 43 minutes, 65K frames• Usual events: people exiting subway• Unusual events: entering subway, loitering, etc.
– Subway entrance: 1 hour 36 minutes, 144K frames• Usual events: people entering subway• Unusual events: exiting subway, no payment, loitering
• Youtube Videos: 8 short videos– Different camera motion (rotation, zoom in/out, fast tracking, slow
motion, etc.)– Different categories of targets (human, vehicles, animals, etc.)– Wide variety of activities and environmental conditions (indoor, outdoor).
![Page 21: Online Detection of Unusual Events in Videos via Dynamic Sparse Coding](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813b85550346895da4b0ae/html5/thumbnails/21.jpg)
Learned Dictionary
• Subway exit surveillance video
• Subway entrance surveillance video
![Page 22: Online Detection of Unusual Events in Videos via Dynamic Sparse Coding](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813b85550346895da4b0ae/html5/thumbnails/22.jpg)
Quantitative Comparison
• Subway exit surveillance video
• Subway entrance surveillance video
![Page 23: Online Detection of Unusual Events in Videos via Dynamic Sparse Coding](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813b85550346895da4b0ae/html5/thumbnails/23.jpg)
Analysis Experiment
• Online Update of the Learned Dictionary– Our approach: update learned dictionary after
observing new event– Comparing method: fixed dictionary
![Page 24: Online Detection of Unusual Events in Videos via Dynamic Sparse Coding](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813b85550346895da4b0ae/html5/thumbnails/24.jpg)
Detected Unusual Events
• Subway Exit
![Page 25: Online Detection of Unusual Events in Videos via Dynamic Sparse Coding](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813b85550346895da4b0ae/html5/thumbnails/25.jpg)
Detected Unusual Events (Cont.)
• Subway entrance
![Page 26: Online Detection of Unusual Events in Videos via Dynamic Sparse Coding](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813b85550346895da4b0ae/html5/thumbnails/26.jpg)
Detected Unusual Events (Cont.)
• Youtube Videos– For each video, approximately the first 1/5 of
video data is used to learn initial dictionary– Unusual event detection is carried out in the
remaining video– Red boxes represent sliding windows that result in
unusual event detection
![Page 27: Online Detection of Unusual Events in Videos via Dynamic Sparse Coding](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813b85550346895da4b0ae/html5/thumbnails/27.jpg)
![Page 28: Online Detection of Unusual Events in Videos via Dynamic Sparse Coding](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813b85550346895da4b0ae/html5/thumbnails/28.jpg)
Outline
• Unusual Event Detection• Video Representation• Dynamic Sparse Coding• Empirical Study• Conclusions
![Page 29: Online Detection of Unusual Events in Videos via Dynamic Sparse Coding](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813b85550346895da4b0ae/html5/thumbnails/29.jpg)
Conclusions
• Fully unsupervised dynamic sparse coding approach for detecting unusual events in videos
• Bases dictionary is updated in an online fashion as the algorithm observes more data, avoiding any issues with concept drift.