tracking multiple occluding people by localizing on multiple scene planes professor :王聖智...
TRANSCRIPT
![Page 1: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/1.jpg)
Tracking Multiple Occluding People by
Localizing on Multiple Scene Planes
Professor :王聖智 教授Student :周節
![Page 2: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/2.jpg)
Outline
• Introduction
• Related work
• Localization Algorithm
• Tracking Algorithm
• Result
• Conclusion
• Reference
![Page 3: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/3.jpg)
Outline
• Introduction
• Related work
• Localization Algorithm
• Tracking Algorithm
• Result
• Conclusion
• Reference
![Page 4: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/4.jpg)
Introduction
Surveillance• Detection• Tracking
![Page 5: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/5.jpg)
IntroductionIf the object is isolated…• Physical properties are useful.• Such as color, shape…• It is much simpler.
![Page 6: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/6.jpg)
Introduction
?
But if the objects are not isolated…• Occlusion • Lack of visibility • In crowded and cluttered scenes
![Page 7: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/7.jpg)
IntroductionDifficulty• It’s difficult to track
individual people
when occlusion occurred.
![Page 8: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/8.jpg)
Outline
• Introduction
• Related work
• Localization Algorithm
• Tracking Algorithm
• Result
• Conclusion
• Reference
![Page 9: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/9.jpg)
Related work
• Monocular Approach
• Multi-camera Approach
![Page 10: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/10.jpg)
Monocular Approach
• Handling occlusions (1) Color and shape
(2) Objects contours and appearances
(3) Blob split and merge analysis (4) ……
![Page 11: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/11.jpg)
Monocular Approach
• Drawbacks (1) Can’t deal with full occlusion.
(2) Can’t deal with long periods.
• Therefore (1) Single view is limited.
(2) Multi-camera is preferred.
![Page 12: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/12.jpg)
Multi-camera Approach
• Methods (1) Construct 3D model using voxel
(2) Calibrated camera to obtain 3D locations
(3) Switching camera
(4) Stereo camera
(5) ……
![Page 13: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/13.jpg)
Multi-camera Approach
• Problems (1) Features might be corrupted by occlusions.
(2) Very similar color and shape.
(3) Full occlusion.
• Therefore This paper…
![Page 14: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/14.jpg)
Outline
• Introduction
• Related work
• Localization Algorithm
• Tracking Algorithm
• Result
• Conclusion
• Reference
![Page 15: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/15.jpg)
Localization Algorithm
1. Obtain the foreground likelihood maps
2. Obtain reference plane homographies
3. Compute and find the localization
![Page 16: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/16.jpg)
Localization Algorithm
1. Obtain the foreground likelihood maps
– Model Background using a Mixture of Gaussians.– Perform Background Subtraction to obtain
foreground likelihood information.
![Page 17: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/17.jpg)
Localization Algorithm
2. Obtain reference plane homographies
– Homography Constraint
![Page 18: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/18.jpg)
Homography Constraint
(R , T)
![Page 19: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/19.jpg)
Homography Constraint
![Page 20: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/20.jpg)
Homography Constraint
![Page 21: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/21.jpg)
Homography Constraint
![Page 22: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/22.jpg)
Localization Algorithm
• x1~xn are the observations in images.• X is the event that the pixel is inside a foreground object.• L(xi) is the likelihood of observation xi belonging to the fo
reground.
![Page 23: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/23.jpg)
Example
![Page 24: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/24.jpg)
Example
![Page 25: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/25.jpg)
Example
![Page 26: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/26.jpg)
Localization AlgorithmFor robustness• Modeling Clutter and FOV Constraints
![Page 27: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/27.jpg)
Example
![Page 28: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/28.jpg)
Localization Algorithm
• For robustness– Localization at Multiple Planes
![Page 29: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/29.jpg)
Outline
• Introduction
• Related work
• Localization Algorithm
• Tracking Algorithm
• Result
• Conclusion
• Reference
![Page 30: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/30.jpg)
Tracking Algorithm
• Object trajectories are spatially and temporally coherent.
T=0 T=10 T=20 T=30
![Page 31: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/31.jpg)
Tracking Algorithm• Spatially and temporally coherent
![Page 32: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/32.jpg)
Tracking Algorithm
![Page 33: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/33.jpg)
Tracking Algorithm
DEMO
![Page 34: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/34.jpg)
Outline
• Introduction
• Related work
• Localization Algorithm
• Tracking Algorithm
• Result
• Conclusion
• Reference
![Page 35: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/35.jpg)
Result
• Case 1– Parking lot
![Page 36: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/36.jpg)
Result
• Multi-camera improves performance.
![Page 37: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/37.jpg)
Result
• Case 2– Indoor
![Page 38: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/38.jpg)
Result
• Multi-plane improves performance.
![Page 39: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/39.jpg)
Result
• Case 3– Basketball
• View limited – Error– Miss
![Page 40: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/40.jpg)
Result
• Case 4– Soccer
• Background limited – Error– Miss
![Page 41: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/41.jpg)
Outline
• Introduction
• Related work
• Localization Algorithm
• Tracking Algorithm
• Result
• Conclusion
• Reference
![Page 42: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/42.jpg)
Conclusion
To resolve occlusions and localize people:
• Planar homography constraints are used to fuse foreground likelihood information.
• Detection and tracking are performed simultaneously in the space-time occupancy likelihood data.
![Page 43: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/43.jpg)
Conclusion
• Advantage– Very good performance when occlusions occur.
– No calibration is needed.
– Only 2D constructs , purely image-based .
![Page 44: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/44.jpg)
Conclusion
• Disadvantage– If a person’s appearance is very similar to the
background or – If a person is occluded by some portion of the
background itself.– If a part of the scene is occluded in all views by the
foreground objects.
• Solutions– Add other models – like color model ,human motion model…
![Page 45: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/45.jpg)
Reference
• Saad M. Khan and Mubarak Shah, “Tracking Multiple Occluding People by Localizing on Multiple Scene Planes”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume: 31, Issue: 3, March 2009
• S.M. Khan and M. Shah, “A Multi-View Approach to Tracking People in Crowded Scenes Using a Planar Homography Constraint,” Proc. Ninth European Conf. Computer Vision, 2006.
![Page 46: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/46.jpg)
Property1. Do not use color models or shape cues of individual people2. Our method of detection and occlusion resolution is based on geometrical constructs
and requires only the distinction of foreground from back-ground, which is obtained using standard background modeling techniques.
3. the core of our method is a planar homographic occupancy constraint that combines foreground likelihood information from different views to resolve occlusions and determine regions on scene planes that are occupied by people.
4. consistently warp (under homographies of the reference plane) to foreground regions in every view.
5. The reason we use foreground likelihood maps instead of binary foreground maps is to delay the thresholding step to the last possible stage.
6. multiple planes parallel to the reference plane to robustly localize scene objects.7. To track,we obtain object scene occupancies for awindow of time and stack them tog
ether, creating a space-time volume. 8. designing an energy functional that combines scene occupancy information and spati
o-temporal proximity. 9. Homographies induced by the reference plane between views are computed using SI
FT feature matches and employing the RANSAC algorithm.10. The result is that our approach is purely image based and performs fusion in the imag
e plane without requiring to go in 3D space, and thus eliminating the need for fully calibrated cameras.
![Page 47: Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節](https://reader035.vdocuments.site/reader035/viewer/2022062216/56649d005503460f949d2d59/html5/thumbnails/47.jpg)
Monocular Approach
• Methods (1) Blob tracking
(2) Shape and Color
(3) Adaboost and Particle filter
(4) ……