![Page 1: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/1.jpg)
Regionlet Object Detector with Hand-crafted and CNN Feature
Xiaoyu Wang Snapchat Research
Ming Yang Horizon Robotics
Shenghuo Zhu Alibaba Group
Yuanqing Lin Baidu
Xiaoyu Wang Snapchat Research
![Page 2: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/2.jpg)
Snapchat Overview of this section
• Regionlet Object Detector • Regionlet Localizer (re-localization) • Regionlet with Deep CNN Feature
• CNN Feature Extraction • Support Pixel Integral Image
• Application Examples • Car Detection for Fine-grained Image Classification • Pedestrian, Car, Cyclist Detection for Autonomous Driving
![Page 3: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/3.jpg)
Snapchat What is Regionlet Object Detector
• A significant extension to traditional boosting object detector
• Together with OverFeat and R-CNN, the Regionlet detector is one of the first several detectors that successfully adopt deep CNN features for generic object detection.
![Page 4: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/4.jpg)
Snapchat How does Regionlet detector connect to past/future
2013 Past future
Boosting Feature Selection
Object Proposal
Generalized Spatial Pyramid for CNN Feature Pooling
Spatial Pyramid Pooling in SPP-Net4
RoI Pooling in Fast R-CNN5
RealBoost1
Segmentation as Selective Search2
Low-level Feature
Deep CNN3
2. K. E. A. Van de Sande, et. al. Segmentation as selective search for object recognition. ICCV 2011
1. C. Huang, et. al. Boosting nested cascade detector for multi-view face detection. ICPR, 2004.
CNN-based Object Detection
3. Krizhevsky, et. al. ImageNet Classification with Deep Convolutional Neural Networks. NIPS 2012 4. He, et. al. Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition. ECCV 2014 5. Ross Girshick. Fast R-CNN. ICCV 2015
![Page 5: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/5.jpg)
Snapchat Boosting Object Detector
𝒙𝟏
𝒙𝟐 𝒙𝑵
𝑓 𝑋 = �𝛽𝑖ℎ𝑖(𝑥𝑖)𝑁
𝑖=0
A detection window
Weak classifier
A sub-region where weak classifier is built based on
![Page 6: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/6.jpg)
Snapchat Traditional Boosting Detection Framework
Operate on multiple scales to detect objects in different scales
Model 1
Model 2
Use multiple components to detect objects with various aspect ratios
How about a single model, but flexible during testing, no feature
pyramids, no multiple components
![Page 7: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/7.jpg)
Snapchat
• A boosting classifier that can take inputs of different scales
• A boosting classifier that can take inputs of different viewpoints
• A boosting classifier containing feature pooling learning
What the Regionlet Detector Proposed
![Page 8: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/8.jpg)
Snapchat Regionlet: Definition
• Region(𝑅): Feature extraction region
• Regionlet(𝑟1, 𝑟2, 𝑟3): A sub-region in a feature extraction area whose position/resolution are relative and normalized to a detection window
Region
Regionlet
![Page 9: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/9.jpg)
Snapchat Regionlet: Definition(cont.) • Regionlet coordinates are normalized
𝑤
ℎ
(𝑙, 𝑡, 𝑟, 𝑏) (50,50,180,180)
𝑙𝑤
,𝑡ℎ
,𝑟𝑤
,𝑏ℎ
(.25, .25, .90,.90)
Traditional
Normalized
(50,50,180,180)
(.25, .25, .90,.90)
![Page 10: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/10.jpg)
Snapchat Regionlet: Definition(cont.) • Regionlet definition = Generalized Spatial Pyramid
• Similar • Both use relative coordinates
• Difference • Regionlet: coordinates are relative to the detection window (not the image) • Regionlet: coordinates are flexible (do not have to evenly divide the image/window)
• Regionlet feature extraction = Generalized Spatial Pyramid Pooling
Rectangles in Spatial Pyramid Rectangles in Generalized Spatial Pyramid
![Page 11: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/11.jpg)
Snapchat Connection to other methods in pooling design
Object Proposal
Generalized Spatial Pyramid for CNN Feature Pooling
Spatial Pyramid Pooling in SPP-Net1
RoI Pooling in Fast R-CNN2
CNN-based Object Detection
1. He, et. al. Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition. ECCV 2014 2. Ross Girshick. Fast R-CNN. ICCV 2015
![Page 12: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/12.jpg)
Snapchat Regionlet: Feature extraction
Could be Hand-crafted features or deep CNN features, whatever
feature your like!
Non-local pooling
![Page 13: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/13.jpg)
Snapchat Regionlet Classifier
𝑥 Regionlets
Feature extraction
Feature
Weak Classifier
Strong Classifier H 𝑋 = �β𝑖
𝑇
𝑖=1
ℎ𝑖(𝑥𝑖)
• Each weak classifier is based on a 1-D feature extracted from a region
ℎ 𝑥 = �𝑣𝑜𝟙 𝐵 𝑥 = 0𝑛−1
𝑜=1
![Page 14: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/14.jpg)
Snapchat Detection Framework
1. K. E. A. Van de Sande, et. al. Segmentation as selective search for object recognition. ICCV 2011 2. B. Alexe , et. al. Measuring the objectness of image windows. T-PAMI 2012
(a) : Input image (b) : Generate object regions1,2,3
(c) : Feature extraction and pooling Generalized Spatial Pyramid Pooling inside Regionget
Low-level features CNN features (will talk later)
Max-pooling among Regionlets
Regionlet
Region
(a) (b) (c)
3. S. Ren, et. al. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. NIPS 2015
![Page 15: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/15.jpg)
Snapchat Multiple scale & viewpoints Handling Not a
motorbike
Adjusting the model to a candidate bounding box Adjusting the model to a candidate bounding box
Regionlet Model
![Page 16: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/16.jpg)
Snapchat Multiple scale & viewpoints Handling
Motorbike Detected
Adjusting the model to a candidate bounding box Adjusting the model to a candidate bounding box
Regionlet Model
![Page 17: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/17.jpg)
Snapchat Weak Classifier Construction
• Weak learner on each REGION • Eight lots lookup table • Lookup table is learned • Lot value is learned • One lot is activated for one feature
0.01 -0.2 -0.5 -0.4 0.02 0.15 0.5 0.3
Regionlet feature 𝑥𝑖 (after pooling)
Weak learner output: -0.5
Assign lot
H(𝑋) = �𝐿𝐿𝑇𝑖(𝑥𝑖)𝑁
𝑖=1
![Page 18: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/18.jpg)
Snapchat Regionlet Training
• How to get regions and regionlets • Regions
• Regions are randomly sampled • Effective Regions are greedily selected to reduce learning cost
• Regionlets • Each Region & Regionlet configuration are randomly configured • A Region and its regionlets configuration are selected simultaneously
• Region & Regionlet pool is fixed for each cascade learning
![Page 19: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/19.jpg)
Snapchat Regionlet: Training • Constructing the regions/regionlets pool
• Small region, fewer regionlets -> fine spatial layout • Large region, more regionlets -> robust to deformation
• Learning realBoost1 cascades • 16K region/regionlets candidates for each cascade • Learning of each cascade stops when the error rate is achieved (1% for
positive, 37.5% for negative) • Last cascade stops after collecting 5000 weak classifiers • Result in 4-7 cascades • 2-3 hours to finish training one category on a 8-core machine
1. C. Huang, et. al. Boosting nested cascade detector for multi-view face detection. ICPR, 2004.
![Page 20: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/20.jpg)
Snapchat Regionlet: Testing • No image resizing • Any scale, any aspect ratio • Adapt the model size to the same size as the object candidate
bounding box
+
+
+
One model, resize image
Multiple models, original image
Ours, One model, original image
![Page 21: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/21.jpg)
Snapchat Overview of this section
• Regionlet Object Detector • Regionlet Localizer • Regionlet with Deep CNN Feature
• CNN Feature Extraction • Support Pixel Integral Image
• Application Examples • Car Detection • Pedestrian, Car, Cyclist Detection for Autonomous Driving
![Page 22: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/22.jpg)
Snapchat Regionlet Localizer (object re-localization) • Why a localizer is needed (classification & localization precision
dilemma)
Data augmentation during
training to accommodate inaccurate localization
As accurate location as possible
during testing
VS
![Page 23: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/23.jpg)
Snapchat Regionlet Localizer • Regionlet feature can be reused for localization
• Each Regionlet feature is associated with a spatial location
• The location is learned during classifier training
![Page 24: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/24.jpg)
Snapchat Regionlet Localizer • Regionlet feature can be reused for localization
Regionlet classifier 1 Regionlet classifier N
0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0
8N dimensional binary vector
![Page 25: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/25.jpg)
Snapchat Regionlet Localizer
0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0
∆𝒍,∆𝒕,∆𝒓,∆𝒃
× 𝑾
⋯
![Page 26: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/26.jpg)
Snapchat Regionlet Localizer Training • Random sample examples which have > 0.6 overlap with ground
truth • Less overlap gives poor results
• The regression task learns the location difference
![Page 27: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/27.jpg)
Snapchat Regionlet Localizer • Experiment result on our car dataset for autonomous driving
• 17501 cars for training • 12546 cars for testing
0.5 overlap 0.7 overlap
Regionlet 62.7% 34.6%
Regionlet + localization 65.3% 43.9%
Improvement 2.6% 9.1%
Detection performance (% AP)
![Page 28: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/28.jpg)
Snapchat Overview of this section
• Regionlet Object Detector • Regionlet Localizer • Regionlet with Deep CNN Feature
• CNN Feature Extraction • Support Pixel Integral Image
• Application Examples • Car Detection • Pedestrian, Car, Cyclist Detection for Autonomous Driving
![Page 29: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/29.jpg)
Snapchat Regionlet with DCNN
• Deep CNN • Deep structure learns high-level information • Max-pooling is robust to parts misalignment • Information are jointly learned
• How to establish a bridge for DCNN and Regionlet object
detection framework?
![Page 30: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/30.jpg)
Snapchat Regionlet with DCNN • Deep CNN structure
• Features from convolution layers retain spatial information
Convolutional layers
![Page 31: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/31.jpg)
Snapchat Regionlet with DCNN • Deep CNN structure
• Features from convolution layers retain spatial information
A feature vector
![Page 32: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/32.jpg)
Snapchat Regionlets with DCNN • Deep CNN structure
• ‘image-convolution’ to generate features for the whole image
(a) Convolution kernels (b) Output the dense neural patterns
+
![Page 33: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/33.jpg)
Snapchat Support Pixel Integral Image • CNN Feature
• Not available on each pixel • Feature dimension is high for integral image
Does not change if rowsum = 0 Does not change if colsum = 0
We want dense integral feature
We want to save memory
Integral Image
Computation
![Page 34: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/34.jpg)
Snapchat Support Pixel Integral Image • Support Pixel
• Where the integral vector computation is inevitable
![Page 35: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/35.jpg)
Snapchat Support Pixel Integral Image • Support Pixel Integral Image for CNN Feature1
Support pixel integral vector (sparse) Dense virtual integral vector map
1. Wang et. al. Regionlets for Generic Object Detection. T-PAMI, 2015
![Page 36: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/36.jpg)
Snapchat Regionlet with DCNN • Deep CNN feature for detection1, 2
1. Zou et. al. Generic Object Detection with Dense Neural Patterns and Regionlets. BMVC, 2014 2. Wang et. al. Regionlets for Generic Object Detection. T-PAMI, 2015
(a) Input image (b) Densely extracted feature maps
⋯
⋯
(c) Generalized Spatial Pyramid Deep CNN Feature
Pooling
(d) Detected object bounding box
![Page 37: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/37.jpg)
Snapchat Experiment Results • Regionlet + CNN feature (No fine-tuning)
mAP Regionlet 41.7 Regionlet-CNN pool5 49.3 R-CNN pool5 44.2 R-CNN FT fc7 BB 58.5
Average precision on PASCAL VOC 2007 (%)
![Page 38: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/38.jpg)
Snapchat Experiment Results • Visualization of selected neuron patterns
![Page 39: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/39.jpg)
Snapchat Running speed
• Regionlet detector + localizer : 5 fps using a single CPU core, (>30 fps) using 8 cores
• Regionlet + CNN: 3fps using a single CPU core
![Page 40: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/40.jpg)
Snapchat Application examples
• Car, pedestrian, cyclist detection for autonomous driving • Ranked #1 on KITTI detection dataset
Moderate Easy Hard 3DVP 75.77% 87.46% 65.38% SS 74.30% 85.03% 59.48% SVM-Res 67.49% 78.11% 54.28% Regionlet 76.45% 84.75% 59.70%
Car Detection (mAP %)
![Page 41: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/41.jpg)
Snapchat Application examples
• Car, pedestrian, cyclist detection for autonomous driving • Ranked #1 on KITTI detection dataset
Moderate Easy Hard pAUCEnsT 1 54.49% 65.26% 48.60% R-CNN 50.13% 61.61% 44.79% Fusion-DPM 46.67% 59.51% 42.05% Regionlet 61.15% 73.14% 55.21%
Pedestrian Detection (mAP %)
1. S. Paisitkriangkrai, C. Shen and A. Hengel, Pedestrian Detection with Spatially Pooled Features and Structured Ensemble Learning
![Page 42: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/42.jpg)
Snapchat Application examples
• Car, pedestrian, cyclist detection for autonomous driving • Ranked #1 on KITTI detection dataset
Moderate Easy Hard pAUCEnsT 38.03% 51.62% 33.38% R-CNN 34.47% 50.07% 32.12% DF+DPM+ROI 30.90% 41.86% 27.75% Regionlet 58.72% 70.41% 51.83%
Cyclist Detection (mAP %)
![Page 43: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/43.jpg)
Snapchat Application examples • Car detection
• Totally 170K cars for fine-grained car recognition • 11000 labelled for training detector • 2745 labelled for testing • 100% AP
![Page 44: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/44.jpg)
Snapchat Take Away Messages • Regionlet extend the traditional boosting object detector
• It operates on object proposal (the generalized spatial pyramid pooling is the key)
• It integrates a pooling process • It can easily integrate various features
• Regionlet is FAST ( 5 fps, single core CPU) • Regionlet is FAST even with CNN Feature (3fps, single core
CPU)
![Page 45: Regionlet Object Detector with Hand-crafted and CNN Featuremp7.watson.ibm.com/ICCV2015/slides/Regionlets_CNN_ICCV... · 2015-12-16 · Regionlet Object Detector with Hand-crafted](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ee30047ad6a402d666d2507/html5/thumbnails/45.jpg)
Snapchat
Thank you! We are hiring!