learning collections of parts for object recognition and transfer learning university of illinois at...
TRANSCRIPT
Learning Collections of Parts for Object Recognition and Transfer
Learning
University of Illinois at Urbana-Champaign
Flexible Part Based Model
• Why?– Interested in learning a large number of object
categories– Avoid learning new category from scratch when
useful information can be borrowed from other categories
Flexible Part Based Model: Objectives
• Simple to train– Minimal manual initialization effort– Train each part independently– Simple spatial model
3How Can We Adapt Existing Part Models to New Categories?
Boosted Collections of Parts
4
• Simple to train– Minimal manual initialization effort– Train each part independently– Simple spatial model
ECCV 2010
How Can We Adapt Existing Part Models to New Categories?
Part Refinement
Retrain with new examples
Train Part Detector
Collect Consistent PositivesInitialize with
Single Exemplar
5
• Compute expected part position from exemplar:
• Transfer to other examples:
Encouraging Spatial Consistency
6How Can We Adapt Existing Part Models to New Categories?
• Only allow candidates with sufficient overlap with expected position
Encouraging Consistency
7
GOODBAD
How Can We Adapt Existing Part Models to New Categories?
Learned Part Models
8How Can We Adapt Existing Part Models to New Categories?
Learned Part Models
9How Can We Adapt Existing Part Models to New Categories?
Part Evaluation: Discrimination
1. How discriminative are our parts? (mean AP)Plane Bike Boat Cat Dog Sofa
Exemplar 15.2 17.4 3.5 23.6 18.1 6.6
Refined: All-in 36.5 39.7 4.0 42.3 25.8 8.0
Selective: Appearance 38.1 39.9 5.7 46.5 29.5 8.3
Selective: App.+Spatial 37.3 37.2 4.6 39.5 24.4 8.7
Part Evaluation: Spatial Consistency
1. How discriminative are our parts? (mean AP)2. How well can we localize Poselet Keypoint
annotations? (mean best AP per keypoint type)Plane Bike Boat Cat Dog Sofa
Exemplar 14.1 34.6 12.4 12.8 8.9 9.1
Refined: All-in 21.3 41.3 9.6 22.0 12.9 7.2
Selective: Appearance 23.9 41.6 13.9 22.5 14.7 11.1
Selective: App.+Spatial 27.3 42.4 14.8 22.2 13.3 10.8
Pooling Part Detections
Propose 500 candidate object regions per image(Endres and Hoiem 2010)
Pooling Part Detections
Collect highest scoring response for each part:
Pooling Part Detections
Collect highest scoring response for each part:
Scoring Object Candidates
Classify vector of scores using boosted classifier
Relocalization
Loose spatial model: Good parts can be assigned to bad regions
Relocalization
Loose spatial model: Good parts can be assigned to bad regions
Solution 1: •Region shape features to down-weight bad regions•HOG of segmentation mask
Relocalization
Loose spatial model: Good parts can be assigned to bad regions
Solution 2: •Use parts to repredict bounding box•Each part votes for box•Weighted average based on appearance score and learned reliability
Results:Beating state of the art
Our ModelFelzenszwalb et al.
Aeroplane44.3 -> 48.4 AP
Cat24.1 -> 36.9 AP
Dog8.5 -> 20.9 AP
Results:Competitive with state of the art
Bicycle49.6 -> 43.0 AP
Boat6.7 -> 5.0 AP
Sofa17.2 -> 14.1 AP
Our ModelFelzenszwalb et al.
Conclusion
• Goal: Recognition systems that can give as much detail about any object they encounter
• Consider supervised tasks that generalize across categories
• Capture shared similarities across categories and differences within categories
25