“poof: part based one-vs-one features for fine grained categorization, face verification, and...
TRANSCRIPT
![Page 1: “POOF: Part Based One-vs-One Features for Fine Grained Categorization, Face Verification, and Attribute Estimation” Thomas Berg and Peter Belhumeur CVPR](https://reader036.vdocuments.site/reader036/viewer/2022062511/551ae16055034606048b597f/html5/thumbnails/1.jpg)
“POOF: Part Based One-vs-One Features for Fine Grained Categorization, Face Verification, and Attribute Estimation”
Thomas Berg and Peter Belhumeur
CVPR 2013
VGG Reading Group 4.7.2013
Eric Sommerlade
![Page 2: “POOF: Part Based One-vs-One Features for Fine Grained Categorization, Face Verification, and Attribute Estimation” Thomas Berg and Peter Belhumeur CVPR](https://reader036.vdocuments.site/reader036/viewer/2022062511/551ae16055034606048b597f/html5/thumbnails/2.jpg)
Summary
• A POOF is a scalar defined – for a discriminative region between two classes and
two landmarks– for a set of base features (e.g. HOG or colour hist.)
• Perks:– Regions automatically learned from data set– Great Performance– transfers in knowledge from external datasets
![Page 3: “POOF: Part Based One-vs-One Features for Fine Grained Categorization, Face Verification, and Attribute Estimation” Thomas Berg and Peter Belhumeur CVPR](https://reader036.vdocuments.site/reader036/viewer/2022062511/551ae16055034606048b597f/html5/thumbnails/3.jpg)
Motivation:
Standard approach to part based recognition:
- extract standard feature (SIFT, HOG, LBP)
- train classifier
- relevant regions tuned by hand
Idea: “standard” features hardly optimal for specific problem
“best” according to
- domain (dog features != bird features)
- task (face recognition != gender classification)
![Page 4: “POOF: Part Based One-vs-One Features for Fine Grained Categorization, Face Verification, and Attribute Estimation” Thomas Berg and Peter Belhumeur CVPR](https://reader036.vdocuments.site/reader036/viewer/2022062511/551ae16055034606048b597f/html5/thumbnails/4.jpg)
POOF feature learning:
• From dataset with landmark annotations
![Page 5: “POOF: Part Based One-vs-One Features for Fine Grained Categorization, Face Verification, and Attribute Estimation” Thomas Berg and Peter Belhumeur CVPR](https://reader036.vdocuments.site/reader036/viewer/2022062511/551ae16055034606048b597f/html5/thumbnails/5.jpg)
POOF feature learning:
• Choose feature part f
• Choose alignment part a
• Align and crop to 128x64 region
• Larger/shorter distance -> coarser/finer scale
![Page 6: “POOF: Part Based One-vs-One Features for Fine Grained Categorization, Face Verification, and Attribute Estimation” Thomas Berg and Peter Belhumeur CVPR](https://reader036.vdocuments.site/reader036/viewer/2022062511/551ae16055034606048b597f/html5/thumbnails/6.jpg)
POOF feature learning:
• Scales: 8x8 and 16x16
8*16 + 4*8 = 160 cells
![Page 7: “POOF: Part Based One-vs-One Features for Fine Grained Categorization, Face Verification, and Attribute Estimation” Thomas Berg and Peter Belhumeur CVPR](https://reader036.vdocuments.site/reader036/viewer/2022062511/551ae16055034606048b597f/html5/thumbnails/7.jpg)
POOF feature learning:
• Per cell:– 8 bin gradient direction histogram Dg=8 (‘gradhist’)– Or Felsenszwalb HOG: Dg=31– Color histogram Dc=32
• Concatenated length (Dg+Dc)*160
![Page 8: “POOF: Part Based One-vs-One Features for Fine Grained Categorization, Face Verification, and Attribute Estimation” Thomas Berg and Peter Belhumeur CVPR](https://reader036.vdocuments.site/reader036/viewer/2022062511/551ae16055034606048b597f/html5/thumbnails/8.jpg)
POOF feature learning:• For each scale (8x8, 16x16):
– learn linear SVM, get weights w
• Keep max abs(w_i) per cell• Keep cells with max(w_c)>=median(max(w_c))• keep connected component (4?) starting at f
c1 c2 cn
c1 c2 cn
c1 c2 cn
W:
max:
threshold:
…
…
…
![Page 9: “POOF: Part Based One-vs-One Features for Fine Grained Categorization, Face Verification, and Attribute Estimation” Thomas Berg and Peter Belhumeur CVPR](https://reader036.vdocuments.site/reader036/viewer/2022062511/551ae16055034606048b597f/html5/thumbnails/9.jpg)
POOF feature learning:• retrain SVM on selected cells only
• Get POOF (bitmap+svm weight vector):
![Page 10: “POOF: Part Based One-vs-One Features for Fine Grained Categorization, Face Verification, and Attribute Estimation” Thomas Berg and Peter Belhumeur CVPR](https://reader036.vdocuments.site/reader036/viewer/2022062511/551ae16055034606048b597f/html5/thumbnails/10.jpg)
POOF feature extraction:
• Find corresponding landmarks – Authors use
Belhumeur CVPR 2011
• Align & crop to 128x64 region
• Get base features
• Get SVM score from features in masked region
![Page 11: “POOF: Part Based One-vs-One Features for Fine Grained Categorization, Face Verification, and Attribute Estimation” Thomas Berg and Peter Belhumeur CVPR](https://reader036.vdocuments.site/reader036/viewer/2022062511/551ae16055034606048b597f/html5/thumbnails/11.jpg)
Results: categorization
• UCSD birds dataset, 200 classes
• 13 landmarks used
• About 5m POOF combinations possible
• Randomly chosen subset of 5000 POOFs
• Use as feature vector in one-vs-all linear SVM
• Evaluation on gt bbox of object– gt landmarks– or detected landmarks
![Page 12: “POOF: Part Based One-vs-One Features for Fine Grained Categorization, Face Verification, and Attribute Estimation” Thomas Berg and Peter Belhumeur CVPR](https://reader036.vdocuments.site/reader036/viewer/2022062511/551ae16055034606048b597f/html5/thumbnails/12.jpg)
Results: categorization
![Page 13: “POOF: Part Based One-vs-One Features for Fine Grained Categorization, Face Verification, and Attribute Estimation” Thomas Berg and Peter Belhumeur CVPR](https://reader036.vdocuments.site/reader036/viewer/2022062511/551ae16055034606048b597f/html5/thumbnails/13.jpg)
Results: categorization
gradhist HOG lowlevelbaseline [27]
[4] (MKL)
[33] (RF) [32] [8] [35]
200det 54 56 28
14det 65 70 57
200gt 69 73 40 17 19 19
14gt 80 85 44
5det 55
![Page 14: “POOF: Part Based One-vs-One Features for Fine Grained Categorization, Face Verification, and Attribute Estimation” Thomas Berg and Peter Belhumeur CVPR](https://reader036.vdocuments.site/reader036/viewer/2022062511/551ae16055034606048b597f/html5/thumbnails/14.jpg)
Results: Face Verification
• Are two images of the same person?
• LFW dataset
• 16 landmarks
• 120 subjects
• ~3.5m POOF choices
• Each image yields 10000 random POOFs f(I)
• For image pair concat [|f(I)-f(J)| f(I).*f(J)]
• Train same-vs-different classifier
![Page 15: “POOF: Part Based One-vs-One Features for Fine Grained Categorization, Face Verification, and Attribute Estimation” Thomas Berg and Peter Belhumeur CVPR](https://reader036.vdocuments.site/reader036/viewer/2022062511/551ae16055034606048b597f/html5/thumbnails/15.jpg)
Results: Face Verification
![Page 16: “POOF: Part Based One-vs-One Features for Fine Grained Categorization, Face Verification, and Attribute Estimation” Thomas Berg and Peter Belhumeur CVPR](https://reader036.vdocuments.site/reader036/viewer/2022062511/551ae16055034606048b597f/html5/thumbnails/16.jpg)
Results: Face Verification
• Performance equal to Tom-vs-Pete (bmvc2012)
• But: – Support regions learned automatically– Linear SVM, not RBF faster– Uses same “identity preserving alignment” on
landmark detections [2]
input affine canonical Mean of all closest in dataset
![Page 17: “POOF: Part Based One-vs-One Features for Fine Grained Categorization, Face Verification, and Attribute Estimation” Thomas Berg and Peter Belhumeur CVPR](https://reader036.vdocuments.site/reader036/viewer/2022062511/551ae16055034606048b597f/html5/thumbnails/17.jpg)
Results: Attribute classification
• Attributes such as gender, “big nose”, “eyeglasses” (Kumar [14])
• POOFs learned as before, on LFW dataset
• Extracts POOFs from attribute dataset
• Train linear SVM for each attribute
• POOFs transfer discriminability from different classes no need for fully labelled attribute dataset
![Page 18: “POOF: Part Based One-vs-One Features for Fine Grained Categorization, Face Verification, and Attribute Estimation” Thomas Berg and Peter Belhumeur CVPR](https://reader036.vdocuments.site/reader036/viewer/2022062511/551ae16055034606048b597f/html5/thumbnails/18.jpg)
Results: Attribute classification
• Restricted number of attribute samples
• POOF features don’t latch on to noise
…