ke chen 1, shaogang gong 1, tao xiang 1, chen change loy 2 1. queen mary, university of london 2....
TRANSCRIPT
![Page 1: KE CHEN 1, SHAOGANG GONG 1, TAO XIANG 1, CHEN CHANGE LOY 2 1. QUEEN MARY, UNIVERSITY OF LONDON 2. THE CHINESE UNIVERSITY OF HONG KONG CUMULATIVE ATTRIBUTE](https://reader034.vdocuments.site/reader034/viewer/2022051316/56649ca15503460f9496054a/html5/thumbnails/1.jpg)
K E C H E N 1 , S H A O G A N G G O N G 1 , T A O X I A N G 1 , C H E N C H A N G E L O Y 2
1 . Q U E E N M A R Y , U N I V E R S I T Y O F L O N D O N2 . T H E C H I N E S E U N I V E R S I T Y O F H O N G K O N G
CUMULATIVE ATTRIBUTE SPACE FOR AGE AND CROWD DENSITY ESTIMATION
CVPR 2013, Portland, Oregon
![Page 2: KE CHEN 1, SHAOGANG GONG 1, TAO XIANG 1, CHEN CHANGE LOY 2 1. QUEEN MARY, UNIVERSITY OF LONDON 2. THE CHINESE UNIVERSITY OF HONG KONG CUMULATIVE ATTRIBUTE](https://reader034.vdocuments.site/reader034/viewer/2022051316/56649ca15503460f9496054a/html5/thumbnails/2.jpg)
PROBLEMS
How old are they?
How many persons are in the scene?
What is the head pose (viewing angles) of this person?
![Page 3: KE CHEN 1, SHAOGANG GONG 1, TAO XIANG 1, CHEN CHANGE LOY 2 1. QUEEN MARY, UNIVERSITY OF LONDON 2. THE CHINESE UNIVERSITY OF HONG KONG CUMULATIVE ATTRIBUTE](https://reader034.vdocuments.site/reader034/viewer/2022051316/56649ca15503460f9496054a/html5/thumbnails/3.jpg)
A REGRESSION FORMULATION
Original images/frames
Facial images
Crowd frames
AAM feature
Segment feature
Edge feature
Texture feature
Feature extraction
Feature space Label space
LabelsLearning the mapping
Regression
![Page 4: KE CHEN 1, SHAOGANG GONG 1, TAO XIANG 1, CHEN CHANGE LOY 2 1. QUEEN MARY, UNIVERSITY OF LONDON 2. THE CHINESE UNIVERSITY OF HONG KONG CUMULATIVE ATTRIBUTE](https://reader034.vdocuments.site/reader034/viewer/2022051316/56649ca15503460f9496054a/html5/thumbnails/4.jpg)
CHALLENGE – FEATURE VARIATION
The same age
Extrinsic conditions: Lighting conditions; Viewing angles Intrinsic conditions: aging process of different people glasses, hairstyle, gender, ethnicity
Feature
![Page 5: KE CHEN 1, SHAOGANG GONG 1, TAO XIANG 1, CHEN CHANGE LOY 2 1. QUEEN MARY, UNIVERSITY OF LONDON 2. THE CHINESE UNIVERSITY OF HONG KONG CUMULATIVE ATTRIBUTE](https://reader034.vdocuments.site/reader034/viewer/2022051316/56649ca15503460f9496054a/html5/thumbnails/5.jpg)
CHALLENGE – FEATURE VARIATION
The same person count
Extrinsic conditions: Lighting conditions; Viewing angles Intrinsic conditions: occlusion, density distribution in the scene
Feature
![Page 6: KE CHEN 1, SHAOGANG GONG 1, TAO XIANG 1, CHEN CHANGE LOY 2 1. QUEEN MARY, UNIVERSITY OF LONDON 2. THE CHINESE UNIVERSITY OF HONG KONG CUMULATIVE ATTRIBUTE](https://reader034.vdocuments.site/reader034/viewer/2022051316/56649ca15503460f9496054a/html5/thumbnails/6.jpg)
CHALLENGE – SPARSE AND IMBALANCED DATA
Data distribution of FG-NET Dataset
Max number of samples for each age group is 46
![Page 7: KE CHEN 1, SHAOGANG GONG 1, TAO XIANG 1, CHEN CHANGE LOY 2 1. QUEEN MARY, UNIVERSITY OF LONDON 2. THE CHINESE UNIVERSITY OF HONG KONG CUMULATIVE ATTRIBUTE](https://reader034.vdocuments.site/reader034/viewer/2022051316/56649ca15503460f9496054a/html5/thumbnails/7.jpg)
CHALLENGE – SPARSE AND IMBALANCED DATA
Data distribution of UCSD Dataset
![Page 8: KE CHEN 1, SHAOGANG GONG 1, TAO XIANG 1, CHEN CHANGE LOY 2 1. QUEEN MARY, UNIVERSITY OF LONDON 2. THE CHINESE UNIVERSITY OF HONG KONG CUMULATIVE ATTRIBUTE](https://reader034.vdocuments.site/reader034/viewer/2022051316/56649ca15503460f9496054a/html5/thumbnails/8.jpg)
RELATED WORKS
• Most focused on feature variation challenge
• Few focused on sparse and imbalanced data challenge
• Two challenges are related
1. Improve feature robustness [Guo et al, CVPR, 2009; Guo et al, TIP, 2012; Ryan et al, DICTA, 2009; Zhang et al, IEEE T ITS, 2011].
2. Improve regressor
[Guo et al, TIP 2008; Chang et al, CVPR 2011; Chao et al, PR 2013; Chan et al, CVPR 2008; Chen et al, BMVC 2012]
![Page 9: KE CHEN 1, SHAOGANG GONG 1, TAO XIANG 1, CHEN CHANGE LOY 2 1. QUEEN MARY, UNIVERSITY OF LONDON 2. THE CHINESE UNIVERSITY OF HONG KONG CUMULATIVE ATTRIBUTE](https://reader034.vdocuments.site/reader034/viewer/2022051316/56649ca15503460f9496054a/html5/thumbnails/9.jpg)
OUR APPROACH
Solution:• Attribute Learning can address data sparsity problem
--Exploits the shared characteristics between classesHas sematic meaningDiscriminative
Problems:• Applied successfully in classification but not in
regression• How to exploit cumulative dependent nature of
labels in regression?…… …… ……
Age 20 Age 21 Age 60
![Page 10: KE CHEN 1, SHAOGANG GONG 1, TAO XIANG 1, CHEN CHANGE LOY 2 1. QUEEN MARY, UNIVERSITY OF LONDON 2. THE CHINESE UNIVERSITY OF HONG KONG CUMULATIVE ATTRIBUTE](https://reader034.vdocuments.site/reader034/viewer/2022051316/56649ca15503460f9496054a/html5/thumbnails/10.jpg)
CUMULATIVE ATTRIBUTE
Age 20
1
1
0
1
…20
0…0
the rest
Cumulative attribute (dependent)
Vs.
0
1
…
20th
0…
0
Non-cumulative attribute (independent)
0
0
![Page 11: KE CHEN 1, SHAOGANG GONG 1, TAO XIANG 1, CHEN CHANGE LOY 2 1. QUEEN MARY, UNIVERSITY OF LONDON 2. THE CHINESE UNIVERSITY OF HONG KONG CUMULATIVE ATTRIBUTE](https://reader034.vdocuments.site/reader034/viewer/2022051316/56649ca15503460f9496054a/html5/thumbnails/11.jpg)
LIMITATION OF NON-CUMULATIVE ATTRIBUTE
Age 200
1
…
20th
0
…0
Age 60
60th0
…
0
0
0
1
…
0
…
0
0…0
0
21st
0
1
…
0
…
0
0
0
0
Age 21
![Page 12: KE CHEN 1, SHAOGANG GONG 1, TAO XIANG 1, CHEN CHANGE LOY 2 1. QUEEN MARY, UNIVERSITY OF LONDON 2. THE CHINESE UNIVERSITY OF HONG KONG CUMULATIVE ATTRIBUTE](https://reader034.vdocuments.site/reader034/viewer/2022051316/56649ca15503460f9496054a/html5/thumbnails/12.jpg)
Age 21
ADVANTAGES OF CUMULATIVE ATTRIBUTE
Age 20
1
1
0
1
…20
0
…0
the rest
Age 60
1
1
1
…60
0…
0
1
0
… 1…1 attribute changes
1
1…21
0
…
0
1
1
0
40 attributes change
![Page 13: KE CHEN 1, SHAOGANG GONG 1, TAO XIANG 1, CHEN CHANGE LOY 2 1. QUEEN MARY, UNIVERSITY OF LONDON 2. THE CHINESE UNIVERSITY OF HONG KONG CUMULATIVE ATTRIBUTE](https://reader034.vdocuments.site/reader034/viewer/2022051316/56649ca15503460f9496054a/html5/thumbnails/13.jpg)
OUR FRAMEWORK
Imagery Features xi
Facial images Crowd frames
Labels yi
Regression Learning
Cumulative Attributes ai
Feature Extraction
Multi-output Regression Learning
Regression Mapping
Conventional frameworks
1 1 0 0… …1
1 2 yi yi+1 N
![Page 14: KE CHEN 1, SHAOGANG GONG 1, TAO XIANG 1, CHEN CHANGE LOY 2 1. QUEEN MARY, UNIVERSITY OF LONDON 2. THE CHINESE UNIVERSITY OF HONG KONG CUMULATIVE ATTRIBUTE](https://reader034.vdocuments.site/reader034/viewer/2022051316/56649ca15503460f9496054a/html5/thumbnails/14.jpg)
JOINT ATTRIBUTE LEARNING
• Joint Attribute Learning
with quadratic loss function
• Regression Learning with attribute representation as input is not limited to a specific regression model
min12‖𝐖‖ 2
𝐹+𝐶∑𝑖=1
N
‖𝐚𝑖𝑇−(𝐱 𝑖
𝑇𝐖+𝐛)‖ 2𝐹
min12‖𝐰 𝑗‖ 2
2+𝐶∑𝑖=1
N
𝑙𝑜𝑠𝑠(𝑎𝑖𝑗 , 𝑓 𝑗 (𝐱 𝑖))¿¿
![Page 15: KE CHEN 1, SHAOGANG GONG 1, TAO XIANG 1, CHEN CHANGE LOY 2 1. QUEEN MARY, UNIVERSITY OF LONDON 2. THE CHINESE UNIVERSITY OF HONG KONG CUMULATIVE ATTRIBUTE](https://reader034.vdocuments.site/reader034/viewer/2022051316/56649ca15503460f9496054a/html5/thumbnails/15.jpg)
COMPARATIVE EVALUATION
Age Estimation
CA-SVR: our method; AGES: Geng et al, TPAMI, 2007; RUN: Yan et al, ICCV, 2007; Ranking: Yan et al, ICME, 2007; RED-SVM: Chang et al, ICPR, 2010; LARR: Guo et al, TIP, 2008; MTWGP: Zhang et al, CVPR, 2010; OHRank: Chang et al, CVPR, 2011; SVR: Guo et al, TIP, 2008;
![Page 16: KE CHEN 1, SHAOGANG GONG 1, TAO XIANG 1, CHEN CHANGE LOY 2 1. QUEEN MARY, UNIVERSITY OF LONDON 2. THE CHINESE UNIVERSITY OF HONG KONG CUMULATIVE ATTRIBUTE](https://reader034.vdocuments.site/reader034/viewer/2022051316/56649ca15503460f9496054a/html5/thumbnails/16.jpg)
COMPARATIVE EVALUATION
Crowd Counting
CA-RR: our method; LSSVR: Suykens et al, IJCNN, 2001; KRR: An et al, CVPR, 2007; RFR: Liaw et al, R News, 2002; GPR: Chan et al, CVPR, 2008; RR: Chen et al, BMVC, 2012;
![Page 17: KE CHEN 1, SHAOGANG GONG 1, TAO XIANG 1, CHEN CHANGE LOY 2 1. QUEEN MARY, UNIVERSITY OF LONDON 2. THE CHINESE UNIVERSITY OF HONG KONG CUMULATIVE ATTRIBUTE](https://reader034.vdocuments.site/reader034/viewer/2022051316/56649ca15503460f9496054a/html5/thumbnails/17.jpg)
CUMULATIVE (CA) VS. NON-CUMULATIVE (NCA)
Crowd Counting
Age Estimation
![Page 18: KE CHEN 1, SHAOGANG GONG 1, TAO XIANG 1, CHEN CHANGE LOY 2 1. QUEEN MARY, UNIVERSITY OF LONDON 2. THE CHINESE UNIVERSITY OF HONG KONG CUMULATIVE ATTRIBUTE](https://reader034.vdocuments.site/reader034/viewer/2022051316/56649ca15503460f9496054a/html5/thumbnails/18.jpg)
ROBUSTNESS AGAINST SPARSE AND IMBALANCED DATA
Age Estimation
Crowd Counting
![Page 19: KE CHEN 1, SHAOGANG GONG 1, TAO XIANG 1, CHEN CHANGE LOY 2 1. QUEEN MARY, UNIVERSITY OF LONDON 2. THE CHINESE UNIVERSITY OF HONG KONG CUMULATIVE ATTRIBUTE](https://reader034.vdocuments.site/reader034/viewer/2022051316/56649ca15503460f9496054a/html5/thumbnails/19.jpg)
FEATURE SELECTION BY ATTRIBUTES
Shape plays a more important role than texture when one is younger.
![Page 20: KE CHEN 1, SHAOGANG GONG 1, TAO XIANG 1, CHEN CHANGE LOY 2 1. QUEEN MARY, UNIVERSITY OF LONDON 2. THE CHINESE UNIVERSITY OF HONG KONG CUMULATIVE ATTRIBUTE](https://reader034.vdocuments.site/reader034/viewer/2022051316/56649ca15503460f9496054a/html5/thumbnails/20.jpg)
CONCLUSION
• A novel attribute framework for regression
• Exploits cumulative dependent nature of label space
• Effectively addresses sparse and imbalanced data problem
![Page 21: KE CHEN 1, SHAOGANG GONG 1, TAO XIANG 1, CHEN CHANGE LOY 2 1. QUEEN MARY, UNIVERSITY OF LONDON 2. THE CHINESE UNIVERSITY OF HONG KONG CUMULATIVE ATTRIBUTE](https://reader034.vdocuments.site/reader034/viewer/2022051316/56649ca15503460f9496054a/html5/thumbnails/21.jpg)
Thanks a lot for your attention! Any questions?
Welcome to our poster 3A-2 for more details.
Ke Chen Shaogang Gong Tao Xiang Chen Change Loy Ph.D student Professor Associate Professor Assistant Professor