amfg 2015 sparse coding trees with application to...
TRANSCRIPT
Sparse Coding Trees with Application to Emotion Classification
Kevin H.C. ChenMarcus Z. ComiterH. T. KungBradley McDanel
AMFG 2015
Harvard University
Business Applications
User feedback systems, advertising, security systems
Politics
Automatic derivation of voter preferences, focus group testing
Medicine
Additional metrics for patient care, helping children with autism
Application Motivation
Emotion Classification for IoT and Beyond
Methodology Motivation
Machine Learning and Unsupervised Feature Extraction
Feature 1
Feat
ure
2
- Sparse coding makes data more linearly separable
- Labels are not required
Sparse Coding Pipeline for Classification
Sparse Coding
Classifier (e.g., SVM)
Transform data into feature representation
Prediction with simple classifiers such as SVM
x
z
label
Unsupervised
Supervised
Representation by Sparse Coding
argminz || x - Dz|| 2 + λ|| z|| 02
Express the input signal (x) as the weighted (z) sum of a few features (D)
Note: we can also penalize L1 norm instead of L0 norm
Dictionary Learning
argminD,Z || X - DZ|| 2 + λ|| Z || 0
Sparse Coefficients
- Finds common patterns in training data- Solved by alternating updates of D and Z
2
Our Enhancement to SC
Sparse Coding tree (SC-tree)to learn features with hierarchy
Non-negative constraintsto mitigate over-fitting in SC
Mirroringto increase variation tolerance
Sparse Coding TreeLearning Features for Hard Cases
- Some discriminating features can be subtle- Finding clusters within clusters, similar to how
hierarchical k-means works
Fear can be confused with happiness because they both display teeth
Constructing Sparse Coding Tree
Input
Sparse Coding
Classifier (e.g., SVM)
Group/Label Assignment
label
label label
If certain classes get confused consistently, put them through another layer of feature extraction
anger
contempt
sadness
disgust
fear
happiness
surprise
Branching in Sparse Coding TreeBased on the confusion matrix from the coarse predictor
Features Learned in SC-tree
Features learned in the root node
Features learned in a happy v.s. fear node
happy
Could be happy or fear
fear
Input
label label
label
maxpoolingsplit
Sparse coding(LASSO/OMP)
flip
Sparse coding(LASSO/OMP)
Mirroring for Reflection InvarianceUsing max pooling to capture the horizontal symmetry inherent in emotion classification
A reflected image would get the exact same representation
Improved Robustness with MirroringWith max pooling, we always pick up response from the side of face with stronger features
Nonnegative Sparse Coding
argminz || x - Dz|| 2 + λ|| z|| 02
s.t. D ≥ 0, z ≥ 0D with NN-constraintD without NN-constraint
Tends to learn regional components
Nonnegativity prevents cancelation of components, and therefore mitigates over-fitting
Datasets
Cohn-Kanade Extended Dataset (CK+)Emotions in the Wild Dataset (EitW)GENKI-4K DatasetAM-FED Dataset
CK+
GENKI
AM-FED
Multi-classMulti-class
BinaryBinary
after pre-processingoriginal data
Performance on Emotion Classification
Results reported in average recall
The sparse coding tree improves the performance of our pipeline consistently.
79.9
SC NNSC MNNSC
75.1
70.1
73.6
71.5
76.8
w/SC Tree
w/o SC Tree
CK+ dataset
Performance on Emotion Classification
Results reported in average recall
The sparse coding tree improves the performance of our pipeline consistently.
33.0
SC NNSC MNNSC
29.4
26.5
28.628.1
29.7
w/SC Tree
w/o SC Tree
EitW dataset
MNNSC Performance
Results reported in area under curve
with Mirroring and the non-negativity constraint, even greedy methods like OMP (L0) can be competitive
90.0
L0-min L1-min
best reported96.1
L0-min L1-min
best reported95.1
96.792.396.2
95.7
93.1
91.2
89.7
92.1
88.8
86.0
97.0
sparse coding
Non-negativity
Mirroring
GENKI-4K AM-FED
Applying Sparse Coding Tree to Action Recognition
Tested on KTH dataset
with SC Tree 92.13 %without: 86.57 %
ConclusionSparse coding, as an effective feature extraction method, can be enhanced by these techniques:
Sparse Coding tree (SC-tree)to learn features with hierarchy
Non-negative constraintsto mitigate over-fitting in SC
Mirroringto increase variation tolerance