facial expression analysis - aalborg universitet · facial expression analysis jeff cohn ......
TRANSCRIPT
![Page 1: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/1.jpg)
1Facial ExpressionAnalysis F. De la Torre/J. Cohn Looking @ People (CVPR-12)
Facial Expression Analysis
Jeff Cohn
Fernando De la Torre
Tutorial Looking @ People
June 2012
Human Sensing
Laboratory
![Page 2: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/2.jpg)
2
Outline
• Introduction
• Facial Action Coding System (FACS)
– Discrete vs. dimensional approaches
• Applications of FEA
• Databases
• Algorithms
– Supervised
– Unsupervised
• Conclusions and open problems
![Page 3: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/3.jpg)
3
Outline
• Introduction
• Facial Action Coding System (FACS)
– Discrete vs. dimensional approaches
• Applications of FEA
• Databases
• Algorithms
– Supervised
– Unsupervised
• Conclusions and open problems
![Page 4: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/4.jpg)
Supervised Facial Expression Analysis (FEA)
![Page 5: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/5.jpg)
• Most work on FEA has been supervised using
different registration, features and classifiers.
Supervised FEA (II)
2D/3D Face
tracking (AAM)
Classifiers (identity,
discriminate Aus)
Features (illumination,
identity)
Registration (remove 3D
rigid motion)
AU present?
![Page 6: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/6.jpg)
• Generative (Parameterized Appearance Models)– Active Appearance models (e.g., Cootes et al. 98, Romdhani et al. 99, De la Torre 00,
Matthews & Baker 05, De la Torre & Nguyen 08, Gong et al. 00)
– Eigentracking (e.g., Black & Jepson 98)
– Morphable models (e.g., Jones & Poggio 98, Blanz & Better 99)
• Discriminative– Regression:
• Classifier fitting (e.g., Liu 09)
• Continuous regression (e.g., Sauer et al. 11, Saragih 11)
• Cascaded regression (e.g., Dollar et al. 10, Cao et al. 12)
– Local models:• Constrained Local model (e.g., Cristanace & Cootes 08, Lucey et al. 09, Saragih et al. 10)
• Part-based model (Zhu & Ramanan 12)
Facial feature detection
![Page 7: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/7.jpg)
Procrustes
Shape modes
Shape normalised images
Hand-Labeled
Training Data
Appearance modes
B1 B2B0
Parameterized Appearance Models
![Page 8: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/8.jpg)
8
Detection as an Optimization Problem
)),0 a(f(xs...a))(f(x,s n++
Learned off-line
aa
•Translation
rotation, scale
sc0
scn
•Non-rigid
parameters
0.7 0.2
2
2
)( Bcd −
B1 B2B0
•Appearance
parameters
c
![Page 9: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/9.jpg)
Problems• Prone to local minima
• Not generalize well (e.g., different people)
(Nguyen & De la Torre 10)
![Page 10: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/10.jpg)
10
Discriminative models
)( 11
0)∆aa,)∆cS(cf(
0 ++ Sfeatures
Rigid
parameters
Non-rigid
parameters
0.7
[ ]s1
1 ca ∆∆
[ ]S2
2 ca ∆∆ )( 2
02 )∆aa,)∆cS(cf(
0 ++ sfeatures
• In general improve generalization(e.g., Liu 09, Sauer et al. 11, Saragih 11, Dollar et al. 2010, Cao et al. 2012)
![Page 11: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/11.jpg)
Discriminative models (II)• Local discriminative models
• Constrained Local model (e.g., Cristanace & Cootes 08, Lucey et al. 09, Saragih
et al. 10)
• Part-based model (Zhu & Ramanan 2012)
Thanks Saragih/Lucey
![Page 12: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/12.jpg)
Face registration• What are the three most important aspects of face recognition?
• Similarity registration (e.g., Barlett et al. 05, Whitehill et al. 11)
• Piece-wise warping (e.g., Cootes et al. 98, Gong et al. 00, Tong et al. 07, De la Torre &
Nyugen 08, Jones & Poggio, 1998, Lucey et al. 09, Saragih et al. 10)
Rotate,
scale
Piece-wise
warping
Benefits:
- Subtle AUs
- Out-of plane
rotation (3D
models)
“registration, registration, registration” (Takeo Kanade ‘90)
![Page 13: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/13.jpg)
• 3D registration (Thanks Laszlo Jeni)
Face Registration
![Page 14: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/14.jpg)
• Most work on FEA has been supervised using
different registration, features and classifiers.
Supervised FEA (II)
Face tracking
(AAM)
Classifiers Features
Registration
AU present?
![Page 15: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/15.jpg)
Features• Three types: (1) Shape, (2) Appearance, (3)
Temporal features.
(e.g., Sebe et al. 07, Asthana et al. 09,
Lucey et al., 2007; Chew et al., 2011, Zhou
et al. 10; Valstar et al. 12)
(e.g., Zhou et al. 2010)
• Shape features
![Page 16: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/16.jpg)
Appearance featuresRaw pixels
Gabor bank
Box filtersSIFT/HOG
Local binary patterns
(e.g., Kanade et al., 2000)
(e.g., Donato et al., 99;
Barlett 04, Littlewort et al.,
2006, Whitehill et al. 11)(e.g., Shan et al 09, Zhao et
al., 10 Jiang et al. 11)
(e.g., Whitehill & Omlin, 2006)(e.g., Zhu et al. 2011,
Simon et al., 2010, Dhall et al. 11)
NMF
(e.g., Zhi et al. 11,
Zafeiriou and Petrou 10)
• Warning!!: Appearance features typically need
dimensionality reduction and/or feature selection
![Page 17: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/17.jpg)
Temporal features
Motion units/trajectories
(e.g., Cohen et al., 02, Li et al. 01)
Optical flow
(e.g., Essa and Pentland 97, Gunes and Piccardi 05)
Motion history
(e.g., Valstar et al., 04,
Koelstra et al. 10)
Bag of temporal words
(e.g., Simon et al., 10)
![Page 18: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/18.jpg)
• Most work on FEA has been supervised using
different registration, features and classifiers.
Supervised FEA (II)
Face tracking
(AAM)
Classifiers Features
Registration
AU present?
![Page 19: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/19.jpg)
Classifiers• Static
– Exemplar + GMM (Wen and Huang, 2003)
– Neural Network (Kapoor and Picard, 2005)
– SVM/Adaboost (Bartlett et al., 2005)
– Linear Discriminant Classifiers (Wang et al., 2006)
– Gaussian Process (Chen et al., 2009)
– Boosting (Shan et al. 2006 , Zhu et al. 2010)
• Dynamic
– Hidden Markov models (Lien et al, 2000)
– Dynamic Bayesian Network (Tong et al., 2007)
– Conditional random field (Chang and Liu, 2009)
– Temporal Bag of Words (Simon et al. 2010)
![Page 20: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/20.jpg)
The million $ question
• Which is the best feature and classifier?
• Data
– Have access to reliable and well annotated data
– The more data the better
• Features
– It is AU dependent
– In general feature fusion is the best (e.g., multiple kernel
learning)
• Classifier
– Depends on the amount of training data
– How familiar you are with the classifier
![Page 21: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/21.jpg)
• Most work on FEA has been supervised using
different registration, features and classifiers.
Supervised FEA (II)
Face tracking
(AAM)
Classifiers Features
Registration
AU present?
![Page 22: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/22.jpg)
• Most work on FEA has been supervised using
different registration, features and classifiers.
Sample selection
… … … …
Onset OffsetPeak
Time
Intensity
AU
+ --• Make good use of the data!!! (Zhu et al 11, Simon 10)
![Page 23: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/23.jpg)
The first number between lines | denotes the area under the ROC, the second number is the size of positive samples in the testing dataset and separated by / is the size of negative samples in the testing dataset. The third number denotes the size of positive samples in training working sets and separated by / the total frames of target AU in training data sets.
Results for AU4 and AU12
![Page 24: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/24.jpg)
Bayesian networks• Bayesian networks to model spatial and temporal
relationships among different Aus (Tong et al. 05, Shang et al 07).
![Page 25: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/25.jpg)
25
Outline• Introduction
• Facial Action Coding System (FACS)
• Databases
• Applications of FEA
• Algorithms
– Supervised
– Unsupervised
• Conclusions and future work
![Page 26: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/26.jpg)
Motivation
• Mining facial expression for one subject
![Page 27: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/27.jpg)
• Mining facial expression for one subject
Motivation
• Mining facial expression for one subject
• Summarization
• Visualization
• Indexing
![Page 28: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/28.jpg)
• Mining facial expression for one subject
Looking up Sleeping SmilingLooking
forwardWaking up
Motivation
• Summarization
• Visualization
• Indexing
![Page 29: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/29.jpg)
• Mining facial expression of one subject
Motivation
• Summarization
• Embedding
• Indexing
![Page 30: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/30.jpg)
• Summarization
• Embedding
• Indexing
Mining facial expression across subjects
RU-FACS database (Bartlett et al. ’06)
![Page 31: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/31.jpg)
Aligned Cluster Analysis
3hStart and end of the segments (h)
mh 1+mh1h 2hLabels (G)
(Zhou et a. ‘10)
![Page 32: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/32.jpg)
=MGxy
Kernel k-means and spectral clustering(Ding et al. ‘02, Dhillon et al. ‘04, Zass and Shashua ‘05, De la Torre ‘06)
2||||),( FJ MGXGM −=
1
2
3
4
5
6
7
8
9
10x
y
=G
xy=X
)))((()( 1
n GGGGIKG−−= TTtrJ
)(ϕ
)()( XXK ϕϕ T=
=Mxy
=G
![Page 33: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/33.jpg)
2)(),,(
FacaJ MGXGM −= ϕ
Problem formulation for ACA
H )..[)..[)..[ 13221,...,,
+mm hhhhhh XXX
)..[ 21 hhX )..[ 32 hhX )..[ 1+mm hhX
Labels (G)3h
Start and end of the segments (h)mh 1+mh
Dynamic Time Alignment Kernel (Shimodaira et al. 01)
1h 2h 4h
2)(),,(
FkkmJ MGXGM −= ϕ
![Page 34: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/34.jpg)
Matrix formulation for ACA
GGGGILKL1
n )(with)( −−== TT
kmk trJ
samples
seg
me
nts
}{ 2371,0
×∈H
GHGGGHILWLK1
n )(with))o(( −−== TTT
aca trJ
2323×∈RW
clu
ste
rs
segments
}{ 731,0
×∈G
)()( XXK ϕϕ T=
Dynamic Time
Alignment Kernel
(Shimodaira et al. 01)
23 frames, 3 clusters
![Page 35: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/35.jpg)
Facial image features
Appearance
• Active Appearance Models (Baker and Matthews ‘04)
Upper
face
Lower
face
Shape• Image features
![Page 36: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/36.jpg)
• Cohn-Kanade: 30 people and five different expressions
(surprise, joy, sadness, fear, anger)
Facial event discovery across subjects
![Page 37: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/37.jpg)
• Cohn-Kanade: 30 people and five different expressions
(surprise, joy, sadness, fear, anger)
Facial event discovery across subjects
![Page 38: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/38.jpg)
ACA Spectral
Clustering (SC)
0.87(.05) 0.56(.04)
• Cohn-Kanade: 30 people and five different expressions
(surprise, joy, sadness, fear, anger)
Facial event discovery across subjects
• 10 sets of 30 people
![Page 39: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/39.jpg)
Unsupervised facial event discovery
![Page 40: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/40.jpg)
FACS coding
Nose Wrinkler (AU9)Upper lid raiser (AU5) Lip Tightener (AU23)Outer Brow Raiser (AU2)
ACA Spectral Clustering
(SC)
Lower face 0.53(.09) 0.39(0.14)
Upper face 0.69(.12) 0.47(0.12)
![Page 41: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/41.jpg)
Conclusions and open problems
• Supervised and unsupervised algorithms for FEA
• Tracking/registration
– Registration, registration, registration… changes in pose (3D models)
– Robustness to occlusion
• Features
– Subtle facial expressions
– Dynamics (e.g., temporal envelope)
• Classifiers
– Expression intensity
– Individual differences
– Predicting onset/offset
– Truly multi-class AU detection
![Page 42: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/42.jpg)
Conclusions and open problems• Data
– Attention to reliability of ground truth
– Shared, well-annotated video
– Innovative ways to use video that cannot be shared
• Segmentation and timing
– Intra-personal
– Interpersonal
• User-in-loop approaches
– User-assisted coding, e.g., Fast FACS (e.g., Simon et al., 2011)
– Combining manual and automated measurement (e.g. Ambadar
et al. 2009)
– Person-dependent classifiers
• Other issues
– Multimodal
– Context
![Page 43: Facial Expression Analysis - Aalborg Universitet · Facial Expression Analysis Jeff Cohn ... •Constrained Local model (e.g., Cristanace& Cootes08, ... The million $ question](https://reader034.vdocuments.site/reader034/viewer/2022051801/5ae0ff737f8b9afd1a8eb07f/html5/thumbnails/43.jpg)
43Facial ExpressionAnalysis F. De la Torre/J. Cohn Looking @ People (CVPR-12)
Questions?
Jeff Cohn
Fernando De la Torre
Tutorial Looking @ People
June 2012
Human Sensing
Laboratory