Micro expression Detection using Strain Patterns
-Sridhar Godavarthy
Based on
V.Manohar, D.B. Goldgof, S.Sarkar, Y. Zhang, "Facial Strain Pattern as a Soft Forensic Evidence", IEEE Workshop on Applications of Computer Vision (WACV'07),pp 42-42
Microexpressions
• What are microexpressions?– Subtle movements of the human body– Quick enough to be completed within the blink
of an eye– Not large enough to be called a micro
expression
Microexpressions Contd…
• Examples:• Raising an eyebrow• Shrugging of shoulders• Pout of lips• Fast blinking of eye
• Non Examples:• Talking• Smiling• Laughing• Anger
How do we go about it?
• Majority of the work “referenced” from : V.Manohar, D.B. Goldgof, S.Sarkar, Y. Zhang,
"Facial Strain Pattern as a Soft Forensic Evidence", IEEE Workshop on Applications of Computer Vision (WACV'07),pp 42-42
However, to keep things clear, we discuss each step in detail.
• Feature Based– Need to identify features – Difficult!– Features may be ill defined( when camouflaged)– Usually requires manual intervention– Produces a sparse motion field– Produce Good correspondence in large motion
• Optical Flow based– Fully automated– Dense Motion field.– Requires constant illumination
First Step – Obtaining Motion Field
• 3D Strain– Ideal– No high speed equipment available to capture
range images
• 2D Strain– Well – not much of a choice– Authors could use existing data.
Second Step – Strain Computation Type
• Finite Element Method– Forward modeling when Dirichlet condition is satisfied– Good at handling irregular shapes– Computationally expensive– This method is an approximation to the solution
• Finite Difference Method– Strain, a tensor, can be expressed derivatives of the displacement
vector– This can be approximated by a Finite Difference Method.– Very efficient when carried out on a regular grid.– This method is an approximation to the differential equation
Strain Computation - methods
Video
• Video is a collection of individual images also known as frames
• In reality: spatial and temporal compression using properties of the scene.
• Any video can be decoded into a series of frames.
• 24/30 frames per second of video.
Video Coding
• The science of encoding a video in a manner such that– Minimum number of
bits are used – Motion compensated
prediction can be performed from a previous frame.
Optical Flow• Pattern of apparent motion of objects/surfaces/edges
caused by relative motion between the observer(camera) and the scene
Elasticity• Different materials have different elasticity• Elasticity can be modeled
strain
stressElasticity Known
Calculate
Optical Strain• Variation of displacement values obtained from optical flow
– Calculated by taking the derivative of each pixel
Sobel operator (central difference)
Facial Strain
• What is Facial Strain?– Strain on soft tissue when expressions are
made.– Anatomical method– Uses a pair of frames to measure deformation
Strain Measurement
• Finite Difference Method• Compute spatial derivatives from discrete
points. – Forward Difference Method– Central Difference Method– Richardson extrapolation
The ProcessStart
Divide videos into Training and Testing Sequences
Read Training Video
Read Neutral
Micro Exp Macro
Exp Frames
Decode Video
Calculate OF & OS for Micro &
Macro frames
Training
Sequence
The Process Contd…Read Test
Video
Read Neutral Frames
Decode Video
Calculate Optical Flow between Neutral frame
and each frame
OF within desired range?
Reject Frame
Calculate Optical Strain between Neutral frame and this frame
OS within desired range?
Reject Frame
MICRO EXPRESSION
Stop
NoNo
Yes Yes
For Every other Frame
Test
Sequence
Decisions and Consequences
• Use OF to reject highly improbable frames– Improves performance– Rejects frames with both Micro and Macro
Exp.– Segment image into regions?
• Threshold for classification was set to be ⅔T.O.S < M.O.S < 5/3 T.O.S
T.O.S = Training Optical Strain.
M.O.S = Measured Optical Strain
Possible Options
• Use only Optical Flow
• Use only Optical Strain
• Vary the thresholds
• Use an alternative for Max(OF/OS)
Future work
• Do not compare all frames( Skip frames immediately after a positive)
• Identify only one positive in a sequential list of positives
• Segment images to get separate Micro and Macro expressions
What I could not do
• Display strain as an image.• Segment image into regions and have
Regions of Interest
• V.Manohar, D.B. Goldgof, S.Sarkar, Y. Zhang, "Facial Strain Pattern as a Soft Forensic Evidence", IEEE Workshop on Applications of Computer Vision (WACV'07),pp 42-42
• Vasant Manohar, Matthew Shreve, Dmitry Goldgof and Sudeep Sarkar, "Finite Element Modeling of Facial Deformation in Videos for Computing Strain Pattern", International Conference on Pattern Recognition, Dec. 2008
• Matthew A. Shreve, Shaun J. Canavan, Yong Zhang, John R. Sullins, and Rupali Patil, "Imaging And Characterization Of Facial Strain In Long Video Sequences",xxxx
• Malcolm Gladwell,” Blink: The Power of Thinking Without Thinking”, Back Bay Books (April 3, 2007)
References
Thank You!
Sridhar Godavarthy
Dept. Of Computer Science and Engineering
University of South Florida
About the TemplateFire was considered the most fearful of weapons in Greek Mythology. This template –
“Fireball” is a depiction of the might and power of fire, punching through ignorance and enlightening the audience