![Page 1: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/1.jpg)
clda.co/pycon8-‐facial-‐analysis
Facial Analysis Techniques for Pythonista
(and beyond!)
4/9/2017 PYCON OTTO @ Florence
![Page 2: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/2.jpg)
About Me
@alex_casalboni
clda.co/pycon8-‐facial-‐analysis
Computer Science Background
Master in Sound & Music Engineering
Sr. SoDware Engineer & Web Developer
Cloud Evangelist @ Cloud Academy
![Page 3: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/3.jpg)
Agenda
What is a Face?
Problem decomposiLon
AlternaLves to DIY
clda.co/pycon8-‐facial-‐analysis
![Page 4: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/4.jpg)
What is a Face?
You thought you knew, but you didn’t… ?
clda.co/pycon8-‐facial-‐analysis
![Page 5: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/5.jpg)
About “Normal” Faces
clda.co/pycon8-‐facial-‐analysis
![Page 6: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/6.jpg)
What about rotaCon invariance?
clda.co/pycon8-‐facial-‐analysis
![Page 7: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/7.jpg)
What about animals?
clda.co/pycon8-‐facial-‐analysis
![Page 8: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/8.jpg)
What about painCngs?
clda.co/pycon8-‐facial-‐analysis
![Page 9: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/9.jpg)
What about masks?
clda.co/pycon8-‐facial-‐analysis
![Page 10: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/10.jpg)
What about smilies?
clda.co/pycon8-‐facial-‐analysis
![Page 11: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/11.jpg)
Problem decomposiLon
What are the main building blocks for facial analysis?
clda.co/pycon8-‐facial-‐analysis
![Page 13: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/13.jpg)
Face DetecCon
clda.co/pycon8-‐facial-‐analysis
Input: 1 image & unknown context
Goal: finding faces (how many?)
Output: lists of coordinates
Difficulty: preUy easy
![Page 14: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/14.jpg)
Face DetecCon Results
clda.co/pycon8-‐facial-‐analysis
![Page 15: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/15.jpg)
Face DetecCon Techniques
clda.co/pycon8-‐facial-‐analysis
Algorithmical techniques
Not too much “intelligence”
Real-‐Lme tracking
![Page 16: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/16.jpg)
Face DetecCon Techniques -‐ HOG
clda.co/pycon8-‐facial-‐analysis
Histogram of Oriented Gradients
![Page 17: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/17.jpg)
HOG w/ OpenCV and dlib
clda.co/pycon8-‐facial-‐analysis
* Vectors allow for more advanced analysis (see hUp://www.paulvangent.com/2016/08/05/emoLon-‐recogniLon-‐using-‐facial-‐landmarks/)* That .dat file is 100+MB
![Page 18: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/18.jpg)
Face DetecCon Techniques -‐ Haar Cascades
clda.co/pycon8-‐facial-‐analysis
Haar Feature-‐based Cascade Classifiers
![Page 19: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/19.jpg)
Haar Cascades w/ OpenCV
clda.co/pycon8-‐facial-‐analysis
![Page 22: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/22.jpg)
Face RecogniCon
clda.co/pycon8-‐facial-‐analysis
Input: 1 reference and 1 target image
Goal: finding facial matches
Output: lists of (potenLal) matches
Difficulty: medium
![Page 23: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/23.jpg)
Facial Encoding
clda.co/pycon8-‐facial-‐analysis
Vector RepresentaLon (128D) *
* could be learned with DL
![Page 24: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/24.jpg)
Facial Distance
clda.co/pycon8-‐facial-‐analysis
A and B are the same person if distance(A, B) < tolerance
![Page 25: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/25.jpg)
Face Matching w/ face_recogni2on
clda.co/pycon8-‐facial-‐analysis
![Page 27: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/27.jpg)
Facial Analysis
clda.co/pycon8-‐facial-‐analysis
Input: 1 detected face
Goal: extracLng high-‐level informaLon
Output: gender, age, emoLons, headwear, etc.
Difficulty: preUy hard
![Page 28: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/28.jpg)
Facial Analysis
clda.co/pycon8-‐facial-‐analysis
ML Model (gender)
ML Model (emoLons)
ML Model (….)
ML Model (age)
ML Model (headwear)
![Page 29: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/29.jpg)
Facial Analysis
clda.co/pycon8-‐facial-‐analysis
How many training sets?
Parallel features extracLon & predicLon
Accuracy is more subjecLve (source/target audience)
Real-‐Lme is not guaranteed
![Page 30: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/30.jpg)
AlternaLves to DIY
How about Facial Analysis services?
clda.co/pycon8-‐facial-‐analysis
![Page 31: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/31.jpg)
Facial Analysis Services
Amazon RekogniLon
Google Cloud Vision
Azure Face API
Face++
Kairos
EmoVu
clda.co/pycon8-‐facial-‐analysis
![Page 32: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/32.jpg)
Amazon RekogniCon & Python
clda.co/pycon8-‐facial-‐analysis
![Page 33: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/33.jpg)
Google Cloud Vision & Python
clda.co/pycon8-‐facial-‐analysis
![Page 34: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/34.jpg)
Azure Face API & Python
clda.co/pycon8-‐facial-‐analysis
![Page 36: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/36.jpg)
clda.co/pycon8-‐facial-‐analysis
Cloud Services Pros
Language agnosLc (RESTful APIs)
Models are updated under the hood
No infrastructure to manage
PAYG model (w/ free Ler)
Great for embedded systems
Granted accuracy (globally)
![Page 37: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/37.jpg)
clda.co/pycon8-‐facial-‐analysis
Cloud Services Cons
Hardly real-‐Lme (HTTPs calls)
ConnecLvity is always needed
Training set is never customizable
ML Models are a black box
![Page 38: Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9b71a1a28ab91228b4581/html5/thumbnails/38.jpg)
Thank you :)
jobs.cloudacademy.com
P.S. we are hiring!
PYCON OTTO @ Florence