face recognition in low resolution imageshji/cs519_slides/face recognition in low... · dwayne...
TRANSCRIPT
![Page 1: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/1.jpg)
Face Recognition
in
Low Resolution Images
Trey AmadorScott MatsumuraMatt Yiyang Yan
![Page 2: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/2.jpg)
Introduction
![Page 3: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/3.jpg)
❖ Purpose: low resolution facial recognition➢ Extract
image/video from source
➢ Identify the person in real time given a trained-database
taken from https://github.com/alexjc/neural-enhance
![Page 4: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/4.jpg)
Face Recognition
Libraries
histogram of oriented gradients (HOG)
dlibSupport VectorMachines (SVM)
![Page 5: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/5.jpg)
Process
● Neural Enhance library○ increase the resolution
of low pixel density○ Theano (neural network)
→ Lasagne (train) →upsampled image
● dlib○ Histogram of oriented
gradients (HOG)○ SVM○ feature descriptor for
detecting faces
→
![Page 6: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/6.jpg)
Database
● IMDb○ Internet Movie Database is an online database of
information related to films, television programs and videogames
○ low and high resolution versions of the same image○ high-resolution 'base' image to train the Support Vector
Machine (SVM)
![Page 7: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/7.jpg)
Support Vector Machine
for
Face Recognition
Arnold Schwarzenegger
![Page 8: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/8.jpg)
SVM
Identify the Rock
Imagessimilar to Dwayne Johnson
Image ofDwayne Johnson
![Page 9: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/9.jpg)
SVM ++
+
+
+
-
-
-
--
+ The Rock- not The Rock
Imagessimilar to Dwayne Johnson
Image ofDwayne Johnson
![Page 10: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/10.jpg)
SVM ++
+
+
+
-
-
-
--
Separate data
Imagessimilar to Dwayne Johnson
Image ofDwayne Johnson
![Page 11: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/11.jpg)
SVM ++
+
+
+
-
-
-
--
Which line?
Imagessimilar to Dwayne Johnson
Image ofDwayne Johnson
![Page 12: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/12.jpg)
SVM ++
+
+
+
-
-
-
--
Thickest line
Imagessimilar to Dwayne Johnson
Image ofDwayne Johnson
![Page 13: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/13.jpg)
SVM +
-
-
+
+
+ +
+
- -
-
-
-
Separate data?
Imagessimilar to Dwayne Johnson
Image ofDwayne Johnson
![Page 14: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/14.jpg)
SVM +
-
-
+
+
+ +
+
- -
-
-
-
Non-linear separation
Imagessimilar to Dwayne Johnson
Image ofDwayne Johnson
![Page 15: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/15.jpg)
Generative
Adversarial Network
for
Upsampling Images
![Page 16: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/16.jpg)
GAN
Back with The Rock
Imagessimilar to Dwayne Johnson
Image ofDwayne Johnson
![Page 17: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/17.jpg)
Image ofDwayne Johnson
Imagessimilar to Dwayne Johnson
GAN
Generate this image?
+
![Page 18: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/18.jpg)
Generative Network
produce an imageDiscriminative Network
real or fake
vs
![Page 19: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/19.jpg)
How to train your
Generative
Adversarial Network
![Page 20: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/20.jpg)
Discriminative
Network
real
fake
GAN
Train discriminative
network
![Page 21: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/21.jpg)
Discriminative
Network
Generative
Network
random noise
backpropagation
GAN
Train both networks
negative gradient
positive gradient
Fake
![Page 22: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/22.jpg)
Discriminative
Network
Generative
Network
random noise
backpropagation
GAN
Eventually?
Real
![Page 23: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/23.jpg)
Discriminative
Network
Generative
Network
GAN
Upsampled
Real
![Page 24: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/24.jpg)
Code
can be found at:
https://github.com/PresidentDwayneCamacho/super-res-face
![Page 25: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/25.jpg)
super resolution
video samples
![Page 26: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/26.jpg)
face recognition in enhanced-resolution video
![Page 27: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/27.jpg)
super resolution image enhancement
boring Bruce Springsteen
100 x 100
enhanced Bruce Springsteen
200 x 200
actual Bruce Springsteen
high res
![Page 28: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/28.jpg)
super resolution face recognition
unrecognized Bruce Springsteen
100 x 100
that’s Bruce Springsteen!
200 x 200
![Page 29: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/29.jpg)
experimental paradigm
true face false face
high res high res
low res low res
enhanced res enhanced res
![Page 30: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/30.jpg)
![Page 31: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/31.jpg)
![Page 32: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/32.jpg)
![Page 33: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/33.jpg)
![Page 34: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/34.jpg)
future directions
❖ find robust metric with which to filter data
❖ test efficacy of various algorithms
❖ generate larger dataset
![Page 35: Face Recognition in Low Resolution Imageshji/cs519_slides/Face Recognition in Low... · Dwayne Johnson. Image of Dwayne Johnson Images similar to Dwayne Johnson GAN Generate this](https://reader036.vdocuments.site/reader036/viewer/2022070107/6020c556a6555d20e32b8141/html5/thumbnails/35.jpg)
References
[1] W. Zhao, et al. “Face Recognition: A Literature Survey.” ACM Computing Surveys, vol. 35, pp. 399-458, Dec. 2003.
[2] S.C. Park, M.K. Park, and M.G. Kang. “Super-Resolution Image Reconstruction: A Technical Overview.” IEEE Signal Processing Magazine. May
2003.
[3] D. Glasner, S. Bagon, and M. Irani. “Super-Resolution from a Single Image,” in IEEE 12th ICCV, 2009, pp 349-356.
[4] W.W. Zou and P.C. Yuen. “Very Low Resolution Face Recognition Problem.” IEEE Transactions on Image Processing, vol. 21, pp. 327-340, July
2012.
[5] A. Geitgey, "Face Recognition," GitHub repository, [Online]. Available: https://github.com/ageitgey/face_recognition. [Accessed 29 10 2017].
[6] N. Dalal and B. Triggs. “Histogram of Oriented Gradients for Human Detection” in CVPR, 2005, pp. 1-8.
[7] P. Felzenszwalb, et al. “Object Detection with Discriminantly Trained Part Based Models.” IEEE Transactions on Pattern Analysis and Machine
Intelligence, vol. 32, pp. 1627-1645, Sept. 2010.
[8] C. Cortes and V. Vladimir, "Support-Vector Networks," Machine Learning, vol. 20, no. 3, pp. 273-297, 1995.
[9] A. J. Champandard, "Neural Enhance," GitHub repository, [Online]. Available: https://github.com/alexjc/neural-enhance. [Accessed 29 10 2017].
[10] D. G. Lowe, "Object Recognition from Local Scale-Invariant Features," Computer Vision, vol. 2, pp. 1150-1157, 1999.
[11] K. Simonyan, M. O. Parkhi, A. Vedaldi and A. Zisserman, "Fisher Vector Faces in the Wild," British Machine Vision Conference, vol. 2, no. 3, p. 4,
Sept. 2013.
[12] P. Fischer, A. Dosovitskiy and T. Brox, "Descriptor Matching with Convolutional Neural Networks: a Comparison to SIFT," arXiv, p. 10, 22 May
2014.
[13] M. O. Parkhi, A. Vedaldi and A. Zisserman, "Deep Face Recognition," British Machine Vision Conference, vol. 1, no. 3, p. 6, 2015.
[14] U. Karn, "An Intuitive Explanation of Convolutional Neural Networks," The Data Science Blog, [Online]. Available:
https://ujjwalkarn.me/2016/08/11/intuitive-explanation-convnets/. [Accessed 29 10 2017].
[15] C. Ledig, et al. "Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network," arXiv, p. 19, 25 May 2016.
[16] A.V. Nefian. “Georgia Tech Face Database.” Nov. 15, 1999. [Online]. Available: www.anefian.com/research/face_reco.htm. [Accessed: Nov. 5,
2017].
[17] Y.D. Wong. “ChokePoint Dataset.” [Online]. Available: arma.sourceforge.net/chokepoint/. [Accessed: Nov. 5, 2017].