Smart Face Recognition Analysis Report- Optimization, Response Time & Efficiency

Introduction: Face Recognition using opencv python to detect face & recognize all facial features of particular person can be achieved with more than 90% efficiency in opencv-3.1.0. The results depends on trained model & the database of grayscale images supplied.

Purpose of Analysis: The objective is to find the best face recognition model/algorithm for less memory utilization & low processing from servers possible i.e, millions of images can be processed much faster by segmentation, multi-threading or parallel processing. Also, explore & test best computational models like knn(k Nearest Neighbour), cnn(Computational Neural Network), dnn(Deep Neural Network). Hypothesis: From the official documentation of Computer Vision[1], the currently available algorithms for opencv2 & later versions are:

● Eigenfaces (see createEigenFaceRecognizer()) ● Fisherfaces (see createFisherFaceRecognizer()) ● Local Binary Patterns Histograms (see createLBPHFaceRecognizer())

We choose LBPHFaceRecognizer() for our project as this algorithm can recognize at different angles, color modes, ambiance(lights) for all trained datasets. The basic idea of Local Binary Patterns is to summarize the local structure in an image by comparing 1 Smart Face Recognition System

each pixel with its neighborhood. Take a pixel as center and threshold its neighbors against. If the intensity of the center pixel is greater-equal its neighbor, then denote it with 1 and 0 if not. You’ll end up with a binary number for each pixel, just like 11001111. So with 8 surrounding pixels you’ll end up with 2^8 possible combinations, called Local Binary Patterns or sometimes referred to as LBP codes. The first LBP operator described in literature actually used a fixed 3 x 3 neighborhood just like this:

By definition the LBP operator is robust against monotonic gray scale transformations. We can easily verify this by looking at the LBP image of an artificially modified image (so you see what an LBP image looks like!):

Watch this video for better idea about feature extraction: https://vimeo.com/12774628 Problem Statement: Required a face recognition model that can capture image from wireless camera and if the user’s face exists in database then allow him/her to enter else display option to train than face & put it to existing database by the permission of admin else trigger an alarm for unauthorized entry. This whole thing must be further

2 Smart Face Recognition System

implemented to android application & the face recognition model can be processed via backend servers. Further, optimize the whole process and increase the response time Methodology: One possible approach/solution to the above problem can be as follows.

● Create database for facerec with possible structure as table facerec{id,name,dob,time}

● Get image from camera over wireless network(IP & PORT) only if motion of human is detected then capture face.

● Get image sent over network then process it: ○ Get ID from facerec_database & if ID of image received is same then

process image for that ID dataset available from trained model ○ Get the prediction & confidence of result.

● Start the server and return the results in json format (ID, result).

○ ID: image ID from the database ○ result: true/false

● Get the json result in android application & display the name corresponding to that image ID from facerec database.

Flow Chart: All possible flow charts of face recognition models

3 Smart Face Recognition System

Deploy in cloud-servers:

1. Using Google Cloud: VM instances (https://console.cloud.google.com) VM Instance- Ubuntu 16.4 4Gb RAM, 375 GB Storage, 3 Years Usage Cost- 45.60$ per month Benefits: All kinds of servers can be installed LAMP, Apache Tomcat, WAMP etc.

4 Smart Face Recognition System

Test Results: success when passed images for each trained model in the facerec_app. Using Mozilla Browser for ssh into VM instance.

2. Using Python Hosting: Heroku (https://www.heroku.com/pricing) Cost upto: 500$ per month

5 Smart Face Recognition System

Deploy in Raspberry Pi 3:

● Hardware Details: 1Gb RAM, 1.2GHz ARM Cortex Processor, 16Gb Storage,

64-bit Quad Core Processor( increases efficiency by 33%) ● Cost: Rs.4499 (RPi) + Rs. 2349 (Pi Cam) = Rs. 6848/- ● Opencv build using cmake requires: 1h 12m to compile ● Problems: 24x7 Running Mode, Heating of Chip, May hang ● Requires testing for 24x7. ● Benefits: Small & portable, All modules built-in same chip.

Problems with other ARM Processor: Required interfacing of all modules like WiFi module, Memory etc again embedded person required.

6 Smart Face Recognition System

Multiple Camera Tests: Connected USBCam & WebCam Using opencv multiple streaming is achieved

IP Camera: Video stream can be captured from multiple IP Cameras over WiFi network & further can be converted into frames to numpy array for processing.

After mapping the HTTP video stream of IP Camera to video driver /dev/video* videoCapture(1) will work for that mapped device ID & all opencv processing is possible with the same.

7 Smart Face Recognition System

Problems/Parameters for IP Camera: ● Bandwidth required to process multiple cameras: Video CODEC (Compression),

Streaming Mode, Frame Rate, Resolution, Video Quality. ● Network topology

● Resource calculation for servers to process & capture streams of ip camera from

http to /dev/video*

Using IP Camera Simulation Software to get above parameters for 100+ cameras. IP Video System Design Tools by Vivotek: To monitor the network & check for all parameters. http://www.vivotek.com/ip-video-system-design-tool/#downloads

8 Smart Face Recognition System

100 Viviotek cc8130 cameras 30 days storage required 1,01,868.6 GB & Bandwidth= 314.11Mbit/s

Using this tool we can estimate everything & plan a efficient systems for cctv monitoring & processing wireless through servers.

9 Smart Face Recognition System

WebCam Streaming: using opencv https://github.com/benhowell/examples/tree/master/WebcamStreamingOpenCV

Face Detection using IP Camera connected to Netgear Router is tested and found some frame lags and error detection. Now, training of data sets should be done with same situation, lightning conditions & camera. Python Client Server Connection: To deploy in My Attendance App

● Create php web service to upload image and store to database. ● Get images in python.

10 Smart Face Recognition System

Results: ● Faces can be trained in real time into datasets using LBPH model with response

time of 0.03 ~ 0.05 secs from Ubuntu 16.04, 4GB RAM intel i5 Processor. ● Recognition of faces is done from wireless camera using web server remotely

with success rate of 95% approx. ● Success on Cloud Servers (Google VM Instance: Ubuntu 16.4, 4GB RAM, 10GB

Storage) Future Scope:

● Object Recognition: It can be used to check terrorist activities like human with weapons, knife, face-masks at ATM etc.

● Camera Surveillance in traffic, hotels, institutions & companies etc. ● All those big companies like Facebook, Google, Microsoft are now relying on

their own algorithms & using Tensorflow and Deep Neural networks in A.I to predict their datasets and for the same openface is utilized.

● Also, with the use of Google Cloud for Vision API object recognition is very easy in tensorflow & python. (https://cloud.google.com/vision/)

● Emotion Detection like mood of person happy, sad, crying faces. References:

1. http://docs.opencv.org/2.4/modules/contrib/doc/facerec/facerec_tutorial.html#face-recognition-with-opencv

2. http://www.pyimagesearch.com/2016/10/24/ubuntu-16-04-how-to-install-opencv/ 3. https://medium.com/@ageitgey/machine-learning-is-fun-part-4-modern-face-reco

gnition-with-deep-learning-c3cffc121d78 4. http://cmusatyalab.github.io/openface/ 5. https://multi-link.net/much-bandwidth-need-ip-video-surveillance-application/

GitHub: https://github.com/vsltech/smartfacerecognition Project Details: http://www.vslcreations.com/2017/12/smart-face-recognition-system-using.html

11 Smart Face Recognition System