develop a face recognition system on arm board using opencv

Develop A Face Recognition System on ARM Board Using OpenCV

WU JiaOpenCV China Team

Outline

• Face recognition in brief•Build a FR system on ARM board using OpenCV•Assignment

1

Face recognition is to identify or verify a person from an image or a video frame.

2

Face

Reco

gniti

on

A

B

C

Who is this person, 1:N Is this person X, 1:1

Face

Reco

gniti

on

Is A

Not A

Not A

Face Recognition Pipeline

3

input images face detection face alignment

calculate similarity

face representation

same person

different person

Face Detection • Traditional methods

Viola-Jones face detector, DPM etc.

• Deep learning based

4

Hang Du et al., The Elements of End-to-end Deep Face Recognition: A Survey of Recent Advancesthe development of representa0ve face detec0on methods

multi-stagesingle-stage

anchor-basedanchor-freeother

Face Alignment To obtain a canonical layout of the unconstrained face.

• Landmark-based methods• Landmark-free methods• Face frontalization

5

coordinate regressionheatmap regression3D model fittinglandmark-freeface frontalization

Hang Du et al., The Elements of End-to-end Deep Face Recognition: A Survey of Recent Advances

Face Representation To extract discrimina6ve features from aligned face images, which is key to a face recogni6on system.• Tradi:onal methods

Eigenface, Gabor, LBP etc.

• Deep learning based

6Hang Du et al., The Elements of End-to-end Deep Face Recognition: A Survey of Recent Advances

classificationfeature embedding

semi-supervisedhybrid

Outline

• Face recogniCon in brief•Build a FR system on ARM board using OpenCV•Assignment

7

8

usb cameraARM dev board

face detection face preprocessing

featuredatabase

face detec<on face preprocessing

feature extraction

Registration Image

Camera Image result

Registration

Recognition

feature extrac<on

calculatesimilarity

Face Recognition System Architecture

9

EAIDK-310

EXT_IO RECOVER RK3228H MASKROM

USB2.0X2

USB3.0X1USB2.0X1

RJ45Ethernet

WIFI/BT

LPDDR3

RESET

DC5V IN HDMI OUT SPEAKER

OpenCV DNN Module

• Inference framework• Compatible to many popular deep learning frameworks

• Run on multiple devices: CPU, GPU, VPU, FPGA • Run on different OS: Linux, Windows, Android, MacOS• Backend framework to leverage different acceleration libs

10

Lightness

Convenience

UniversalityAccelerated

well tested networks

11

Image Classification Object DetectionSemantic

SegmentationPose

EstimationImage Processing Person

Identification

Text Detection and

Recognition

Depth Estimation

CaffeAlexNetGoogLeNetVGGResNetSqueezeNetDenseNetShuffleNet

TensorFlowInceptionMobileNetEfficientNet

Darknetdarknet19

ONNXAlexNetGoogleNetCaffeNetRCNN_ILSVRC13ZFNet512VGG16VGG16_bnResNet-18v1ResNet-50v1CNN MnistMobileNetv2LResNet100E-IREmotion FERPlusSqueezenetDenseNet121Inception v1, v2Shufflenet

CaffeSSD VGGMobileNet-SSDFaster-RCNNR-FCNOpenCV Face Detector

TensorFlowSSDFaster-RCNNMask-RCNNEASTEfficientDet

DarknetYOLOv2tiny YOLO YOLOv3tiny YOLOv3YOLOv4tiny YOLOv4

ONNXTinyYolov2PyTorchYOLOv3PyTorchYOLOv3-tinyPyTorch SSD VGG

CaffeFCN

TorchENet

ONNXResNet101_DUC_HDC

TensorFlowDeepLab

PytorchUNetDeepLabV3FPNUNetPlusBiSeNet

CaffeOpenPose

PyTrochAlphaPose

CaffeColorization

TorchFast-Neural-Style

ONNXStyle Transfer

TorchOpenFace

PyTorchTorchreid

TensorFlowMoibleFaceNet

PyTorchEasyOCRCRNN

PyTorchMonodepth2

For more detail: https://github.com/opencv/opencv/wiki/Deep-Learning-in-OpenCV

12

https://www.learnopencv.com/cpu-performance-comparison-of-opencv-and-other-deep-learning-frameworks/

85.4 182.8

3008.3

53.9 70.1

1184.7

0500

100015002000250030003500

Squeeze Net 1.1 MobileNet SSD 1.0 VGG16

Inference time(ms) @Cortex-A72x2

OpenCV OpenCV+Tengine-lite

Tengine: • edge AI inference framework. • Surports CPU, GPU, DSP, NPU. • Surports TensorFlow, Caffe, MxNet,

PyThorch, MegEngine, DarkNet, ONNX, ncnn.

AWS t2.large instance (2 vCPUs and 8 GB of RAM, no GPU), Ubuntu 16.04 LTS

Speed comparison

13

Easy-to-useload dl modelchoose accelerating backendset target device

inferencing

Now let’s try to build the system

14

landmark detection

face detection

feature extractionOpenFace

model

libfacedetection

similarity l2 distance

Speed can reach 1000FPS.https://github.com/ShiqiYu/libfacedetection

face detection face preprocessing

feature extraction

Input Image

face and landmark detection

15

face alignment

16

feature extraction

17

calculate similarity

Outline

• Face recognition in brief•Build a FR system on ARM board using OpenCV•Assignment

18

Assignment

A. Build a complete face recognition system on ARM board using OpenCV. Write a report in English about the system.

B. Build any other kind of biometric recognition system on ARM board using OpenCV. Write a report in English about the system.

• Submit to: jia.wu@opencv.org.cn• Deadline: 23:59 Jan. 27, 2021• Top 3 teams will be awarded the advanced EAIDK-610 Kit.

19

Issues

20

• Training‒ use deep learning frameworks

•Registration

•UI

face detec<on face preprocessing

featuredatabase

Registration Image

feature extraction

Thank You!

www.opencv.org.cn