facial image super resolution using sparse …cs.ioc.ee/~tarmo/tday-kao/rasti-slides.pdf•image...

Facial Image Super Resolution

Using Sparse Representation for

Improving Face Recognition in

Surveillance Monitoring

Tõnis Uiboupin

Pejman Rasti

(Head of Image Processing division of iCV Group)

Gholamreza Anbarjafari,

Outline

•Problem

•Introduction to super resolution

•Introduction to face recognition

•Proposed method

•Experimental results

•Conclusion

Face Recognition

•Face recognition is of great importance in

many computer vision applications, such

as human-computer interactions, Security

systems, Military and Homeland Security.

Problem

•face recognition systems mostly work with

images\videos of proper quality and

resolution.

•In videos recorded by surveillance

camera, due to the distance between

people and cameras, people are pictured

very small and hence challenge face

recognition algorithms

Problem

Essex database Feret database HP database iFR database

99.20% 75.87% 31.6% 20% 50.67% 26.67% 98.06% 76.13%

Image up-sampling/enhancement

•Image Interpolation

•Super Resolution

image Interpolation

•Image interpolation is one of the basic methods for up-sampling images

•Some of the famous interpolation techniques are:–Nearest neighbor

–Bilinear

–Bicubic

•The high frequency details are not restored

Image Super Resolution

•The desire for high-resolution comes from two principal application areas:

• Improvement of pictorial information for humaninterpretation

• Helping representation for automatic machine perception

•The application of SR techniques covers a wide range of purposes such as Surveillance video, Remote sensing, Medical imaging (CT, MRI, Ultrasound.).

Image Super Resolution

Methodsdomain

Fourier

Wavelet

Frequency

Multiple Images

Single image

Spatial

Image Super Resolution

HowType

Set of low res. imagesMulti-Images

Image model/priorSingle-Image

Multiple-image super-resolution

algorithms

•Receive a couple of low-resolution images

of the same scene as input and usually

employee a registration algorithm to find

the transformation between them.

Multiple-image super-resolution

algorithms

•Iterative back projection

•Iterative adaptive filtering

•Direct methods

•Projection onto convex sets

•Maximum likelihood

•Maximum a posteriori

single-image super-resolution

algorithms•During the sub-sampling or decimation of an image, the desired high-frequency information gets lost. Multiple super resolution methods cannot help recover the lost frequencies, especially for high improvement factors

•Single-image super-resolution algorithms do not have the possibility of utilizing sub-pixel displacements, because they only have a single input.

single-image super-resolution

algorithms

•Learning-based single-image SR

algorithms

•Reconstruction-based single-image SR

algorithms

single-image super-resolution

algorithms

•Learning-based single-image SR algorithms

–These algorithms, as learning-based or

Hallucination algorithms were first introduced in

which a neural network was used to improve the

resolution of fingerprint images.

–These algorithms contain a training step in which

the relationship between some HR examples

(from a specific class like face images,

fingerprints, etc.) and their LR counterparts is

learned.

single-image super-resolution

algorithms

•Reconstruction-based single-image SR

algorithms

–These algorithms similar to their peer multiple

image based SR algorithms try to address the

aliasing artifacts that are present in the LR

input image.

Face recognition

•In general, face recognition consist of 5

steps

–pre-processing

–face detection

–The facial components of region of interest

(ROI)

–feature extraction

–classification

Face recognition

•pre-processing

–image enhancement

–noise removal

–both of them

Face recognition

• face detection

–Viola-Jones

Face recognition

• The facial components of region of interest (ROI)– mouth

– eyes

– ear

– cheeks

– nose

– fore-head

– eyebrow

Face recognition

•feature extraction

–Local Binary Patterns (LBP)

– Gabor filters

–Linear Discriminant Analysis (LDA)

–Principal Component Analysis (PCA)

–Local Gradient Code (LGC)

–Independent Component Analysis (ICA)

Face recognition

• classification

–support vector machine (SVM)

–artificial neural network (ANN) classifier

–Hidden Markov Model

solution

•we investigate the importance of the state

of-the-art super-resolution algorithm in

improving recognition accuracies of the

state-of-the-art face recognition algorithm

for working with low-resolution images.

Proposed Method

•Having a low-resolution input images, the

proposed system upsamples it by the

sparse representation super-resolution

algorithm. Then, the SVD and Hidden

Markov Model algorithm are used to

perform face recognition on the high-

resolution image.

Proposed Method

Databases

Experimental Result

•The Essex, HP, ferret and ifr database has

been employed.

Conclusion

•State-of-the-art face recognition algorithms, like Hidden

Markov Model and SVD have difficulties handling

videos\images that are of low quality and resolution.

•we have proposed to use upsampling techniques.

•Experimental results on a down-sampled version of the

benchmark databases show that the proposed is efficient

in improving the quality of such lowresolution images and

hence improves the recognition accuracy of the face

recognition algorithm