3d face structure estimation using evolutionary algorithms

1

“3D Structure Estimation Using Evolutionary Algorithms Based on Similarity Transform”

AuthorsK. Punnam Chandar &

Dr. T. Satya Savithri

Eighth Asia International Conference on Mathematical Modeling and Computer Simulation

(AMS-2014).

AMS-2014 Sep-25; Kuala Lumpur - Malaysia


2

Outline:---

• Introduction: 3D Model Acquisition• Koo and Lam Algorithm - SFM• 3D to 2D Projection Model• Objective function• Optimization using GA• Differential Evolution and other EA• Results: Pose and Depth Estimation• Conclusion

3

Introduction

• 3D Models of face are gaining importance in the fields of face recognition, face Tracking, 3D Virtual Worlds & Games, 3D Simulation due to their superior performance over 2D Models.

• Currently there are two main streams of creating the 3D face models, one approach is to use specialized 3D Depth sensing cameras and the other is reconstructing the 3D face model from 2D images.


3D FaceSource:3D Face Home Page


4

• The high cost of 3D depth sensing cameras limit their deployment in Security Applications.

• The alternative is to develop algorithms to reconstruct the 3D face model from 2D images such as video sequences and multi-view photographs.

• The goal of the reconstruction algorithm is to derive the 3D shape information of the face from N-2D images (N≥2), one frontal view and others non-frontal view images.


5

• During the past decade 3D reconstruction algorithms based on 2D images have been developed to estimate the 3D Structure.

• Representative algorithms can be categorized into four groups Shape-from-X (ref.1) 3D Morphable Model (ref.2) Learning (ref.3) Structure from motion (ref.4)


6

• Notable difference among the mentioned four techniques is that different information is utilized to perform the task of 3D reconstruction.

• Among various structure from motion techniques spatial transformation approach is one important branch.

• The beauty of the spatial transformation model is that they are sparse in nature and extract the depth information of only important features.


7

• Koo and lam (ref.5) proposed a 3D reconstruction algorithm(SFM) based on Similarity Transform Measurements.

• The algorithm utilizes group of face images to reconstruct the sparse 3D structure.

• 3D to 2D projection model is formulated using the 2D point sets.

• The solution vector minimizing the model is searched using the Genetic Algorithm


8

Five Sample Poses of person (myself) :Different Pan Angles

Front (0,15,0) (0,30,0) (0,-15,0)

3a 3b 3c 3d 3e

(0,-30,0)


9

Four Sample Poses of person 5 of Head Pose Database:Different Tilt and Pan Angles

Front (-15,0,0) (-15,15,0) (15,0,0)

4a 4b 4c 4d


10

3D to 2D Projection Model

• The projection of the 3D face model to the corresponding 2D face via given rotation matrix and scale is given by 3D to 2D transformation under orthographic projection is performed using the transformation:

pi = si * Ri2x3 * C + Ti for i = 1,2,3,4,5…N.

where N is the number of non-frontal-view 2D face images, si, Ti and Ri denote the scaling factor, the translation Matrix and the rotation matrix between the frontal view image and the ith non-frontal-view face image, C: 2D coordinates with Candie Depths, respectively.


11

11 12 13

21 22 23

1 2 3 411

1 2 3 412

1 2 3 41 2 3 4

i i i

i i ic c c c

X X X Xr r r tx x x xY Y Y Y

ty y y y r r rZ Z Z Z

2D Co-ordinatesNon-Frontal View Rotation

Matrix

2D Coordinates Frontal View &Initial Candide

Depths

2D Translation


12

Objective Function to be optimized

2 3

2min 2 3,

mini i x

i i i x is RD p s R C T


13

Optimization using Genetic Algorithmkoo and Lam Algorithm

• The GA encounters a heavy computational burden.

• Moreover the GA is time consuming and the accuracy depends on the control parameter set which requires adjustment, which presents practical difficult problems for feasible operation for a chromosome of moderate size and the situation is difficult if the chromosome size increases.

• To overcome these practical difficulties in finding the solution vector, Differential Evolution Optimization is employed.


14

Differential EvolutionOur Approach

• The method of Differential Evolution functioning is similar to genetic Algorithm approach.

• DE can be applied to real-valued problems with much more ease than a GA.

• The ideal behind the method of differential evolution is that the difference between two vectors yields a difference vector which can be used with a scaling factor to traverse the search space.

• The Solution vector is known as Genome.


15

DE GA PSOSA

Cr = 0.1F = 0.6Population Size =120Iterations = 250Strategy [1 − 5]

Crossover rate = 80%Mutation rate =20%Population Size = 1200 Iterations = 250Rank SelectionMax. Run Time = 2.6 Sec

c 1 = 0.6 c2 = 1.0Population Size = 200Iterations = 250

T start = 10T end = 1E − 9 Exponential CoolingSchedule: T k+1 = 0.8 · T k

Parameters Used in Optimization Algorithms


16

Fig - Index

Actual Pose

DE – S1 GA PSO SA

4c (-15,15,0) (17,-2,-1) (14,8,11) (13,4,3 ) (15,0,-1)

4d (15,0,0) (14,-16,0) (-4,-11,-9) (12,-21,-1) (15,-16,-1)

Table.IIIBest Estimated Poses of Person 5 Using DE & other Optimization Algorithms.

4c: (-15,15,0) 4d: (15,0,0)


17

Fig - Index

ActualPose

DE – S1 GA PSO SA

3b (0,15,0) (0,13,-1) (7,12,0) (2,19,-4) ( 0,15,-1)

3e (0,-30,0) (0,-31,0) (-3,-29,0) (-3,-29,-1) (0,-30,0)

Table. IVBest Estimated Poses of Person 1 Using DE & other Optimization Algorithms.

3b : (0,15,0) 3e: (0,-30,0)


18

OptAlg 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

DE_S1 6 1 3 0 15 0 21 15 16 6 0 3 0 15 2

GA 6 2 2 1 22 2 31 19 19 8 1 2 1 22 7

PSO 3 1 1 1 14 0 19 12 13 6 0 2 0 13 4

SA 6 0 2 0 15 0 21 14 15 6 0 2 0 15 2

Table : VEstimated Depth Values of Person 1 (myself)

Note: Depth values obtained are floating point numbers they are rounded to nearestInteger and mentioned in the paper and here


19

OptAlg 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

DE_S1 4 0 2 1 15 1 20 14 14 7 1 3 1 14 3

GA 6 1 2 1 18 0 25 17 17 8 1 3 1 18 4

PSO 5 0 2 0 13 0 18 12 13 4 0 1 0 13 1

SA 6 0 2 0 15 0 21 14 15 6 0 2 0 15 2

Table : VIEstimated Depth Values of Person 5

Note: Depth values obtained are floating point numbers they are rounded to nearestInteger and mentioned in the paper and here


20

Conclusion• Differential Evolution Optimization is used to optimize the objective

function.

• Other soft computing techniques are implemented and compared for this task.

• Experimental results signify that DE outperformed the other techniques in estimation of DEPTHS of important face feature points.


21

References1. Zhang, Ruo, et al. "Shape-from-shading: a survey." Pattern Analysis and Machine

Intelligence, IEEE Transactions on 21.8 (1999): 690-706.2. Romdhani, Sami, and Thomas Vetter. "Efficient, robust and accurate fitting of a 3D

morphable model." Computer Vision, 2003. Proceedings. Ninth IEEE International Conference on. IEEE, 2003.

3. Castelán, Mario, and Edwin R. Hancock. "A simple coupled statistical model for 3d face shape recovery." Pattern Recognition, 2006. ICPR 2006. 18th International Conference on. Vol. 1. IEEE, 2006.

4. Shapiro, Larry S., Andrew Zisserman, and Michael Brady. "3D motion recovery via affine epipolar geometry." International Journal of Computer Vision 16.2 (1995): 147-182.

5. Koo, Hei-Sheung, and Kin-Man Lam. "Recovering the 3D shape and poses of face images based on the similarity transform." Pattern Recognition Letters 29.6 (2008): 712-723.


22

Queries ?


23

THANK YOU

3d face structure estimation using evolutionary algorithms

Engineering