© copyright omron corporation sensing technology research center evaluation result of the interim...

© Copyright OMRON Corporation Sensing Technology Research Center

Evaluation Result

of the Interim Software

Sep.2, 2002

　

OMRON SENSING TECHNOLOGY RESEARCH CENTER

Omron CorporationSensing Technology Research Center

Vision-based Human Understanding Technologies Laboratory

To: Hong Kong Baptist University


Thanks

• Thanks to your considerable efforts, better results obtained with this interim software.


How to Evaluate

21

3

421

3

6

45

Eye(open)Mouth

1.In2.Out3.Upper4.Lower

1.left2.right3.top of upper lip4.bottom of upper lip 5. top of lower lip6. bottom of lower lip

Evaluation Criteria: ・ Distance between Ground truth and Detected position. (Normalized the width of face to 100 pixels) (See bellow figures: 4 points of open eye, 3 points of close eye and 6 points of mouth)Software version: ・ MD09Aug2002.zipUsing Program: ・ OMRON use one function (Fixed_DetectFeature) . ( To know the potential performance of the ASM Algorithm for mouth )

Ground Truth

Detected Position

21 4

1.In2.Out4.Lower

Eye(close)

OMRON employed the following evaluation criteria for the outline detection of the facial organ .


Evaluation Data Sets

OMRON evaluated with following Data Sets.

INC_EYEC INC_EYEO INC_MA INC_MI

StateNumber of Images

Number returned the result

Error or not return result

INC_EYEC Wide opened Eye 146 145 1(*)

INC_EYEO Closed Eye 147 147 0

INC_MA Shapes of Japanese vowel “a” 146 146 0

INC_MI Shapes of Japanese vowel “I” 147 147 0

Omron_a1fb1Normal face:

But high resolution image116 72 44(*)

*1) The list of these errors : See appendix


Summary of Evaluation

[Eye]- The number of error detections of right eye is more than the left eye. I guess that the different model or algorithm is used for the left and right eye detection.- The detection accuracy falls down due to lower eyelid, upper eyelid and eye blow.- I guess the main reason is the error of facial feature point detection.

[Mouth]- Top of upper lip and bottom of lower lip is more accurate.- The opened mouth, i.e. “A” and “I”, should be improved. The variety of the active shape models is important for robustness.- The detection accuracy falls down due to teeth and nose. [ etc ] - Many face and feature point detection error occurred with high resolution images.


Suggestion

Our goal is to construct the principle algorithm of the state estimation of facial organ until February 2003. It is important for us to develop the ASM algorithm for the outline detection of facial organ as soon as possible.

- We need to investigate the cause of failure in detecting the outline of facial organ. We should consider the errors of ASM algorithm and the others separately. This is the reason why we should evaluate the outline detection of facial organ using fixed corners of eyes and mouth.

- In order to realize the last goal until February, OMRON want you to begin to concentrate the research of the state estimation of facial organ.


Correct Examples

Closed eye

Wide opened Eye

shape of ”A” Mouth

shape of ”I” Mouth


Error Example:Closed Eye

I guess the main reason is the error of facial feature detection

Eye Brow Lower eyelid

Too short

Etc.

-> The length of left and right eye feature points differ greatly.

INC_EYEC


Error Examples:Wide opened Eye

Opened eye is mistaken as closed eye.

Influence of the lower eyelid

Etc.

Nonexistent relation of between outline of eye and outline of black eye

Eye blow

Identify the inside eye and the black eye

INC_EYEO


Error Examples:shape of ”A” Mouth

The left and right points of mouth are fixed landmark points.

Can’t spread the model Upper-lower is wrong. This cases is too much.

Influence of the ChinDon’t leach the Upper-upper.Influence of the teeth

INC_MA


Error Examples: shape of ”I” Mouth

Influence of the nose. (Noise of nose)thin Upper-lip

Influence of teeth?Can’t stretch the Upper-lower and Lower-Upper

INC_MIThe left and right points of mouth are fixed landmark points.


Error Examples: high resolution image (no.1)

These images were saved by JPG format. If I convert JPG to BMP, we got slightly different results. But Results is not so good.

omron_a1fb1


The accuracy of facial feature detection is low.Please check your software of face detection and facial feature detection.

Error Example: high resolution image (no.2)omron_a1fb1


Appendix


Detection Rate within 3pix (Left eye)Result

Detection Rate within 3pix(LeftEye)

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

90.0

100.0

Out In Upper Lower

%

INC_MAINC_MIINC_EYECINC_EYCOomron_a1fb1


Detection Rate within 3pix (Right eye)Result

Detection Rate within 3pix(RightEye)

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

90.0

100.0

In Out Upper Lower

%



Detection Rate within 3pix (Mouth)Result

Detection Rate within 3pix(Mouth)

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

90.0

100.0

Left Right Top of Upper Bottom of Upper Top of Lower Bottom of Lower

%





0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

90.0

100.0

Out In Upper Lower

%





0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

90.0

100.0

In Out Upper Lower

%



Detection Rate within 5pix (Mouth)Result


0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

90.0

100.0


%





0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

90.0

100.0

Out In Upper Lower

%





0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

90.0

100.0

In Out Upper Lower

%



Detection Rate within 7pix Mouth)Result


0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

90.0

100.0


%



Speed(Only outline detection of facial organResult

Average Time(Only facial organ detection)

0.0

5.0

10.0

15.0

20.0

25.0

30.0

35.0

40.0se

c

INC_MA 2.8 2.0 26.5 INC_MI 2.7 1.9 19.6 INC_EYEC 2.0 1.2 22.8 INC_EYCO 3.0 2.2 24.9 omron_a1fb0- F1 3.1 2.3 35.8

Left Right Mouth

*) Pentium4 2.2GHz


The List of Error FileFixed_DetectFeature(‘XXX\INC35\omron-ct\EYEC\

930077M3INC1EYECG0_000');

Fixed_DetectFeature('XXX\OmronDB\a1fb1\0066F1B1A1F6');

Fixed_DetectFeature('XXX\OmronDB\a1fb1\0004F1B1A1F6'); Fixed_DetectFeature('XXX\OmronDB\a1fb1\0070M2B1A1F6');



Fixed_DetectFeature('XXX\OmronDB\a1fb1\0044M1B1A1F6'); Fixed_DetectFeature('XXX\OmronDB\a1fb1\1017M2B1A1F6');



Fixed_DetectFeature('XXX\OmronDB\a1fb1\1076M2B1A1F6'); Fixed_DetectFeature('XXX\OmronDB\a1fb1\1029F2B1A1F6');













Fixed_DetectFeature('XXX\OmronDB\a1fb1\0055M4B1A1F6'); Fixed_DetectFeature('XXX\OmronDB\a1fb1\1069F1B1A1F6');


© copyright omron corporation sensing technology research center evaluation result of the interim...

Documents

omron use

lower eyeclose omron

right eye detection

left eye

upper lip

eye blow

eyeoclosed eye

outline of facial organ