Pamela FoxENG 449Viet Nguyen

Creative Project: Can Someone’s Race be Predicted by their Facial Features?

The purpose of this project is to discover whether a person’s race can be determined solely by looking at a black&white photo of their face, and if so, determine which facial features are the most important in determining their race. Both because this project is ultimately for an Asian-American class and because of my own bias in believing East Asians have particularly characteristic faces, the project focuses on contrasting East Asians with other races.

Methodology:I setup a quiz that, when taken by multiple participants, would

yield statistics on the likelihood of a particular facial feature determining a race or just the general ability to determine race. There are four sections to the quiz, and an introduction screen to obtain demographic information about the respondents.

Screen 1:In the introduction, I explained the terms that would be used

throughout the quiz. Racial terms are always controversial, so I found those definition that satisfied the distributions of photos I was using and instructed users to use that definition. This is a tricky matter however – many people already have a definition in their brain corresponding to these terms, and it is difficult in timed quizzes to not follow instinct. I considered instructing the users to use their own definitions, but then I wouldn’t be able to verify a correct answer

versus an incorrect answer when analyzing the results. I also asked for their race and gender in case I wanted to compare their stats to their ability to determine race in the results.

Quiz 1:This is one of two quizzes to test the respondent’s ability to

determine race based on particular facial features. All of the quizzes are based on a database of 20 photos: 8 East Asian, 4 Hispanic, 4 Indian, and 4 Caucasian. Because of the ease of finding female photos online, all of the photos are of females from that racial group. For this quiz and the second quiz, I cut the photos up into 3 sections: eyes, nose, mouth. The respondent is randomly presented with a face section and asked to, as quickly as possible, choose the correct race for the photo that the section came from. The user will see 60 face sections by the end of the quiz, covering all 20 faces.

Quiz 2:This is the exact same as the last quiz, except it is not timed. I

added in the timing factor to see if it would make a difference in number of correct answers for the respondent to have unlimited response time. This would perhaps indicate another level of reasoning occurring, perhaps one resulting from social education of some kind.

Quiz 3:This is the first of two quizzes to test the respondent’s ability to

determine race determined on seeing the complete face. Since it is the first, it is also timed.

Quiz 4:This is the exact same as the last one, but it is not timed.

Results:

The quizzes were advertised mostly to students through an online social network. There were a total of 230 respondents, but 100 of those were thrown out due to problems in the data or their failure to complete the entire survey. All the data shown reflects the 130 remaining respondents. The graph below shows the race of the respondents. The gender variable was lost due to a technical difficulty. This shows mostly white respondents, so we can take their responses to represent those of the white community.

Demographics of Respondents

0

10

20

30

40

50

60

70

80

Hispanic Other White East Asian SoutheastAsian

PacificIslander or

NativeHawaiian

AfricanAmerican

South Asian AmericanIndian orAlaskaNative

Races

Nu

mb

er

of

Re

sp

on

de

nts

Series1

Mouths, Timed - FRACTION OF CORRECT GUESSES PER PHOTO

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

A-1 A-2 A-3 A-4 A-5 A-6 A-7 A-8 C-1 C-2 C-3 C-4 H-1 H-2 H-3 H-4 O-1 O-2 O-3 O-4

% C

orr

ec

t G

ue

ss

es

AVG: .55 AVG: .64 AVG: .32 AVG: .1

Noses, Timed - FRACTION OF CORRECT GUESSES PER PHOTO

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

A-1 A-2 A-3 A-4 A-5 A-6 A-7 A-8 C-1 C-2 C-3 C-4 H-1 H-3 H-4 O-1 O-2 O-3 O-4

Fra

cti

on

of

Co

rre

ct

Gu

es

se

s

AVG: .54 AVG: .66 AVG: 33 AVG: .1

Question: Can facial features be used to determine the race of a black&white photo?Answer: Yes, depending on the feature. There were consistently significantly more correct answers by respondents when presented with eye segments. The nose and mouth segments had approximately the same percentage of correct answers. The graphs below illustrate this result (photos in order of Asian, Caucasian, Hispanic, Other):

Eyes, Timed - FRACTION OF CORRECT GUESSES PER PHOTO

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

A-1 A-2 A-3 A-4 A-5 A-6 A-7 A-8 C-1 C-2 C-3 C-4 H-1 H-2 H-3 H-4 O-1 O-2 O-3 O-4

Race

% C

orr

ec

t G

ue

ss

es

AVG: .79 AVG: .86 AVG: .49 AVG: .23

Noses, Untimed

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

A-1 A-2 A-3 A-4 A-5 A-6 A-7 A-8 C-1 C-2 C-3 H-1 H-2 H-3 H-4 O-1 O-2 O-3 O-4

Race

% C

orr

ec

t G

ue

ss

es

AVG: .54 AVG: .55 AVG: .34 AVG: .11

Mouths, Untimed

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

A-1 A-2 A-3 A-4 A-5 A-6 A-7 A-8 C-1 C-2 C-3 C-4 H-1 H-2 H-3 H-4 O-1 O-2 O-3 O-4

Race

% C

orr

ec

t G

ue

ss

es

AVG: .52 AVG: .62 AVG: .31 AVG: .12

Eyes, Untimed

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

A-1 A-2 A-3 A-4 A-5 A-6 A-7 A-8 C-1 C-2 C-3 C-4 H-1 H-2 H-3 H-4 O-1 O-2 O-3 O-4

Race

% C

orr

ec

t G

ue

ss

es

AVG: .71 AVG: .81 AVG: .49 AVG: .23

Question: Are particular races easier to distinguish by facial feature than others?Answer: Yes. According to our results, both East Asians and Caucasians are significantly easier to distinguish by facial features than others. This may be because Hispanic is a rather broad category (and not actually a race according to our census anymore), and because the other category was “Other,” which doesn’t imply any particular characteristics. People may be inclined to categorize as non-Other. If I were to re-do the quiz I would probably replace Hispanic with Mexican, or something more equivalent in geographic scope to East Asians and Caucasians.

Question: Can the race of a particular person be determined by a complete black&white photo of their face?Answer: Yes, with a more than 50/50 chance for the races chosen. Once again, East Asians & Caucasians were significantly easier to guess correctly than Hispanic. It was difficult for people to categorize the Indian photos as Other, so it may help to have a subset of options to choose from when deciding the race. It seemed that makeup made it more difficult to determine race, as some of the lowest correct answer percentages came for photos of women in makeup. It may be that women sometimes use makeup to transform themselves into the current stylish race, creating a confused set of visual cues. The graphs below illustrate these results (notice A5, H1, O4):

Faces, Timed - ANSWERS

0

20

40

60

80

100

120

140

A-1 A-2 A-3 A-4 A-5 A-6 A-7 A-8 C-1 C-2 C-3 C-4 H-1 H-3 H-4 O-1 O-2 O-3 O-4

Races

Ra

ce

Gu

es

se

s

A C H O

Mouths, Untimed

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

A-1 A-2 A-3 A-4 A-5 A-6 A-7 A-8 C-1 C-2 C-3 C-4 H-1 H-2 H-3 H-4 O-1 O-2 O-3 O-4

Race

% C

orr

ec

t G

ue

ss

es

AVG: .52 AVG: .62 AVG: .31 AVG: .12

Mouths, Timed - FRACTION OF CORRECT GUESSES PER PHOTO

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

A-1 A-2 A-3 A-4 A-5 A-6 A-7 A-8 C-1 C-2 C-3 C-4 H-1 H-2 H-3 H-4 O-1 O-2 O-3 O-4

% C

orr

ec

t G

ue

ss

es

AVG: .55 AVG: .64 AVG: .32 AVG: .1

Question: How does timing affect a person’s ability to correctly determine race?Answer: It has negligible effect. There were not significant differences between timed and untimed quizzes. Occasionally people did better, occasionally worse. This would indicate that race can actually be determined quickly on a day-to-day basis- it can be thought of as a snap judgment. The graphs below illustrate two similar results, timed vs. untimed:

Conclusions:When I

proposed the project, I expected to find that only East

Asians would be easy to recognize based on facial features. However, I’ve been proved wrong. East Asians and Caucasians showed comparable results throughout. This surprises me, as Caucasians is really a much bigger category than East Asians, and I always think of us as the unmarked race. Of course, this is typical ethnocentric behavior. But now, seeing these results, I realize that to East Asians, they may think of themselves as unmarked and us as marked. In fact, it appears that both of us are marked, in our particular way. I also believed at the beginning I would discover that eyes would be the best determiner of race, and I was correct on that hypothesis. It is significant to note that overall accuracy did increase between viewing just eyes and viewing complete faces however – this indicates that the nose and mouth, though not distinguishing on their own, can combine with eyes in a recognizable way. There is a synergy in facial features that enhances visual cues for race.

Future Work:This was an amateur quiz created by someone not well versed in

social experiments. It would be interesting to see it redone at a larger scale, with more careful attention paid to control factors. In this quiz I mixed up celebrities and non-celebrities, made-up women vs. plain-faced women, etc. These should be separated out to test the actual effect of them. Also, I eliminated color from the photos to prove that it was not necessary to determine race, and that race is not just a result of color. There could be a final quiz with color to see how greatly that affects the accuracy of the answers. There are many interesting questions that quizzes like these could answer about our visual perception of race.