project report: team 1 miss universe prediction 2014 · project report: team 1 miss universe...

Project Report: Team 1Miss Universe Prediction 2014

Kanchan Chandnani, Utpal Thakar, Sreenath Mullassery, and RuchiKhandelwal

Department of Computer Science, Stony Brook University,Stony Brook, NY 11794-4400

{kchandnani,uthakar,smullassery,rkhandelwal}@cs.stonybrook.eduhttp://www.cs.stonybrook.edu/~skiena/591/projects

1 Challenge

Our challenge is to predict Miss Universe 2014. The 63rd Miss Universe pageantwill be held on January 25, 2015 at the U.S Century Bank Arena at Florida Inter-national University in Doral, Florida, USA. Gabriela Isler(current Miss Universe)of Venezuela will crown her successor at the end of the event. Our model com-putes the probability of winning for each of the 88 countries(refer Fig. 1) thatwill be contesting for the title this year. Kyrgyzstan and Rwanda will be debut-ing this year. There are 27 contestants this year who are crossovers, i.e., theyhave previously competed or will compete at other international beauty pageants(e.g. Miss World, Miss International, Miss Tourism International, etc.).

Fig. 1. Miss Universe 2014 participant countries[1]

2 kchandnani, uthakar, smullassery, rkhandelwal

2 History/Background

2.1 The Beginning of Beauty Pageants[2]

Beauty Contests have been around since ancient Greece and the Judgment ofParis. Beauty pageants were considered entertainment in the old city of Troy.The judges included the great minds of the city like warriors, public speakers,philosophers, poets, actors and sculptors. There exists an ancient custom inEurope, wherein symbolic kings and queens for May Day are chosen and beautifulyoung women symbolize their nations’ virtues and other abstract ideas.

2.2 The Maiden Beauty Queen[2]

It was in 1854 when the first beauty pageant in America was held, but this wasclosed down due to public protests. It was in 1880s when the modern beautypageants commenced. Specifically in 1888, the first woman with the ‘beautyqueen’ title was crowned in Spa, Belgium. Contestants had to send their pho-tos and a short description about themselves to join the competition. Only 21contestants were short-listed for the final judging, which was a formal eventattended by men in tuxedos and women in long gowns. The selected beautieswere secluded from the general public, and they had to ride in a closed carriageand live in a separate building without access to the outside world. 18-year-oldCreole won the very first ‘beauty queen’ title.

By the early decades of the twentieth century, attitudes had begun to changeabout beauty pageants. Prohibitions against the display of women in public be-gan to fade, though not to disappear altogether. It was not until the twentiethcentury that beach resorts began to hold regular beauty pageants as entertain-ments for the growing middle class. In 1921, in an effort to lure tourists tostay past Labor Day, Atlantic City organizers staged the first Miss AmericaPageant in September. Stressing that the contestants were both youthful andwholesome, the Miss America Pageant brought together issues of democracyand class, art and commerce, gender and sex – and started a tradition thatwould grow throughout the century to come.

2.3 Origin of the “Miss America” title[2]

The example of Belgium was followed by other countries, most specially byGermany and USA. In 1921, the first ‘Miss America,’ a 16-year-old MargaretGorman, was crowned. The event would have been known as the National BeautyTournament if not for a local newsman who exclaimed, “Let’s call her ‘MissAmerica!’ ”

2.4 Statewide Beauty Pageants[2]

Before participating in Miss America, every state in the country had to choosea representative through their state-wide beauty pageants like Miss Florida,

Miss Universe Prediction 2014 3

Miss Texas and Miss California for other states. Miss America is a scholarshippageant. Unlike its famous modern match, Miss USA, it is not a preliminary toan international event.

Nowadays, winning a state-wide beauty contest is a big leap towards a moreglorious crown—maybe a national crown, or even better, an international title.There are at least 24 national pageants in the USA. But the most publicized andpopular aside from Miss America and Miss USA are Miss World United States,Miss United States International and Miss Earth United States. These are thepreliminaries to even more glorious international competitions.

2.5 Worldwide Beauty Contests[2]

Miss USA will represent the nation to Miss Universe, which has been a popularinternational beauty contest since 1952. There are, so far, seven Miss Universewinners from USA. The long-time rival of this Trump-owned contest is MissWorld, which was formed in 1951. Miss World United States is held to choosethe country ‚ s representative for this event. And so far, there are three MissWorld winners from USA. In 1960, Miss International was formed in Tokyo,the same year Miss United States International was formed. At least three MissInternational title holders are from USA. Miss World is the youngest among themajor beauty pageants. It was just formed in 2001 and is more environment-focused. USA chooses the representative for this through Miss Earth UnitedStates. So far, USA has never won a title. The mere participation in a beautypageant is already a great achievement that any contestant can be proud of. Itovercomes fear and shows confidence and self-esteem among candidates. Winningis just a bonus. After all, beauty is a very subjective matter. And the differentbeauty pageants aim to define that subjectivity differently[2].

3 Literature Review

Sense Beauty via Face, Dressing, and/or Voice[3] talks about the aspectsabout finding a beauty score of a person given their facial characteristics, voicesample and dressing characteristics:

- The paper describes how elements of beauty collaborate. There are lot of papersthat takes one aspect (such as the face) to determine the beauty quotient of aperson. This paper derives a correlation matrix to determine how the three facetsof beauty correlate in order to determine beauty score of a person. Also, easternand western country data is distinguished and people from both the countrytypes rate all contestants. This also gives information about how differentlyeastern and western countries rate beauty. For example, the authors states thatthe correlation between dressing and voice is highest for eastern countries.

- The process of determining a beauty score of a person was divided into 3 stages.The first stage involved 40 participants rating each of the people on their face,


dressing and voice. The participants gave pair wise preferences rather than arating to each person. This is found to be an easier way to judge the beauty ofpeople. Even though the pair wise preferences are an easier way to rate people onbeauty, they are difficult to comprehend when modeling. Ranking SVM was usedto convert preferences to a global attractiveness score. The next stage involvedanalyzing the images and video samples in order to get a global attractivenessscore. Sub-modules considered for each of feature measurement are listed below:

– Facial FeaturesLocal Binary Patterns : which provide skin texture details, i.e. how thesmoothness of the skin determines beauty.Gabor Filter Response : The Gabor filter response provides informationabout the facial shape.Color Moment : Color Moment provides the mean and variance of color ofthe face. The uniformity of color is correlated with beauty.

– Dressing FeaturesColor moment and Color histogram : The Pattern and color of the dressesare correlated with beauty.Local Binary Patterns and Histogram of Oriented Gradients : These twofeatures are used to determine the texture of the dresses.

– Voice FeaturesEach voice feature is related to one of the audio dimensions traditionallydefined in audio theory. The audio sequence is decomposed into successiveframes, which are then converted into the spectral domain, frequency do-main and pitch domain. Accordingly, the audio features related to pitch,to spectrum (zerocross, low energy, rolloff, entropy, irregularity, brightness,skewness, flatness, roughness), to tonality (chromagram, key strength andkey self-organising map)are extracted in order to find the best combinationof these characteristics which can form a melodious voice sample.

The final stage involves combining these three features in order to get a globalattractiveness score.

A Survey of Perception and Computation of Human Beauty[4] is an-other technical paper in this domain. It talks about various approaches taken inorder to determine the beauty of a person. It talks about the theories of attrac-tiveness which include the following:

– The composite faces theory: Studies have shown that the more the facesadded to the composite, the more is the perceived beauty. Thus an averageof multiple faces is perceived as more beautiful than the original faces.

– The symmetry theory: Face symmetry is considered to be an importantaspect to the perception of human beauty.


– The skin and texture theory: The appearance of the skin has an overallimpact on the perception of attractiveness. Homogeneous skin( in terms ofdistribution of features and skin surface topography are important charac-teristics of beauty. Skin texture, thickness, elasticity, and wrinkles impactbeauty.

– The geometric facial feature theory: Soft tissue reference points are impor-tant aspects to determine beauty. A facial representation is obtained bycalculating a set of geometric features (i.e., landmarks on the face) usingthe major facial points, including facial outline, eyebrows, eyes, nose, andmouth. It is possible to modify the attractiveness score by modifying thesegeometric features.

– The facial thirds theory: This theory aims to assess the facial height. Thetheory states that a well-proportioned face may be divided into roughlyequal thirds by drawing horizontal lines through the forehead hairline, theeyebrows, the base of the nose, and the edge of the chin. Moreover, thedistance between the lips and the chin should be double the distance betweenthe base of the nose and the lips.

– The facial fifths theory: This theory is similar to the facial thirds theory butit measures the width between soft tissue points on the face.

– Bodily attractiveness: The most dominant bodily cue that affects the per-ception of female attractiveness (excluding the face) appear to be shape andweight. The shape cue is concerned with the ratio of the width of the waistto the width of the hips (the waist-to-hip ratio (WHR)).

– Vocal attractiveness: A beautiful sounding voice increases the attractivenessquotient of a person.

The analysis of beauty is done using the following three algorithms:

– K nearest neighbors to classify participants into 4 beauty categories.– Support Vector Machines.

Amachine learning predictor of facial attractiveness revealing human-like psychophysical biases [5]

Machine learning predictor of facial attractiveness revealing human-like psycho-physical biases presents a learning model that automatically extracts measure-ments of facial features from raw images and obtains human-level performance inpredicting facial attractiveness ratings. The machine’s ratings are highly corre-lated with mean human ratings, markedly improving on recent machine learningstudies of this task.

The research involved gathering a database of images which could be classi-fied. The chosen database was composed of 91 facial images of American females,taken by the Japanese photographer Akira Gomi. All 91 samples were frontalcolor photographs of young Caucasian females with a neutral expression. Allsamples were of similar age, skin color and gender, to give an unbiased judgmentof beauty. The crowd sourcing involved rating the 91 images in the database forattractiveness by 28 human raters (15 males, 13 females) on a 7-point Likertscale (1 = very unattractive, 7 = very attractive). Each rater was asked to view


the entire set before rating in order to acquire a notion of attractiveness scaleso as to maintain an unbiased judgment.

To extract facial features they developed an automatic engine that is capableof identifying eyes, nose, lips, eyebrows and head contour. In total, they measured84 coordinates describing the locations of those facial features. Several regionsare automatically suggested for sampling mean hair color, mean skin color andskin texture. The feature extraction process was basically automatic but somecoordinates needed to be manually adjusted in some of the images. The facialcoordinates are used to create a distances-vector of all 3486 distances betweenall pairs of coordinates in the complete graph created by all coordinates. Foreach image, all distances are normalized by face length (as measured from thecoordinate at the top of the forehead to the coordinate at the bottom of thechin). In a similar manner, a slopes-vector of all the 3486 slopes of the linesconnecting the facial coordinates is computed.

Central fluctuating asymmetry (CFA) is calculated from the coordinates aswell. CFA corresponds to the sum of the absolute values of the differences of themidpoints of adjacent horizontal lines which connect matching bilateral facial.The application also provides, for each face, hue, saturation and value (HSV)values of hair color and skin color, and a measurement of skin smoothness.Smoothness of skin was calculated with an edge-detection algorithm in whichmany detected edges suggest a low level of skin smoothness. Combining thedistances-vector and the slopes-vector yields a vector representation of 6972 ge-ometric features for each image. Since strong correlations are expected amongthe features in such representation, principal component analysis (PCA) wasapplied to the 6972 geometric features, producing 90 principal components thatspan the sub-space defined by the 91 image vector representations. The geo-metric features are projected on those 90 principal components to produce 90de-correlated eigen features representing the geometric features of the images.Eight non-geometric measured features were not included in the PCA analy-sis, including CFA, smoothness, hair color coordinates (HSV) and skin colorcoordinates. These features are assumed to be directly connected to human per-ception of facial attractiveness and are hence kept at their original values. These8 features were added to the 90 geometric eigen features, resulting in a total of98image-features representing each facial image in the dataset.

Machine attractiveness ratings of all sample images obtained a high Pearsoncorrelation of 0.82 (P-value < 10−23 ) with the mean ratings of human raters(the learning targets), corresponding to a normalized mean squared error of 0.39.


4 Data Sets

We have collected data by scraping and manually entering data from Wikipediaand the Miss Universe website. Our dataset can be divided into the following:

Country Data: We have data about the performance of countries in past MissUniverse Contests. We have data about how many times the country has wonMiss Universe, how many times the country was the 1st/2nd/3rd/4th runnerup, how many times the country has reached the semi-final stage and how manyattempts the country has made in the Miss Universe pageant. We used OlympicsScoring mechanism for calculating country score on basis of our dataset of per-formance of countries in Miss Universe contest.

The matrix consists of 86 rows(countries). We have also included the CountryPopulation, Country Age, Literacy Rate and GDP/ Capita. We have similar datamatrices for Miss International and Miss World Contests.

Contestant Characteristics: We also have Age, Height and Beauty Scoresvalues for all Winners since 1952(Year Miss Universe started) and contestantsfrom 2009 to 2013. The Resultant Data Matrix is for 488 contestants. This datahas been collected from Wikipedia.

Our final data matrix is as below

Fig. 2. Final Data Matrix with 488 rows

5 Beauty score

In this section, we discuss three approaches followed to generate beauty scorefor contestants. We have close-up photos of contestants for years 2009 to 2013from the Miss Universe Website. And we have collected images of winners from1952 to 2013.

5.1 Le Hou’s Model[6]

Le Hou’s Model uses Neural Networks in order to obtain a beauty score for im-ages according to the facial feautures. The faces are detected from the imagesusing Viola-Jones’ face detection method. The model involves a two-layer feed-forward neural network with adaptive piecewise polynomial activation functions.A feed forward neural network is one wherein there is no feedback between the


layers of the neural network. A piecewise polynomial activation function con-siders two activation functions and cuts them at a value of a variable in thefunctions for which the value of both the functions are equal. This makes theactivation function more flexible in nature. The model is adaptive in nature andthis increases the learning rate according to the error in each iteration. Thetraining of the model involved minimizing the error function :

Er(W ) = 0.5 ∗ (∑x

(yx − tx)2 + λ‖W‖22) (1)

W is the vector of parameters to be learned in the training stage. yx is thepredicted value of x and tx is the ground truth of x. λ is the regularizationparameter which is determined by cross validation tests. In order to train themodel from the input images, each image is preprocessed to produce 3 images.These 3 images (called eigenfaces) and the pixels of the image were given asfeautures to the model. On testing, this model outperformed sigmoid neuralnetworks.

5.2 Anaface[7]

Anaface determines the beauty score of a face based on its symmetry. The pro-gram guides the users to place 17 precise markers at various points of their face,such as the forehead, tips of the ears, sides of the mouth and so on, providing ref-erence points so the size and placement of features can be measured and rated.The algorithm for Anaface is based on a combination of neo-classical beautymeasures and statistical measures. The algorithm is based on the hypothesisthat symmetry plays a vital role in determining the beauty of a person. Also,certain ratios between the distances, for example, the ratio of the distance be-tween the two ears and the forehead and lips are proposed to play an importantrole in determining beauty scores.Refer Fig. 3 for rating of image of a contestant(Miss USA 2014) using Anaface.

5.3 Our Beauty Scores

Each one of our team members rated the contestants based on our perception ofbeauty. We then averaged these scores to come up with a beauty score for thecontestants.

5.4 Variation in Beauty Scores Across Scoring Tehniques

We found the beauty scores of a contestant(Miss USA 2014) based on all theaforementioned scoring techniques(refer Fig. 4).


Fig. 3. Rating of a contestant(Miss USA 2014) using Anaface

Fig. 4. Comparison among beauty scores for Miss USA 2014


6 Observations

On computing the correlation between the attributes in our data matrix, wecame across some interesting revelations.

6.1 Correlations between beauty scores

Lee Huo’s model’s beauty score had a negative correlation (-0.22) with the prob-ability of winning Miss Universe(refer Fig. 5). This was against our intuition.One of the possible reasons that we felt that caused this negative correlation wasthe black and white, pixelated, not front-facial images of winners in the earliereditions of the contest. So, we removed beauty scores computed for winners from1952-1980 and observed that the correlation increased to -0.018.

Fig. 5. Correlation among techniques used for beauty score

We tried using Anaface for scoring the Miss Universe Contestants but founda drawback in the scoring mechanism. We observed that if the image had aperfectly straight face, the score was higher than if the image had a tilted face.We tried looking for straight-faced images of contestants but we could not findsuch images for a majority of the contestants. We realised that even if we scored2 different images of the same person, there can be a vast variation between thescores due the angle of the face. We decided not to include Anaface scores in ourmodel as this would give undue advantage to certain contestants.


To further analyze the correlation between beauty score of a contestant andchances of being a winner, we(four of us) experimented by giving scores between1-10 to the contestants based on our perception of beauty and computed themean of our individual scores for each contestant. Correlation between our indi-vidual scores was in the range 0.5-0.6. Our beauty score had a positive correlationof 0.35 with contestant being a winner (as expected).

6.2 Correlations amongst all features

The correlation matrix(refer Fig. 6) suggest some other interesting hypothesis.Age is negatively correlated with winner which means that younger contestantshave better chance of becoming a winner. However height has a small negativecorrelation indicating that taller contestants may not be preferred by judges tobe declared as winners. Country parameters like country age, country GDP percapita, country population and literacy also play a role in deciding the winnerprobability as is evident from their correlations with winner in above matrix.

6.3 Betting odds

Betting Odds Speak: Miss Philippines, Miss Spain and Miss USA Top3 in Miss Universe 2013.[8]

This article talks about the betting odds for Miss Universe 2013. It men-tions how the betting statistics changed over time before the Miss Universe 2013events. The findings are summarized below

Table 1. Performance of Betting odds against Advanced and Baseline for 2013

Results 2013Top 10

OCT 29Country

OCT 29Odds

NOV 3Country

NOV 3Odds

AdvancedModel

BaselineModel

Venezuela Philippines 4.5 Philippines 3.0 USA USASpain USA 8.5 Spain 6.5 Venezuela VenezuelaEcuador Spain 17 USA 11 Japan JapanPhilippines Panama 12 France 12 Sweden BotswanaBrazil India Panama 13 Brazil Puerto RicoDominicanRepublic Australia Venezuela 13 Colombia Brazil

Great Britian France 17 Poland 13 Australia SwedenIndia Russia 15 Russia 15 Russia FinlandUkraine Colombia 15 India 15 Chile AngolaUSA Ukraine 8.5 Australia 6.5 Puerto Rico Australia

The article also talks about how the betting payouts are calculated. Theformula used is: X (Amount of Bet) times Odds(3,0, 6.5, etc) = Payout. If abettor placed a bet $100 on Miss Arida to win the Miss Universe title, when itsbetting odds are 3.0 then the bettor gets a payout of $300.


Fig. 6. Correlation matrix with contestant and country features


7 Baseline Model

This is a simple model that predicts Miss universe 2014 based on all results fromthe first edition of pageant in 1952 to the most recent edition in 2013.

Data (on which the model is based):Source - Wikipedia1) Country-wise listing [9] with count ofa) miss universe titlesb) their contestants being 1st runner-up, 2nd runner-up, 3rd runner-up and 4th

runner-upc) their contestants featuring in the semi-finalsAll these counts are mutually-exclusive (for eg. If miss USA is crowned miss uni-verse for up-coming edition, then number of miss universe titles that USA havewon is incremented by 1 and semifinals count remain the same from previousyear)Though this source had most of the data we needed, our model would seem lessmeaningful if we did not consider the total number of attempts/appearances ofthe county in the competition. There are countries with high success rate. Forinstance, Miss Kosovo have been 2nd runner-up one time and featured in thesemifinals three times of just 5 appearances.2) List of countries with the years competed in the pageant [10]With this data, we were able to parse and get the total number of attempts atthe pageant by the country.

ModelWe devised a ranking function that takes into consideration each of the countvalue in our data-set. This is a static ranking, thus the prediction remains thesame for all years.For example, Miss Colombia have featured in the semifinals 19 times and beenranked 1st runner-up 4 times, but won the title only once. So if we go by onlythe number of wins, Miss Colombia has a very less chance comparable to that ofNamibia (1 miss universe title, 1 semifinals entry). But it is quiet evident thatColombia has been performing far better than Namibia.At the initial stage, we devised a simple ranking function that gives weight factorto each of the count value in the data-set.Set X element (corresponding weight factor): Miss universe title (60%), 1st

runner-up (20%), 2nd runner-up (10%), 3rd runner-up (6%), 4th runner-up (3%)and Semifinals entry (1%).

Let CX : Count of occurrences in X for country over yearsWX :Weight factor attributed for occurrence in XRankingfunction : Average of [CX ∗Wx] for all sets in X for each country

Countries where ranked according to the metric. Refer Table 3 for top 16 coun-tries based on this model.


8 Advanced Model

We are modeling the challenge as a classification problem. Considering the sizeof our data matrix, for arriving at final model we used techniques recommendedby scikit for datasets with less samples (refer Fig. 7).

Fig. 7. Figure shows scikit-learn algorithms cheat-sheet

The dataset has less winners as compared to non-winners(only 1 winner in ayear), so we classify a contestant in years 2009-2013 as winner if she is amongstthe top 16 contestants. Also, the evaluation environment randomly picks thelosers so as to have an equal number of winners in the dataset. This ensurestraining phase of the algorithm is not biased to learn to just classify all contestantas losers to get a higher accuracy in prediction.

Since many of losers will not feature in the training data, we are doing a ran-dom shuffle split of the training data and run the algorithm multiple iterations.The average prediction(probability) of these multiple iterations leads us to ourfinal prediction.

We check the accuracy of our model by comparing its predictions for top 16against actual results. For 2014 contestants, the model predicts the probability ofwinning for every contestant and we rank contestants based on this probabilityto arrive at winner and top16 contestants.

The user interface (refer Fig. 8) provides a pluggable design to ease theprocess of running and analyzing our model for various cases. The classifierchoices allow you to select the classifier using which to make the prediction.

The evaluation year selection menu lets you select an evaluation year andmake prediction for that year. If the evaluation year is selected as 2009, thenthe dataset for year 2009 is removed from our test data and probabilities arepredicted for each of the contestant in 2009. This result is then compared withactual results for 2009.


Fig. 8. Figure shows User interface for our advanced model

Techniques shortlisted for our model:

– Logistic Regression– KNeighbours Classifier– Linear SVM– Naive Bayes– SGD Classifier– Ensemble Classifier

In the following subsections, we will discuss these techniques in brief.

8.1 Logistic Regression[11]

Logistic regression measures the relationship between the dependent variable andone or more independent variables by using probability scores as the predictedvalues of the dependent variable. The logistic function can take as input anyvalue from negative infinity to positive infinity and outputs values between zeroand one. Logistic regression can be binomial or multinomial. Binomial or binarylogistic regression deals with situations in which the observed outcome for adependent variable can have only two possible types (for example, "winner" vs."not winner"). Multinomial logistic regression deals with situations where theoutcome can have three or more possible types (e.g., "disease A" vs. "disease B"vs. "disease C").Our dataset has all top 16 contestants from 2009-2013 and all winners from1952-2013 labelled as 1(representing winners). Using logistic regression, we arepredicting the probabilities of each contestant being in top 16 for that year.Contestants are ranked according to this probability and the top 16 contestantsafter ranking gives our prediction of top 16 for that year.


8.2 KNeighbors Classifier[12]

In k-Nearest Neighbors classification, the output is a class membership. An ob-ject is classified by a majority vote of its neighbors, with the object being as-signed to the class most common among its k nearest neighbors. If k = 1, thenthe object is simply assigned to the class of that single nearest neighbor.Largervalues of k reduce the effect of noise on the classification, but make boundariesbetween classes less distinct. Our model based on this classifier has a k-value of5 and uses Euclidean distance as the distance metric.

8.3 Linear SVM[13]

SVM model is a representation of the examples as points in space, mapped sothat the examples of the separate categories are divided by a clear gap that isas wide as possible. New examples are then mapped into that same space andpredicted to belong to a category based on which side of the gap they fall on.support vector machine constructs a hyperplane or set of hyperplanes in a high-or infinite-dimensional space, which is used for classification, regression, or othertasks. We enabled probability estimates on this classification to come up withour list of top 16 prediction.

8.4 Naive Bayes[14]

The Naive Bayes classifier is probabilistic classifier which is based on Bayes the-orem with strong and naïve independence assumptions. Despite the naïve designand oversimplified assumptions that this technique uses, Naive Bayes performswell in many complex real-world problems. Naive Bayes has many variations. TheMultinomial Naive Bayes, the Binarized Multinomial Naive Bayes, the BernoulliNaive Bayes and Gaussian Naive Bayes. Our model of this classifier is basedon Gaussian Naive Bayes. The classifier is created from the training set usinga Gaussian distribution assumption and assumes that the features used in theclassification are independent.

8.5 SGD Classifier[15]

Ordinary gradient descent is a simple algorithm in which we repeatedly makesmall steps downward on an error surface defined by a loss function of some pa-rameters.Stochastic gradient descent (SGD) works according to the same princi-ples as ordinary gradient descent, but proceeds more quickly by estimating thegradient from just a few examples at a time instead of the entire training set.The gradient of the loss is estimated for each sample at a time and the modelis updated along the way with a decreasing strength. The ’log’ loss function isused in our model of logistic regression, which gives a probabilistic classifier.


8.6 Ensemble Classifier[16]

Since we found the accuracy of different models(refer Table 2) to be very close toeach other, we implemented the ensemble classifier. Ensemble methods combinethe predictions of several base estimators built with a given learning algorithmin order to improve generalizability / robustness over a single estimator. Ensem-ble classifier averages the probabilities calculated by individual estimators(listedabove) and comes up with a prediction for each contestant of the evaluationyear.

8.7 Evaluation of models

We ran above shortlisted algorithms for multiple iterations for years 2009, 2010,2011, 2012 and 2013 and analyzed the results. The results are tabulated inTable 2. Mean represents the number of correct predictions using that algorithmaveraged across years. Mean Accuracy represents ratio of testing data for whichpredicted value matches the actual value (i.e. top16 contestants are predictedcorrectly with value 1 and others with value 0). Mean Precision is the ratio tp/ (tp + fp) where tp is the number of true positives and fp the number of falsepositives. The precision is intuitively the ability of the classifier not to label aspositive a sample that is negative.

Considering the mean of correct predictions, logistic regression was found tohave a better accuracy but the margin in difference was so less that we couldnot base this to select logistic regression for our model.

So, we computed the variance in Mean across years for all the classifiers. Alower variance means that the algorithm has a stable accuracy across years inmaking the prediction. Logistic regression gave the lowest variance among all theclassifiers(refer Table 2). Thus, our final model is based on logistic regression.

Table 2. Performance Comparison of Models

Model K Neighbors LinearSVMLogisticRegression

NaiveBayes SGD Ensemble Baseline Random

Top 16matches

20092010201120122013

48666

54856

57856

46864

38573

47745

43253

23222

Variance 2 2.3 1.7 2.8 5.2 2.3 1.04 0.55Mean 6 5.6 6.2 5.6 5.2 5.4 3.4 2.66MeanP-value 9.99E-4 9.99E-4 9.99E-4 9.99E-4 0.109 9.99E-4 N.A 0.48

MeanAccuracy 78.21 76.87 77.83 76.39 77.34 73.57 77.90 49.43

MeanPrecision 37.5 33.75 36.25 32.5 35 35.59 38.75 15.89


Comparing final model to a random classifier(p-value test): To vali-date that the results of our model has a better accuracy than a random classifier,we used the p-value test for significance(refer Fig. 9). The p-value test determinesthe probability of obtaining the results from the model, if the null hyposthesisis true. The null hypothesis refers to the statement that there is no relationshipbetween the measured phenomena. If the p-value is small, it denotes that thenull hypothesis is unlikely to be true and there exists a relationship between themeasured phenomena. In our model, this denotes that there exists a relationshipbetween the features we have considered and the probability of winning the MissUniverse Title. The significance test was run for 1000 iterations and our modelwas found to have a better accuracy than this random classification even for1000 runs.

Fig. 9. Figure shows significance test of our model with a random classifier

9 Final Prediction and Conclusions

Our finalized model uses logistic regression for predicting probabilities for 2014contestants. The probabilities are computed using the predict_proba() constructin scikit. These probabilities represent the probability of a country to qualify intop 16 (as we had labelled countries qualified in top 16 also as winners). Fig. 10shows the distribution of these probabilities in a country map.Table 3 shows top16 predictions for 2014(in order) by using the advanced model, baseline modeland crowd sourcing.

9.1 Comparison with crowd sourcing results

We developed a hot-or-not model algorithm to predict the winner and top 16contestants for Miss Universe 2014 (refer Fig. 11). Voters played this game where


Fig. 10. Figure shows Miss Universe Probability distribution Map

Fig. 11. Figure shows Miss Universe Predictor game for public opinion polling


Table 3. Top 16 Prediction for 2014

Advanced Model Probability Baseline Model People’s choiceUSA 0.998 USA SpainVenezuela 0.998 Venezuela JapanJapan 0.996 Japan ColombiaBrazil 0.958 Botswana VenezuelaKosovo 0.938 Puerto Rico United StatesColombia 0.925 Brazil KosovoFinland 0.909 Sweden SwedenPuerto Rico 0.889 Finland FinlandAustralia 0.874 Angola AustraliaSweden 0.863 Australia IndiaSpain 0.854 Chile South AfricaSouth Africa 0.844 Colombia BrazilUkraine 0.826 Russia Puerto RicoNorway 0.814 Philippines ThailandGermany 0.804 India MontenegroPhilippines 0.781 Trinidad and Tobago Norway

they had to pick better of 2 choices in each attempt and the algorithm uses theseselections to arrive at the list of top 16. The results of 15 voters are comparedwith our prediction for 2014 and are ranked in the order of people’s choices(referTable 3).

9.2 Improvements

Design of a facial beauty ranking module will help improve our prediction.Also, more information on contestants like their education level, Bust-Waist-Hip(BWH) measurements and others is expected to improve the predictive ac-curacy of our model.

10 Acknowledgments

We would like to thank Professor Steven Skiena for his support and encourage-ment. We would also like to thank Le Hou for his efforts in helping us withcomputing beauty score for contestants.

11 Bibliography

References

1. http://upload.wikimedia.org/wikipedia/commons/thumb/6/61/Miss_Universe_2014_map.png/300px-Miss_Universe_2014_map.png

2. Miss Heart of USA, http://florida-beauty-pageants.com/


3. Sense beauty via face, dressing, and/or voice Tam V. Nguyen National Universityof Singapore, Singapore, Singapore Si Liu National Laboratory of Pattern Recog-nition, Beijing, China Bingbing Ni Advanced Digital Sciences Center, Singapore,Singapore Jun Tan National University of Defense Technology, Hunan, China YongRui Microsoft Research Asia, Beijing, China Shuicheng Yan National University ofSingapore, Singapore, Singapore MM ’12 Proceedings of the 20th ACM internationalconference on Multimedia

4. A Survey of Perception and Computation of Human Beauty, Hatice Gunes School ofElectronic Engineering ’I&’ Computer Science Queen Mary University of London,U.K., J-HGBU ’11 Proceedings of the 2011 joint ACM workshop on Human gestureand behavior understanding

5. A machine learning predictor of facial attractiveness revealing human-like psy-chophysical biases Amit Kagiana, Gideon Drorb, Tommer Leyvanda, Isaac Meil-ijsonc, Daniel Cohen-Ora, Eytan Ruppina Vision Research, Volume 48, Issue 2,January 2008, Pages 235–243

6. Smooth Function Network for Regression, Le Hou, Dimitris Samaras, Departmentof Computer Science, Stony Brook University ()

7. Anaface, http://anaface.com8. "Betting Odds Speak: Miss Philippines, Miss Spain and Miss USA Top 3 in Miss

Universe 2013." International Business Times - US ed. 3 Nov. 2013. Academic One-File. Web. 11 Dec. 2014.

9. Miss Universe country rankings, http://en.wikipedia.org/wiki/Miss_Universe_country_rankings

10. List of Miss Universe countries, http://en.wikipedia.org/wiki/List_of_Miss_Universe_countries

11. Logistic regression, http://en.wikipedia.org/wiki/Logistic_regression/12. K-nearest neighbor http://en.wikipedia.org/wiki/K-nearest_neighbors_

algorithm13. Support vector machines, http://en.wikipedia.org/wiki/Support_vector_

machine#Linear_SVM14. Naive Bayes classsifier, http://en.wikipedia.org/wiki/Naive_Bayes_

classifier15. Stochastic Gradient Descent, http://en.wikipedia.org/wiki/Stochastic_

gradient_descent16. Ensemble learning, http://en.wikipedia.org/wiki/Ensemble_learning

project report: team 1 miss universe prediction 2014 · project report: team 1 miss universe...

Documents