l9. real world machine learning - cooking predictions

Cooking PredictionsA real case in the hotel sector

Andrés González Big Data Prediction Manager

[email protected] Twitter: @data_lytics

CleverTask Solutions SL - Big Data Business Unit 3

Agenda Business Need1

“Cooking” Predictions2

Gathering ingredients3

Cleaning and Transforming4

The recipe (the model)5

Tasting the dish6


Hotel Sector

• % room occupation. • Cancellation risk. • Income.


Business Need

Predict client’s

NATIONALITY

BEFORE

client

check-in


Staff Arrangement

Languages


Prepare Activities


Kitchen Arrangement


Customize Stay


… Details Make the Difference

In short, because…


Machine Learning basics



Can you find patterns in this data?

CleverTask Solutions SL - Big Data Business Unit

13


Historical Data Training Prediction

New Data Re-Training







Tasting the dish6


Tasting the Dish

Cooking

Transforming

15


Go to the market to buy ingredients

Cleaning


Evaluating Prediction Quality

Training the Model

Transforming and Feature Engineering

15


Gathering RAW data

Cleaning Data







Tasting the dish6


Where does Data come from?

Own Website

Partners Websites

RAW Data


RAW Data

One year historical reservation data

(.xlsx file)

Characteristics •260.000 reservations •80 fields

•57 categorical •9 numeric •10 date •3 text •1 incorrect field

•Size: 150 MB


RAW Data







Tasting the dish6


“Dirty” RAW Data

Gathering Data

21

The Process

New Fields

1 3 4

Transformation and Feature Engineering

“Clean” Data

Calculated Fields

2Cleaning Model


Data Cleaning


Data Cleaning

Row Deletion

• Reservations without check-in

• Cancelled reservations • Rows with errors

Column Deletion

• IDs vs names • Columns with little data

Other Actions

• Give dates a format • Delete accents • Transform .xlsx -> .csv


Clean Dataset

Clean

•150.000 reservations •46 fields •26 categorical •9 numeric •10 data •1 text

•Size: 75MB

Dirty

•260.000 reservations •80 fields

•57 categorical •9 numeric •10 data •3 text •1 incorrect field

•Size: 150 MB



Gathering Data

25

The Process

New Fields

1 3 4

Transformations and Feature Engineering

“Clean” Data

Calculated Fields

2Cleaning Model


TransformationsCountry Grouping

•A lot of countries to predict (210)

•Some countries have very few instances

•Grouping objective: mín. 1% of total instances

• Does not affect business objective

•Total number of groups: 20

New Fields

• RESERV_ANTICIPATION (calculated): (reservation date - checkin date)

• COUNTRY_HOTEL (name of the country)

• HOTEL_STARS (1-5)


Clean Dataset

Clean •150.000 reservations •46 fields •Size: 75MB

Dirty •260.000 reservations •80 fields •Size: 150 MB

Transformed •150.000 registers •49 fields •Size: 80MB


What is Feature Engineering

Extract signal from noise


Feature Engineering Techniques

• Detecta fields (features) that are predictorss

(signal) and bypass those that are not (noise)

• Dependand fields (pax, days, pax*days) • Needless fields (reservation number) • Fields with very little data • Random fields (minute and second of reservation)

• Domain knowledge • Experience • Recursive cycle


Field Selection

Algorithm Adjustment

Prediction

Quality Evaluation

Recursive Feature Engineering


Clean Dataset

Clean •150.000 reservations •46 fields •Size: 75MB

Dirty •260.000 reservations •80 fields •Size: 150 MB

Transformed •150.000 registers •49 fields •Size: 80MB

Final Dataset •150.000 registers •10 fields •Size: 55MB







Tasting the dish6


The Process


New Fields

1 3 4Gathering Data

Transformation and Feature Engineering

“Clean” Data

Calculated

2Cleaning Modeling


ModelingTraining Learning


Modeling







Tasting the dish6


Quality Evaluation

80%

20% Evaluation

Training

TestDataset 100%

Modelo


Quality Evaluation

Accuracy Confusion Matrix


Quality Evaluation

54% 75%


Quality EvaluationPredicted vs Real Distribution


Cooking Predictions

80%

20%Tasting the Dish

Cooking

Transforming

Go to the market to buy ingredients

Cleaning


Cooking Predictions

80%

20%Evaluating Prediction Quality

Training the Model

Transforming and Feature Engineering

Gathering RAW data

Cleaning Data


Other TechniquesEnsembles Clusters

Weight Analysis Anomaly Detection


ENDemail: [email protected]

Twitter: @data_lytics

www.clevertask.com

l9. real world machine learning - cooking predictions

Data & Analytics