monetising data science - bjss ltd. · pdf filemonetising data science combining commercial...

A BJSS WHITE PAPER

Monetising Data Science

COMBINING COMMERCIAL DATA WITH MACHINE LEARNING

Monetising Data Science | A BJSS WHITE PAPER

Contents

Introduction 1

Clustering 2

Regression 3

Classification 4

Natural Language Processing 5

Optimisation 7

Case Study - Customer Churn 8

Case Study - Airbnb Pricing 10

Case Study - Product Recommendation 12

Case Study - Twitter Sentiment Analysis 13

Summary 14

About the Author 14

Data Science is arguably the 21st century’s buzz-word for businesses wishing to leverage data-driven decisions, rather than relying on gut instinct and ‘decades of industry experience’. Entire companies, such as Google, were born out of utilising their data better than anyone else. The diversity of Data Science has been an ever-present in the news, with DeepMind’s victory at strategy game Go, IBM Watson’s success at Jeopardy, and Algorithmic Hedge Funds. Now is the right time to be considering how Data Science might (or already has) disrupt your industry.

This White Paper presents an overview of some of the more common problem types that can be addressed with Data Science, some of the business problems that these relate to, and provides some case studies that demonstrate value.

1

Introduction


2

Clustering

We will cover some of the most common classes of problems used in Data Science today, and typical approaches when dealing with these problems, with example applications of each one.

Clustering is how do we group things together. Examples of Clustering include segmenting customers into groups and profiles with distinct characteristics, trying to spot hot-zones for outbreaks of viruses or infections, or group similar types of hotels in an area together. Clustering algorithms help impose order on datasets that may otherwise be hard to gain insight into, and enable the data decide groupings for itself.

However, they suffer from technical problems, such as choosing the optimal number of groups, how best to re-normalise the data used, and whether a solution is the best possible (global) optima, rather than just a local optimal solution (heavily dependent on the starting conditions).

Fig 1 is an example of Clustering.

An example of a two iteration clustering algorithm is presented in Fig 2. We have randomly chosen three points in the first step as our cluster ‘centroid’, and then worked out, for each point, which centroid is closest, before doing an update to the centroid, and re-evaluating all the points. Typically this would be run this for tens or hundreds of iterations to allow it to find a stable answer.

Clustering algorithms enable the use of data to group products together in terms of trading patterns (I.e. highly seasonal vs unseasonal products, products that trade mainly on weekends or weekdays), customers together based on behaviour/spend, or look for anomalous or unusual behaviour in trade fraud, or even identify ‘dead zones’ where a retailer may consider opening a new store.

Online retailers use clustering algorithms to, for example, infer from a customer’s first purchase what behavioural groups they are most like, and what products they may be interested in buying next. Netflix uses very similar analyses to identify different types of customer behaviour, and recommend relevant movies or TV shows – indeed even making the decision on whether or not to produce something based on what customers, in segments it should appeal to, watch.

Original unclustered data6

5

4

3

2

1

0

-1

-26-3 -2 -1 0 1 2 3 4 5

Clustered data6

5

4

3

2

1

0

-1

-26-3 -2 -1 0 1 2 3 4 5

Fig 1

Fig 2


3

Regression is the act of predicting a number based on input data. For example, how much money a person will want to spend on a website based on their previous purchases, age, gender, address, or it may be predicting the amount of demand products may have in the future, or predicting how long a customer may stay before leaving.

In its simplest form, it is a tool many people will recognise – straight line fits - as illustrated in Fig 3.

For example, trying to predict someone’s performance at University, (Y) based on how they performed at High School (X). In more interesting, and business-relevant cases, we can examine how much additional revenue we make from someone who visits a website or shops more frequently vs someone who visits less frequently, and then try to incentivise the correct behaviour.

Of cours one must be careful in the use of a technique. Regression in particular can be very prone to overfitting, see for example Fig 4, where two functions have been fit to the same set of data, but one has large changes in y for relatively small changes in x. This means that the system predictions are not stable as a function of the input – imagine for someone aged 20 we predict they will live another 60 years, for 20.5 75 years, then 21 only an

additional 40 years – the role of the Data Scientist is to be able to spot issues like this and moderate algorithms to prevent it from happening.

Regression can be implemented to calculate things such as the volume of sales anticipated for an upcoming promotion (and optimising this – will Buy One Get One Free perform better than 50% off?, for example).

Regression

Fig 3

Fig 4


4

One of the most commonly used categories of problems in Data Science, Classification is labelling whether a set of data, A, belongs to a category Y or not. For example, typical questions might be ‘will this person I am lending money to default on the loan’ or ‘will this person be entering the hospital be infected by a superbug whilst staying here’, or ‘Is this transaction fraudulent or not’.

At the heart of the classification problem is the logistic regression, illustrated in Fig 5. This provides a response bounded between 1 and 0, which can be interpreted as a probability.

Probability of passing exam versus hours of studying

Hours Studying

Popular examples of classification problems include the Titanic which predicts, based on a sample of all the passengers, who survived the disaster or not – and, more corporately, financial services predicting if someone is likely to default on a loan or not. Hedge funds can and do use it to decide whether a particular equity should be bought or sold.

One famous study by NYU found that the application of Data Science and big data (taking a lot of features or attributes per patient) resulted in >90% accuracy prediction of whether someone would get a superbug when visiting the hospital or not.

It’s being used daily by companies such as Zest, who have opened the credit markets to young millennials in the United States with

no previous credit history – by crunching ‘big data’ features such as social media posts and activity – the result of this has been an over 40% improvement in default likelihood scoring. Aerospace parts manufacturers are using classification techniques to predict when engines might fail, to allow them to proactively and conveniently schedule the next maintenance cycle.

Customer churn and customer conversion are other big application areas of classif ication problems. Retailers can use Classif ication to look at how adding more or less tills at certain times of the day will encourage more visitors to become customers.

Classification

Fig 5


5

Natural Language Processing

Natural Language Processing (NLP) uses software to extract meaning from text. Applications range from simple sentiment analysis (‘is this tweet positive or negative’) to AI assistants in the form of IBM’s Watson.

Naive sentiment analysis attempts to break down the words used in a sentence to understand whether the sentence is negative, neutral, or positive.

This allows companies to quickly build a dashboard which can measure their performance in the customer service arena, and also f lag the ‘most negative’ interactions for timely interaction.

#negative #positive

long queues

busybroken

reductionsover priced

cheapoffers

open

bran

d-na

me

discontinued

closedin-store

handcraftedmade-to-order

well-stocked

all out

soldsale

Given its ability to tackle unstructured text, NLP has been a huge growth area for Data Science. Well known examples include Microsoft’s Tay chatbot, law f irm Baker and Hostetler’s bankruptcy assistant, and the famous victory for AI in the gameshow ‘Jeopardy’.

Fig 6

Fig 7

Positive Neutral Negative

My new coffee table is amazing

I have a new coffee table

My new coffee table

was overpriced


6

Combining customer interaction points with additional data such as linking an ID to a customer account allows customer service employees to automatically prioritise their response list in terms of most valued customers.

Systems like these are used regularly by many customer facing businesses, and to help with other elements (such as car manufacturers looking for customers who have problems with their car, and prioritising them in terms of seriousness of faults).

Other applications of NLP, include IBM’s Watson being used to run first line support for companies, dealing with a large number of comparatively trivial queries which do not rely on human comprehension, saving frontline staff costs and improving response times.

Fig 8


Processes like Reduce to Clear for retailers of fresh produce now rely on optimisation to understand when, and for how much, to sell a particular product just before it goes out of life-cycle. Savings have been tens of millions per annum for some of the largest retailers.

7

Optimisation is a big part of Data Science. Mathematical problems such as the ‘traveling salesman’ which requires the most eff icient route between locations, have been around for a long time, but the era of cheap and vast processing power allows us to brute force many of these problems. See for example Fig 9, the fastest route through all the US state capitals.

Whether it is an international courier company optimising the route used to deliver a parcel (delivering in excess of $60m fuel savings per annum), investment banks optimising trading strategies using historic data, or Google discovering that making adverts slightly bluer increased revenue by $200m per annum, optimisation and A/B testing is a large part of modern Data Science. This is illustrated in Fig 10.

Optimisation

50% of visitors see version A

50% of visitors see version B Version B

Version Aconversion26%

conversion12%

Fig 9

Fig 10


8

For example, a historic set of data is used to def ine the training sample, which allows us to understand the behaviours and relationships in data that are symbolic of someone wishing to leave. By taking this relationship and applying it to ‘live’ data, it is possible to categorise customers into different levels of risk, understand what particular at tributes might be enticing them to leave, and work out how best to retain them.

Using an open telecoms data set, it is possible to build models that predict the

likelyhood of a customer leaving their mobile provider or not. This data set include how many times a specif ic customer has called customer service, how much they spend on day, evening, night time and international calls, and whether or not they have a voicemail plan.

As with many Data Science challenges, the f irst step is to explore the data and understand the basic relationships. For example, examining the three dominant

variables, ‘Customer Service Calls’, ‘Total Day Charge’, ‘Total International Charge’, and understanding how churn performs as a function of these. This is presented in Fig 11.

It is apparent from three variables that the propensity for people to leave increases dramatically with the amount they pay – the highest value customers are also the ones most likely to leave. Unsurprisingly, people who have to call customer service a large number of times are also more likely to leave.

Case Study - Customer Churn

This training data can be used to build a Decision Tree, or a Random Forest, which will enable a machine to learn the pat terns in data, and provide the predictive ability to the business. A very simplif ied version of this is presented in Fig 12. The full model runs six layers deep and dozens of nodes! The accuracy of the full model, however, is in excess of 95%.

Customer service calls < 5YES

Total spend < £15Total spend > £50

Churner International calls over 3 per month Non Churner Churner

NO

YES NO YES NO

Churner Non Churner

YES NO

Fig 11

Fig 12


9

Table 1 shows that churn rate varies depending on the cluster we look at – our highest churn cluster has more than 60% of its members leaving, and our lowest churn one is at a more reasonable 3%. Based on new customers or existing customers changing their behaviours, we can be proactive in predicting when someone will want to leave, and engage them as necessary to prevent this. As demonstrated previously, if someone with a higher total charge calls customer service a couple of times, it becomes more and more likely that they will leave to move to a cheaper plan – the customer service agent can be proactive in predicting this and offer them a more discounted plan in order to retain their business. We can further expand this and create clusters of our customers to try to group together similar behaviour patterns demonstrated in Fig 13.

Cluster Total $ Intl $ CS Calls Churn rate [%]

Population

1 51.0 2.4 2.4 5.4 332

2 41.1 2.9 0.8 4.5 243

3 51.3 3.6 2.4 10.7 225

4 70.8 2.1 1.2 26.9 208

5 68.6 3.1 0.4 28.4 232

6 55.5 2.1 0.6 5 300

7 64.2 2.2 3.3 16.6 145

8 67.5 3.2 2.5 24.9 209

9 51.4 2.9 4.8 61.6 125

10 54.4 1.1 1.3 3.5 114

11 56.1 3.0 0.7 3.0 329

12 55.8 3.9 0.7 15.2 204

The axis range are normalised for technical reasons.

All Data People Who Churned

Fig 13

Table 1


10

Using publicly available Airbnb data (i.e. not clickstream, occupancy stats, or traffic stats), we can build a pricing optimisation algorithm for listings. In particular, we primarily focus on whether or not a listing is appropriately priced for its performance. Through understanding the relationships in data which derive the room pricing, we can then identify rooms which are under-priced relative to their performance, and rooms which are over-priced and taking unnecessary occupancy losses, in order to boost overall revenue.

Initially we explore the data to confirm some of our expectations, for example, in London, Westminster listings are more expensive than average, and Lewisham listings are cheaper than average. This is demonstrated in Fig 14.

We can also understand the relationship between the price and the average review score as seen in Fig 15. The better the reviews, the more can be charged (with a peak below 50% corresponding to listings which are overpriced and rarely utilised). This immediately confirms widely held perceptions that more expensive listings, when under performing in their review scores, should either decrease their pricing or increase the quality of the service they offer guests.

We can understand the importance of various features in extrapolating a price, in Fig 16 we can see that the type of listing is more important for price, followed by then review rating, and the number of people it can accommodate.

Case Study - Airbnb Pricing

Fig 15 Fig 16

Fig 14


With this information in mind, we can try to group similar listings together, to extrapolate high level attributes. This allows us to measure performance within individual groups, and to observe the different characteristics in listings. This is presented in Table 2. For example, Entire Home Cluster 3 is at a similar price point to Cluster 1, but has nearly a third of the reviews per month (a good indicator of occupancy). A price reduction in Cluster 3 will assist in boosting the occupancy, and by adjusting the pricing, expectations will be lowered and should boost the average review score.

By building a regression algorithm, we can use the features previously identified to predict prices. Based on a set of testing data, we are able to predict prices within 5% of the published prices – typically within £3 a night. We can simulate the impact on price, for example, boosting the average review score, will have on a listing, to suggest the obvious areas for improvement.

For example, within Entire Home Cluster 3 in Fig 17 we find underperforming listings which are underperforming. If we moved all listings to their optimal performance, this would result in a £11.20 per night average improvement in price. If we extrapolated this across all of the Airbnb listings in London, we could boost Airbnb’s revenue by somewhere between £130k (very likely) to £880k (maximum uplift) in one quarter.

With the quality of data available to Airbnb internally, we would expect to be able to take this several steps further. For example, with clickstream data we can compare the popularity of a listing being clicked, to a listing being booked. If a listing is only seldom booked but often clicked, it might imply that the listing is at an attractive price point but the quality on offer doesn’t match, and similarly often booked and rarely clicked might imply that it is priced too cheaply and the quality exceeds what is being charged.

By helping to optimise listings and boost revenue for hosts, this will help improve host retention and satisfaction, as well as providing more options to guests that are likely to be booked.

Average Price Average Listing Count

Average Review Scores Rating

Average People Accommodated

Average Reviews Per Month

Entire Home / Flat

Cluster 0 £105.70 1,969 98.4 2.8 1.24

Cluster 1 £96.90 665 80.6 3.2 1.33

Cluster 2 £101.00 1,625 90.7 3.1 1.63

Cluster 3 £95.80 24 33.3 3.0 0.53

Cluster 4 £97.10 152 64.1 3.2 0.71

Room (s)

Cluster 0 £44.00 1,494 80.1 2.2 1.36

Cluster 1 £50.90 3,943 98.4 1.9 1.26

Cluster 2 £47.10 2,684 90.8 2.0 1.75

Cluster 3 £42.10 413 63.2 2.3 1.12

Cluster 4 £48.70 72 33.7 2.3 0.62

11

Fig 17

Table 2


One substantial area of Data Science in the last few years has been the ability to recommend products with better accuracy, thus boosting sales. The particular technique implemented here is Collaborative Filtering with Implicit Feedback, based on the idea that users who purchase or view similar items will have similar tastes. This is represented in Fig 18. This is a form of community learning that provides the benefit of an implicit customer segmentation. A key strength of this methodology is the ability to target niche segments of customers with specific tastes, resulting in higher uptake rates together with rich cross-sell and up-sell opportunities.

Read by both users

Similar Users

Read by him, recommend to her

Read by user

Recommend to user

Similar Articles & Price

Cross-Sell

Read by user

Recommend to user

Similar But More Expensive

Up-Sell

We tested the efficiency of this system by generating a testing data set and a training data set. The training data set represents products which people subsequently purchased, to allow us to check the accuracy of our recommendations.

Above in Table 3(a) we have included an example table representative of the data – i.e. customer 1 has purchased SKU 01, SKU 03, and SKU 04. For customer 3, who has just logged onto our website, we can match him most closely to customers 1 and 4. We can, for all our users, generate a recommendation matrix in Table 3(b).

Where blue represents past purchases, and green represents different confidences of the recommendation strength. For customer 3, unsurprisingly, SKU 01 is their highest recommendation. Through several iterations of training and optimisation, we were able to build a product recommendation system that was 88% accurate on our testing set – reducing the rate of irrelevant recommendations by more than a third. The level of personalisation in the system allows us to target unpopular items, which specific customers like, with a 42% accuracy – rather than a 4% accuracy using a non-intelligent system, representing a ten-fold increase in the utilisation of the real estate of a web page. Similarly for popular products, we could offer up a 93% accuracy, whereas the non-intelligent system could only serve at 82% accuracy, halving the number of irrelevant adverts.

The machine recommendations are being actively included to boost up search results. Work is being carried out in this area with a major retailer, to combine the power of advanced search engines like ElasticSearch & Solr with machine learned recommendation scores, increasing the richness of the user experience and transforming an ordinary ‘search’ into a ‘relevant search’.

User/SKU 01 02 03 04 05 06

01 1 2 7

02

03 1 3

04 5 4 6 3 2

05 3 3 1

06 3 2

User/SKU 01 02 03 04 05 06

01 1 0.9 2 7 0.7 0.3

02 0.6 0.9 1.2 0.9 0.6 0.3

03 1.5 1.1 1 3 0.5 0.3

04 5 4 6 3 2 0.8

05 0.8 3 1.2 0.9 3 1

06 1.1 3 2 1.5 1.1 0.9

12

Case Study - Product Recommendation

Fig 18

Table 3(a)

Table 3(b)


With the advent of social media, collecting very granular opinions is easier than ever before. We initiated a project to look at Tweets in the lead up to the United Kingdom European Membership referendum, using AWS to build a Cloud native scalable solution for the collection of Tweets.

The outline of the architecture can be seen in the diagram above in Fig 19. Tweets are streamed to an EMR instance based on some keywords, analysed for sentiment, and added to an SQL database. A Lambda function, scheduled to run at a specified time interval, pulls stored Tweets and counts how many are positive, neutral, or negative. These numbers are then added to an in-memory cache (memcached). To relieve stress on the database. Instead of processing multiple, long running, queries for each person who connects to our front-end site, using the cache lets us query the database once, and then have that data available for any number of people, in a fraction of the time taken to perform the original SQL query.

We began tracking Tweets on the referendum 15 days before polling day but, considering the large amount of data and the limited space, we have chosen to focus our graph in Fig 20 on the time period around the 23rd. Things got much closer during this time, with both sides winning and losing the upper hand throughout the day. However, this was not the case in the preceding weeks, during which time the Leave camp almost exclusively held the upper hand over Remain, in terms of share of Tweets. The average share of Tweets up until June 23rd was Leave: 63.5%; Remain: 36.5%. This was not surprising: we were expecting there to be a slight bias in favour of Leave on social media.

On polling day however, it was interesting to find that the average share of Tweets mimicked the final result. We found Leave: 51.8%; Remain: 48.2%. At its peak, we were analysing 159 Tweets per second. Over the whole period we collected 3,153,067 Tweets. All of this was done for a negligible cost.

Case Study - Twitter Sentiment Analysis

13

Fig 19

Fig 20


14

About the AuthorHenry BrownData Scientist

Like many Data Scientists, Henry Brown completed a PhD – in particle physics – before transitioning over to industry. During his decade-long undergraduate-postgraduate-postdoctoral experience at the University of Liverpool, Henry was fortunate to get five years’ exposure in CERN, the European Organisation for Nuclear Research. Most recently, Henry managed the Retail Demand Forecasting Development team located in the Supply Chain Transformation team at the UK retailer, Tesco, where he was personally responsible for saving the company around £12m per annum in waste savings.

• Data Science is recognised as one of the fastest growing areas of technology.

• Early adopters of Data Science have already implemented eight or nine figure change programs.

• A range of techniques covering prediction, optimisation and natural language processing is available and always evolving.

• Areas of particular growth include product recommendation, customer understanding, demand forecasting, risk assessment and customer lifetime value.

• The current era of affordable Cloud infrastructure and Platform as a Service, allows companies to collect increasing amounts of both structured and unstructured data at relatively low cost. This enables Data Scientists to embark on more ambitious and complicated data projects than ever before.

• Prototypes are achievable in a relatively short space of time, using existing commodity or Cloud infrastructure, and at low cost. Prototypes provide a clear and transparent path to production

• Petabyte-scale Data Science challenges are now manageable and will now become commonplace.

• ‘Deep learning’, a technique which allows machines to understand rules rather than just relationships is now on the cusp of becoming a standard industrial technique.

• The intelligence provided by Data Science can be transformational, boosting revenues streams, creating new sources of revenue, and identifying areas of inefficiency and waste.

• Many enterprise-grade open source tools are available to reduce cost of ownership.

• Data Science is a cross functional role. It requires both business and technical expertise in tandem with a robust delivery approach, such as BJSS Enterprise Agile®.

Summary

About BJSSBJSS is an award-winning delivery-focused IT Consultancy with over 20 years’ software development and IT advisory experience. The company is renowned for technical excellence, cost-effective delivery and its proven BJSS Enterprise Agile approach.

www.bjss.comCopyright © 2016 BJSS Limited. Enterprise Agile® and BJSS Enterprise Agile® are registered trade marks of BJSS Limited.

monetising data science - bjss ltd. · pdf filemonetising data science combining commercial...

Documents