The Data Science of Retail

Aaron EricksonAgile Analytics Executiveaerickson@thoughtworks.com

The Perfect Retail Experience?

Apple Stores have more than 2x Sales/Square Foot than their nearest

competitor.

(source RetailSails: http://www.retailsails.com.php53-12.dfw1-1.websitetestlink.com/site-content/live/3/

rs200_rankings.pdf)

What Makes Me Likely to Buy?

Provide me products that improve my life…

…found in a place convenient to me

…in a store where the products are easy to find

…where they are provided by friendly people

…who are able to anticipate my needs

…at a time convenient for me

…for a price I am willing to pay*.

* Note, this isn’t necessarily the lowest price

What Makes Me Likely to Buy?

Provide me products that improve my life…

…found in a place convenient to me

…in a store where the products are easy to find

…where they are provided by friendly people

…who are able to anticipate my needs

…at a time convenient for me

…for a price I am willing to pay*.

Given our product set, which products are customers demonstrating the most interest in? Which ones are

they likely to be interested in next season?

Historical Product Sales

Customer Demographi

cs

Provide me products I want…

Customer Research

Social Media

Targeted Upsell in Store

Analytics Informed Merchandising

Targeted Offers Online

Targeted Social Media Advertising

“…..he was able to identify about 25 products that, when

analyzed together, allowed him to assign each shopper a

“pregnancy prediction” score. More important, he could also

estimate her due date to within a small window, so Target could send coupons

timed to very specific stages of her pregnancy.”

What Makes Me Likely to Buy?

Provide me products that improve my life…

…found in a place convenient to me

…in a store where the products are easy to find

…where they are provided by friendly people

…who are able to anticipate my needs

…at a time convenient for me

…for a price I am willing to pay*.

Given our customer buying patterns, demographics, and migration patterns, what are the best locations for our retail locations? Should we offer different types of retail locations oriented at different types of buyers?

Purchasing Patterns

Migratory Patterns

… at a place convenient to me …

Offline/online

purchasing trends

Social Media

Store Differentiation (i.e. Walgreens)

Retail Location Optimization

Optimization of Product Mix per Retail Location

Targeted Physical Print Advertising

Mobile Advertising

What Makes Me Likely to Buy?

Provide me products that improve my life…

…found in a place convenient to me

…in a store where the products are easy to find

…where they are provided by friendly people

…who are able to anticipate my needs

…at a time convenient for me

…for a price I am willing to pay*.

How can I arrange layout of products that customers want? How can I do so in a way that maximizes the

likelihood that customers will purchase higher margin products?

Video capture of in-store shopping behavior

… where products easy to find…

Offline/online purchasing

trends

Further insight into customer

preferences around product

Heat map of which square meters have highest rev/margin

Insight into how to position products in

specific stores

Insight into what to offer people online after an offline visit

Audio analysis of what people

say about products in

store

What Makes Me Likely to Buy?

Provide me products that improve my life…

…found in a place convenient to me

…in a store where the products are easy to find

…where they are provided by friendly people

…who are able to anticipate my needs

…at a time convenient for me

…for a price I am willing to pay*.

Are customers having negative experiences in stores? Can we analyze comments in reviews of selected

locations to know whether our customers are getting the service they expect?

What Makes Me Likely to Buy?

Provide me products that improve my life…

…found in a place convenient to me

…in a store where the products are easy to find

…where they are provided by friendly people

…who are able to anticipate my needs

…at a time convenient for me

…for a price I am willing to pay*.

Can we better predict what customers want in the store based on what they have browsed for online?

How about offering them things online that people like them have looked at or purchased in the store?

Video capture of facial

expressions/emotion of staff

… friendly people who anticipate my needs…

Social media analysis of good/bad

experiences

Insight into what communication modes sell what

products

Greater understanding

salespeople’s non-verbal

communication skills

What Makes Me Likely to Buy?

Provide me products that improve my life…

…found in a place convenient to me

…in a store where the products are easy to find

…where they are provided by friendly people

…who are able to anticipate my needs

…at a time convenient for me

…for a price I am willing to pay*.

Can we adjust hours and sales associate schedules based on predicted traffic flow? Based on level of activity our in-store cameras manage to pick up?

Sales by hour trends over

time

… at a time convenient to me …

Online purchases (planned) v

offline (impulse)

Insight into what people tend to plan as purchases versus

impulse purchase

Further insight into what business hours for which locations

What Makes Me Likely to Buy?

Provide me products that improve my life…

…found in a place convenient to me

…in a store where the products are easy to find

…where they are provided by friendly people

…who are able to anticipate my needs

…at a time convenient for me

…for a price I am willing to pay*.

Can we quickly adjust pricing based on convenience, scarcity/abundance, or demographics in order to

optimize margin? Can we predict what a given type of customer will pay in a more sophisticated way?

Real time inventory levels

per store

… for a price I am willing to pay.

Buyer’s ability to pay

Supply chain adjustments

Further per store pricing optimization

What is Agile Analytics?

What is Agile Analytics?

Agile Analytics is the application of data science…

…to pressing business questions

…which are predictive in nature

…where solutions are usually not obvious

…involving data that is often diverse, messy, and high volume

…where feedback lends itself to continuous improvement

…for which answers have significant business impact.

What is Agile Analytics Not?

Data Warehouses

Consolidate data, get “one true version” of the truth.

Business Intelligence

Drive reports from data. Allow users to explore data and drive their own reports and needs. Good at describing the past, but inadequate for predicting the future.

Analytics

Using advanced maths, statistics, machine learning, monte-carlo simulation, and other advanced techniques to drive insight from data.

What is a Data Scientist

Like many popular buzzwords, “data scientist” is already becoming diluted. When ThoughtWorks uses the label Data Scientist, we are describing someone with at least three of these qualities:

The depth and expertise in mathematics to apply the appropriate statistical techniques to solve a problem

A strong blend of mathematical and development skills to enable them to implement analytical models

Expertise in machine learning techniques and technologies

Expertise in a the use of analytical techniques in a specific domain

To ensure that our people meet these qualifications, we’ve hired individuals with advanced degrees, specifically PhD’s in Physics or Mathematics with research experience in applying statistical methods

What Makes Agile Analytics Different

Traditional Analytics

Often depends on data being in a perfect state. Delayed for years while waiting for long running Enterprise Data Warehouse projects to finish.

Focus on building a perfect predictive model before trying it out. Not designed for iterative learning.

Often focused on the software tool, not the data science that goes into a solution. Software involved are often packages that cost into the millions of USD.

Much higher up-front costs – not just for software licenses, but for implementation.

Much higher risk due to the costs – and more importantly – time spent on the solution before you see results.

Agile Analytics

Data as it is, not how we wish it to be. Understand that there will never be a perfect data warehouse. Data growth is fast outstripping the ability of a data warehouse group to make it perfect.

Focus on time to market. Get a model out there, get feedback, improve it, repeat. Perfect is the enemy of the good!

Think like a startup. Use Open Source Software. FlightCaster’s founders did not seek big enterprise software vendors – yet they are far superior to large airlines at predicting flight delays.

Minimize the “cost-to-experiment”. Ramp up investment based on results, not speculation or hubris.

Putting the Science in Data Science

The Scientifi

c Method

Define Question

Gather Informati

on

Form Hypothesi

s

Test Hypothesi

s

Analyze Results

Draw Conclusio

ns

Publish Results

Retest

Define Question

Gather Informati

on

Form Hypothesi

s

Test Hypothesi

s

Analyze Results

Draw Conclusio

ns

Publish Results

RetestThe Scientifi

c Method:5/8ths of the steps in the

scientific method are

about testing our hypothesis

and doing something with

it.

Idea

BuildTest

Analyze

Define Question

Gather Informati

on

Form Hypothesi

s

Test Hypothesi

s

Analyze Results

Draw Conclusio

ns

Publish Results

RetestAgile Analytic

s:Application of the scientific method, lean

principles, and agile practices

to analytics.

Lean Startup

“The creation of rapid prototypes designed to test market assumptions, and uses customer feedback to evolve them much faster than via more traditional product development practices.”… applies to agile analytics efforts as

much as it does to startups in general.

Getting Started

Start Small – establish a few smaller areas of focus, seek to get some results and momentum as fast as possible. Take a humble approach to this as your organization learns how to apply these techniques. Once you understand how this works for you, then scale up.

Embrace Failure – seek to validation – or invalidate - your first hypothesis as soon as you can. Build out a “minimum viable model”. Don’t be afraid to try something small and fail. Focus on building a capability to measure what works, so you can more effectively iterate over the model and make it great.

People over Tools – agile analytics is much more about intellectual capital than tools, processes, or even data. A small team of data scientists can be much more effective than millions of dollars in hardware and software.

Diversity over Size – data is important, but the hype around the bigness of data obscures the importance of taking advantage of the diversity of data. Remember you will often get insights from smaller sources of data that happen to have the inputs that help drive a great predictive model.