moving forward with big data: the future of retail analytics

Moving Forward with Big Data The Future of Retail Analytics By Bill Bishop, Chief Architect, Brick Meets Click

Overview 1. Where’s the value?

Where big data creates value for retailers

The most valuable data sources

2. What’s the focus?

Demand and supply side perspectives

3. What’s happening today?

Who’s doing what

The status of big data projects

4. What’s ahead for big data?

Clarifying the definition

Overcoming barriers to progress

A process for moving forward

BRICK MEETS CLICK is a strategic resource for retailers, suppliers, and technology providers who want to make insightful decisions about meeting

shoppers needs. Visit us at brickmeetsclick.com.

A BRICK MEETS CLICK ORIGINAL PAPER April 2013

Readers of this paper are invited to comment using

this link. Or, go to http://www.brickmeetsclick.com/big-

data--surveying-the-future-of-retail-analytics

http://www.brickmeetsclick.com/

http://www.brickmeetsclick.com/big-data--surveying-the-future-of-retail-analytics

© 2013 Brick Meets Click 2

Moving Forward with Big Data: The Future of Retail Analytics

Overview

When we surveyed retailing professionals on big data in 2012, fewer than 20% indicated that retailers were actively working on big data and “better

understanding of customers” was the most widely perceived benefit. By March 2013, 64% report they are participating in a project that involves Big Data,

and the biggest focus is on how to use it to create competitive advantage.

Clearly the relationship between big data and retail is evolving rapidly, but it’s also clear that many retailers still need to answer questions about how much

to invest in big data and where to focus efforts initially to produce the best returns.

This survey explored:

• Where and how big data is adding value for retailers on both the demand and supply sides of the business.

• The type of big data projects that are underway, the stage of development or implementation they’ve reached, and whether a business case has

been established.

• The definition of big data and how clarifying this can accelerate progress in using it.

• Barriers that are limiting its application in retail.

So, how can retailers use these findings to more quickly and confidently decide if big data can help them create competitive advantage? The answer

depends on where your particular organization is with big data.

For retailers who haven’t yet made a decision in favor of big data, use these results to build awareness within your organization about how

competitors are already working to measure the benefits and validate the business case for the strategic use of big data.

For retailers who are committed but looking for a starting place, use these findings to provide guidance on which business problems hold the

greatest potential.

For retailers who are already moving forward with big data projects, find a “second opinion” on the direction you’ve taken by learning more

about what others in the business are doing.



ABOUT THE SURVEY AND COMMENTS

Brick Meets Click’s second big data survey was conducted during March 2013.

115 retailing professionals responded. 50% identified themselves as consultants or market researchers, 20% as technology/information suppliers, 13% as

retailers and wholesalers, 9% as product supplier/manufacturer, and 8% as other.

Participants had the option to answer questions related to the demand side of the business or the supply side of the business, or both. 89% chose to

answer demand side questions; 60% chose to answer supply side questions; and 44% answered both sets of questions.

Survey respondents had an opportunity to comment on the findings. In the text, these are set off by a blue background.

Readers of this paper are invited to comment using this link. Or, go to





1. Where’s the value?

The big data experts and leading retailers we’ve talked with in the past year frequently cite the following ways they expect big data to create value for

retailers:

• Analytical results will become more accurate because big data will make it possible to study the entire information set instead of sampling data. Dealing

with sampling errors will no longer be necessary.

• Large-scale controlled experiments will become both possible and practical. This is the only way to establish with certainty what really drives a

particular improvement.

• By creating a single source of data that the entire enterprise could access for analyses, big data could eliminate much of the friction, effort, and

coordination required to bring together data from different parts of the business.

• Big data analytics will support faster, fact-based decision-making by shortening the time that elapses between the identification of a problem or

opportunity and when management is able to take effective action.

• It will strengthen analysis by incorporating sources of data that were

previously impractical or impossible to integrate.

“Supporting faster decisions” ranked #1

While big data offers many ways for retailers to create value, the ability to

execute on fact-based decisions more quickly was ranked highest. The ability to

include new sources of data in analyses ranked second. Using big data to create

a single source of data for the enterprise ranked third. All other choices scored in

the single digits. (See Figure 1.)

Figure 1 * multiple responses allowed



The familiar is still favorite: Shopper transaction data

Five sources of data were identified as having high potential for improving retail performance (Figure 2).

Shopper identified transaction data, selected by 52%, still gets the biggest vote of confidence from retail professionals. These transactions can be tied

directly to a shopper or household via loyalty programs, credit or debit purchases, and

other methods.

Mobile device data that could be used for geo-tracking and geo-targeting is also

seen as having big potential.

In-store tracking has been around for some time, but its potential to create value for

retailers is increased by new data collection methods and more powerful analytics.

Shopper feedback can now be collected on a more continuous basis. Much of

today’s focus is on translating it into information that can be used quickly and

effectively by management.

The ranking of social media reflects the challenge many retailers still face in

connecting these digital breadcrumbs with existing marketing and merchandising

tactics.

Figure 2 * multiple responses allowed



BMC PERSPECTIVE: Where’s the value?

The recent advances in data storage and analysis will allow retailers to extract value from big data but, at the same time, this has created an almost

overwhelming number of never-before-available possibilities. It’s no wonder that there’s confusion about how to begin to capture this value.

At the start of this study, we had a long list of ways big data can create value for retailers; we were interested to find out how others viewed this landscape.

Here are the opportunities we see:

1. DOING THE SAME THING BETTER AND FASTER. While the results show no consensus, the emphasis on supporting faster decisions is clearly the

most popular option. This comes as no surprise since it’s easier and probably safer to try to do something better than to take an entirely new

approach.

2. EXPLORING NEW DATA SETS. Conversations with big data practitioners (outside of retailing) reveal that they find significant value in incorporating

new external sets of data that provide a richer context for the analysis. The survey shows that some retailing professionals see this value as well. We

hope this report will encourage exploration of which external data sets can be of most value to retailers.

3. A SINGLE DATA LIBRARY. The idea of creating a single source of data was endorsed by some in the survey, but challenged by others as not a

legitimate way for big data to create value. Conversations with those deeply involved in analysis, however, opened our eyes to some truly

transformational possibilities. Just think of the time and labor that would be saved if everything from the weekly sales reports to the revenue

component of quarterly financials could be extracted from the same database.

From survey participants

Collecting and managing big data in real-time acts as the catalyst to reach shoppers at the moment of truth, and not just tool for planning

campaigns. A mobile platform will generate relevant data which can be used to further engage shoppers in a precise and personal way and

drive predictable purchases." -- Aaron Roberts, CEO, QThru

As a believer in “CRM Big Data”—the shopper name, address and SKU linked, made into a database and then analyzed to find

applications across the retail enterprise, it is heartening to find that the “big data” concept continues to become embedded as a decision

tool and not just the idea du jour. My takeaway: “big results” from “big data”. -- Francey Smith, Francey Smith & Associates

It’s noteworthy shopper feedback appears in the top five. It isn’t enough to just know what a shopper purchased (which might have been a

substitute for the out-of-stock item they really wanted), how they moved through the store or if they purchased a geo-targeted ad item.

Shopper feedback connects all of these elements with the “experience” behind the purchase, which may be most revealing!

-- Brian Numainville, Principal, The Retail Feedback Group



2. What’s the focus?

Demand and supply side thinking

People working on the demand side of retail tend to think about big data opportunities differently than those who work on the supply side. For demand-

siders, winning market share is the major goal. Supply-siders are responsible for retail operations, distribution, and supply chain relationships and often

focus on managing costs, productivity, and optimization.

For retailers, the decision about which “side” of the business to start with will reflect the retailer’s strategy for growing the business. If the focus is on:

Demand side, these survey findings are likely to encourage that retailer to concentrate on marketing more effectively to individual shoppers.

Supply-side, then the results will direct them to strengthen their management of inventory and look for possible workforce applications.

To capture both types of thinking, the survey offered supply side and demand side “tracks.” Participants had the option to answer either (or both) sets of questions.

Eighty-nine percent of the survey participants chose to answer demand-side questions, 60% chose to answer supply side questions, and 44% answered both.

On the demand side . . .

One of the first questions retailers need to answer as they move deeper into big data on the

demand side of the business is what they want to achieve – what problem do they want to

solve? Once that’s been decided, the job of identifying data and analytics becomes much

easier.

. . . marketing to individuals more effectively is the focus.

The idea of one-to-one marketing has been around for some time, but it’s difficult to implement.

With 69% currently focusing on marketing to individuals, it would appear that retailers are

seeing big data as the light at the end of the tunnel that will enable them to achieve

customization on a mass scale. (See Figure 3.)

Figure 3



Turning to the specific opportunities to drive demand, we asked: Where will big data create the most value in the next two years?

There are some strong contenders. (See Figure 4.)

Given that strengthening shopper engagement is the highest rated opportunity at

76%, expect retailers to interact even more actively with shoppers along the entire

“path to purchase.” This will change marketing and merchandising practices.

High interest in personalized promotions (70%) reflects continuing focus on moving

aggressively toward one-to-one marketing tactics.

The focus on shopper solutions (43%) opens additional opportunities for

collaboration with suppliers who can bring shopper insights to power development of

highly relevant solutions.

The power of implementing store specific assortments (38%) comes from better

ensuring that shoppers will find the products they’re looking for and making sure that

they’re in stock.

Figure 4 *multiple responses allowed



BMC PERSPECTIVE: On the demand side

With continuing slow economic growth, retailers’ top priority is understandably on driving demand.

SHORT TERM. The clear direction from the survey is towards strengthening relationships with existing customers. There’s strong evidence that this is

probably the best way to drive incremental sales, but this is a short-term justification.

LONG TERM. The more strategic reason for this focus is to put in place a clearly defined and manageable base for driving sustainable growth. Enhanced

customer relationships are the basis for customer retention and managing the lifecycle value of each shopper. This is where retailers will capture long-

lasting value from personalized promotions and stronger shopper engagement.


Putting the emphasis on business value (what problem(s) does the retailer need to solve?) is the best

and most practical way to go about conquering big data. It ensures value, investment, and outcomes

are aligned. If executed well, the results should be easy to quantify. Taking a “build it and they will

come” approach to big data is too risky from a time, opportunity cost, and financial standpoint.

-- Jim Butera, FusionPoint

The ability to quickly analyze transaction data down to the SKU level will open a gateway to

understanding how consumers are shopping each product in the store and how those specific items

are inter-related. Retailers that adapt this level of basket analytics will certainly have a competitive

advantange in the marketplace by partnering with vendors/CPG to develop and create very unique

customer-centric promotions that will resonate with their targeted shopper segments.

-- Victor Andedo, Principal, Linque Marketing Inc.



On the supply side . . .

. . . once again, POS is MVP.

By far, POS transaction data was identified as the most valuable source of big data for

the supply side of retailing; 51% of respondents selected it. Three other sources were

identified as potentially most valuable – sensors in the retail environment, shopper

feedback, and automated product recognition – though none even came close to

rivaling the confidence shown POS (Figure 5).

POS has been a key source of data since the inception of checkout scanning and it’s

clear that the growth of big data will not change this any time soon. POS data drives

replenishment, maps labor requirements, and provides a unique operational signature

for each store.

The growing importance of sensors has been evident at recent trade shows where

they were used for cold chain and energy management, as well as, workforce

management.

Shopper feedback is used mainly to identify opportunities for improved employee engagement with customers.

Automated product recognition, the technology at the core of tools for monitoring plan-o-gram compliance on the shelf, was identified by 8% even

though it was unfamiliar to many.

Figure 5



Optimizing inventory is a big opportunity

As retailers move to apply big data analytics to the supply side of the business,

the question is, where will they find the greatest opportunities?

Inventory is by far where respondents see big data has the most positive impact

today. Two other opportunity areas identified – customer service management

and labor management – are closely related. (See Figure 6.)

Attention on inventory management focused mainly on optimization rather than

reduction. Customer service management is becoming a more important

competitive dimension. Labor management -- both scheduling and sales

conversion are both related to shopper engagement.

Workforce management will change

Labor represents an important element in most brick and mortar retail business

models, so the response to this question showed a remarkable degree of

confidence (Figure 7).

Figure 6

Figure 7



BMC PERSPECTIVE: On the supply side.

Expect strong focus on supply side applications because these have the potential to both reduce costs and drive incremental sales. The good news is that

these results show a range of ways to do that.

SHORT TERM. While inventory management was identified as the most important in these results, it’s hard to imagine that there are not equally

impressive ROI opportunities related to labor and customer service management. Solid decisions will leverage accurate measurements in both areas.

LONG TERM. The strategic opportunity is to use big data to reconfigure the supply side of the business to deliver products and services cost-effectively in

a multi-channel shopping environment. Already, retailers show signs of recognizing this need (online retailers are opening stores and brick and mortar

retailers are leveraging digital) but the details of how to optimize inventory in this complex “mixed” environment are still being worked out. Retailers who

solve this problem will have a real competitive advantage.



3. What’s happening today?

Who’s doing what?

Almost two-thirds of those who answered the survey were actively involved in a project

using big data (Figure 8). The projects addressed a broad range of problems, but some

patterns did appear.

About half targeted a better understanding of shopper behavior and buying habits. These

fell mainly in two areas:

Developing more relevant and effective promotional offers.

Creating more sharply focused behavior-based shopper segmentation.

About 20% related to improving product availability: assortment optimization and “getting

the right product to the right place,” and reducing out-of-stock.

Four unique business problems being addressed stood out from the rest:

Leveraging insight from social media to drive product display with specific markets in a way that intersects with natural events (like flu outbreaks).

Maximizing customer retention while minimizing incentive spending.

Monitoring risk in the environment.

Analyzing the shopping behavior of millions of insurance customers.

Figure 8



The status of big data projects

Marketing is driving the bus.

Marketing sponsored almost half of the projects reported in the survey (Figure 8).

This was followed by merchandising, operations, and IT. In the “other” category,

projects were sponsored by:

Research and development

Executive management

Supply chain

Sales/category management

The horses are spreading out on the track.

The status of big data projects spanned the project lifecycle, suggesting that some

companies are pulling ahead of the pack.

A little less than a quarter of the projects are in startup, while nearly half, 49%, are

either in pilot or in production (Figure 10). This indicates that while the practice of

big data is still young, some retailers are already beginning to reap the benefits.

The fact that 58% of those responding report that they have already been able to

measure the benefits indicates that confidence is building in the value of this

approach.

Figure 9

Figure 10



What’s Happening Today?

Having business case for big data projects is no longer the exception, 65% of the projects

had established one (Figure 11). This shows that nearly two-thirds of those working with

big data now see how it will solve important business problems. These people are no

longer groping to figure out how to use a new tool.

BMC PERSPECTIVE: The status of big data projects. The use of big data in retailing has gained significant traction in just a matter of months. While the practices are still in an early stage of development, it’s

clear that many more retailers are making a real commitment to figure this out. This is encouraging, because in the larger scheme of things, this evolution

is not only sensible, but inevitable.

STILL EARLY DAYS. Even this progress doesn’t obscure the fact that big data is still in the early stages of the adoption curve for business innovation. A

lot more evidence will need to be generated about its value before a significant percentage of retailers adopt the practice – but we see this happening fast,

probably a lot faster than most can imagine now.

SOME RETAILERS ARE PULLING AHEAD. The key insight? Now is the time for retailers to begin figuring out how big data can benefit them over the

long term and to take the first steps in that direction, so they don’t fall too far behind.

Figure 11



4. What’s ahead for Big Data?

Clarifying the definition

When McKinsey published their big data report1 in 2011, the definition they offered focused mainly on scale: “Data sets whose size is beyond the ability of

typical database tools to capture, store, manage, and analyze.” The emphasis on size has proved a distraction at times as retailing professionals grappled

with how to make practical use of big data. In BMC’s 2012 survey, participants quickly went beyond “what it is” to focus on “what it does” and how it could

be used to improve sales performance.

Don’t get distracted by the BIG in Big Data

In this 2013 survey, we wanted to explore whether extending and clarifying the

definition could add value and make the term more useful; 94% of agreed that doing

so would help to speed up the adoption and effective use of big data. (See Figure 12.)

Almost 9 out of 10 indicated that a stronger working definition would make it easier for

senior management to see the connection between big data analysis and changes

that would drive improved performance. The focus on the large-scale nature of big

data doesn’t do much to help bridge that gap.

1 Big Data: The Frontier for Innovation, Competition, and Productivity – McKinsey Global Institute 2011

Figure 12 *Multiple responses so total exceeds 100%



Nearly half said a better definition would make it easier to gain agreement on the scope of projects and help to build justification for projects. Better

agreement on scope would help in minimizing costs and coordinating project support across business functions.

In “other” comments, participants said a stronger definition would:

Help focus on the real value – speeding up innovation, fine-tuning customer focused offers, providing competitive context for analysis, and

emphasizing important business questions

Clarify ambiguity. The term big data feels too ambitious for many people.

Said one participant: We need to replace it with more focused definitions . . . since often the data we are talking about is closer to “mid-sized” or even

“small sized.”

Favorite addition: New data combinations

We asked those surveyed to start with a core definition – an approach to making the

handling of data at large scale “effective and affordable” – and then indicate how helpful it

would be to add each of three features to the definition.

Nearly 70% chose “Analyzing data not previously combined.” Almost 40% wanted to include

opportunities to extract value from unstructured data, and about a quarter wanted to include

the opportunity to use data not previously collected or previously considered of low value.

(See Figure 13.)

These additions make it easier to talk about what big data does. When a retailer combines

short-term temperature forecasts with their beverage replenishment system, they are

analyzing data not previously combined in ways that improve their results. The grocery retailer who is analyzing comments on social media to tailor menu

and recipe recommendations is extracting value from unstructured data. And the major retailer who is using sensors to track shoppers in order to make

short-term adjustments in labor at the front end is making use of data not previously collected to reduce customer wait time and increase labor productivity.

Figure 13



BMC PERSPECTIVE: Clarifying the definition. With appreciation to the McKinsey Global Institute and those who developed the initial working definitions of big data, it’s now clear that more meaningful

progress will require a next-generation definition – or set of definitions. More descriptive definitions will be easier to use in business settings, and benefit all

stakeholders by encouraging more rapid adoption of big data practices.

WHAT’S NEXT? The findings in this survey make it clear that it is time for the retail community to work together to hammer out this new generation of

definitions. The only caution would be to not leave this work to those whose self-interest would influence the results.


I believe we are all just talking about input, because that's what data (big, medium, or small) is. It may be

more productive from a leadership perspective to talk of Testing (big, medium, or small - although I like

the idea of Big Testing), and focus on continually learning through tests that can now be fully analyzed and

acted upon because of the masses of data that we can now gain access to and the speed at which it can be

crunched.

-- Craig Elston, SVP Insight and Strategy, The Integer Group



Overcoming barriers to progress

Survey participants were concerned that organizations weren’t making full use of

already available data (Figure 14.). This comes either from not having enough

staff to analyze the data or to make full use of the insights from existing data.

They were also concerned that organizations don’t have the staff or infrastructure

needed to handle all the insights generated by big data and translate them to

action.

The low concern for privacy and security was perhaps the only surprise among the

other barriers identified (Figure 15). While shoppers are showing less concern

about privacy, the incidence of security breaches indicates that this is an area of

risk warranting careful management.





BMC PERSPECTIVE: Overcoming barriers to progress.

The main barriers to successfully using big data are not technical; they are related to organizational alignment and capability. This may sound like jargon,

but it’s the truth and those participating in the survey know it.

This raises a key question: What it will take for senior retail leaders to make big data and the organizational readiness to exploit it a key strategic priority?

The increase in big data projects is encouraging, but by itself it doesn’t reveal much about the vision for and the magnitude of commitment to taking full

advantage of this important new resource. Effective use of big data by itself won’t guarantee success in 21st century retailing, but it will be a key part of

that success formula.

From survey respondents

The issue companies face is not the technology to amass the data but the analytical skill to understand

the insights from the data to take the proper marketing action. We must invest in people with those

skill sets to be competitive. As an industry we have never really invested in staff to do the analytical

work necessary to build the right targeted programs. Kroger clearly figured it out when they began

working with Dunnhumby. Technology has moved quickly and at a lower cost than before. We need to

harness the information to make businesses successful.

-- Ann Raider, President and CEO, inStream

The report shows a glimpse of the future. We will forget today’s clunky user-intensive big data and

mobility platform interaction model one day. Background-processed real-time prescriptive analytics

will ultimately “get big data out of the way.” Retailers and consumers alike with engage truly

seamless Brick Meets Click shopping experiences. Supply chains, markets, and buyers will integrate

without regard to location. I can’t wait for this new future.

-- Andrew John Stein, Pervasive Strategy Group



BMC PERSPECTIVE: A process for moving forward.

As we reviewed the findings from our big data survey at Brick Meets Click, we also thought about the many conversations we’ve had with colleagues

during the past year on this topic. It’s clearly going to take large-scale solutions to address some of the big issues identified – like developing the

organizational alignment and skill sets needed to take full advantage of big data’s potential. But, the fact that some big issues remain to be resolved hasn’t

stopped people from moving forward to apply big data practices to solve retail problems and create value.

5 SIMPLE STEPS. Here’s an approachable, grounded process that any retailer can use to move forward with big data. It unlocks the power of big data by

focusing on specific business challenges.

1. Identify a key business challenge worth overcoming through the analysis and use of big data.

2. Find a tool – an application, analysis, method, etc. – with the potential to address that business problem in a new and better way.

3. Put in place a baseline measurement you can use to evaluate progress in overcoming the challenge.

4. Focus first on incremental improvements to ensure you’re making real progress.

5. Expand to bring in new external sets of data that will improve results.


It seems clear that senior management needs to adjust as failure to could lead to disaster.

Just ask Blockbuster, Borders, and on the ropes retailers: Best Buy, JC Penney, Radio

Shack, Sears and Abercrombie & Fitch. All have in common a failure to adjust to

changing conditions. Perhaps this is the real promise of Big Data. Used correctly it

offers the ability to not only adjust but to lead the way to realizing the holy grail of

creating a truly multi-channel shopping environment.

-- Tom Van Aman, Marketing Strategy, Allstate