1

Developing a ROMI AnalysisIntroduction and Discussion

2

Agenda

Overall Approach

GMAX™ Modeling– Benefits

Case examples

Discussion– Data availability

33

Methodology Overview

Build database

Normalize data from various sources

Build parallel models to look at different variables and variable combinations

Refine models (focus on DMAs with enough advertising data to make confident conclusions)

Generate and test hypotheses with models

Find themes that emerge from the models

Translate mathematical results into actionable business recommendations

Drill down to gain better

understanding of relationships

4

Why is the OG ROMI approach different?

Proprietary tools and data platforms permit examination of more variables– Typically 3 orders of magnitude more

Identifies complex relationships and patterns in the data– Interactions– Curvilinear functions (multi-order polynomials)

Models the real-world environment

“X in combination with Y”

5

MethodologyThe approach/methodology used a combination of analytical techniques

GMAX – Genetic Approach

- 10s of thousands of model combinations

- Determines important variables

Regression

- Seeks to understand and calibrate individual variable influence

6

Classic Regression

$10 $9 $5

$4$7

$8$3

$2$1

$60

5

10

15

20

25

30

35

40

45

50

0 1 2 3 4 5 6

RR

R

N

R

N

N

N

R

7

Genetic Programming

$10 $9 $5

$4$7

$8$3

$2$1

$60

5

10

15

20

25

30

35

40

45

50

0 1 2 3 4 5 6

X2

X1

RR

R

N

R

N

N

N

N

R

GMAX™ Benefits

Cost-effective examination of more variables– Including non-linear relationships– …and interactions between variables

Helps avoid “errors of omission” (filtering potentially useful data based on heuristics or prior experience of the modeler)

8

9

Case example

HP: Small Business Target

10

Project Objective

Develop model and understanding of relationships between marketing expenditures and sales

Direct MailCatalogPrint AdsEmailsOnline AdvertisingAdvertisingPricingCustomer AwarenessCustomer ExperienceSalesMarket Share

Total Sales $

ClientControlled

AttitudinalOutcomes

SalesOutcomes

11

Print Costs While print costs appear in the GMAX model, the

relationship is not clearly seen in graphical analysis of print costs by themselves

Print Out of Pocket

400000300000200000100000

AL

L E

nte

rpri

se

400000000

300000000

200000000

100000000

12

Marketing Communications Variable Tree

Share of voice, print, online, and direct mail all have an affect on sales Sales

Shipments

Prod BShare of voice

Prod AShare of voice

PrintOut of pocket

DirectMail

PrintOut of pocket Online costs

Note how Print has an impact by itself AND in combination with Direct Mail

13CATQTY

300000020000001000000

AL

L S

MB

400000000

300000000

200000000

100000000

Catalog Circulation

Higher volumes of catalogs correspond with higher sales volumes

– Diminishing returns at approximately 160,000 pieces– Suggests that higher cost catalogs (i.e. CPM) produce results

14

Other variables examined: Supply Chain

An example from one set of modeling …

Variable RelativeFrequency

AvFqcy Variable

1.83000 PRTTOOP Print out of pocket costs

1.07000 WTPcompAP Wtd average pricing vs Comp A

0.75000 Category SOVShare of voice

0.53500 BRANDSOV Share of voice

0.31000 CATTOOP Catalog out of pocket costs

0.15500 PRTCIRC Print circulation

0.08500 SOVSOM Share of voice / Share of market ratio

0.06500 WTD compaPR Wtd average proposed pricing vs. CompA

0.06500 SERSPEND Total spending on XXXX

0.04500 BACKDOL $ value of backlog

0.04500 CATQTY Catalog circulation (quantity)

0.03000 SPCompBPR Proposed price vs CompB

0.02500 DIRMAIL Direct mail circulation

0.01000 SPRICE Price vs CompS

0.00500 DMCOST Direct mail total out of pocket costs

0.00500 BACKQTY Backlog quantity (units)

15

This analysis yields a moderately complicated, but understandable and interpretable model

Total Sales $ =

240,000,000 + 1.984 *

((Print Out of Pocket $/Print Circulation) *

(Catalog Out of Pocket $) *

(Weighted Pricing vs. compA/Weighted Proposed Pricing vscompA))

+ $335 * (Direct Mail Cost)

ROMI Model

16

Sales per $

For most marketing expenditures it was possible to calculate an estimated of return (SMB sales) per dollar spent

SMB Total Sales Est Return per $

Total SPO Expense 6.6Online Total Costs 77SMB Catalog (per cat) 32Catalog $ 76

1% price change vs IBM 7,644,000 One article 1,339,000

17

Case example

U.S. Navy: Recruiting

1818

Lead Contract Lag

Although the lag between a lead and a contract varies, almost 60% of contracts are signed within 4 months of generating a lead

Lead-Contract Lag

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Lag in Months

% o

f co

ntr

acts

Cume58.2%

1919

Positive Impact on Leads

Spending non-Production

80000006000000400000020000000-2000000

Le

ad

s T

ot

Act

En

l Na

t

70000

60000

50000

40000

30000

20000

10000

Total Spend

General

6000000

5000000

4000000

3000000

2000000

1000000

0

-1000000L

ea

ds

To

t A

ct E

nl N

at

70000

60000

50000

40000

30000

20000

10000

General Mkt Spend

Television

500000040000003000000200000010000000-1000000

Le

ad

s T

ot

Act

En

l Na

t

70000

60000

50000

40000

30000

20000

10000

TV Spend

Radio

1600000

1400000

1200000

1000000

800000

600000

400000

200000

0

-200000

Le

ad

s T

ot

Act

En

l N

at

70000

60000

50000

40000

30000

20000

10000

Radio Spend

Cat Internet

1100000

1000000

900000

800000

700000

600000

500000

400000

300000

Le

ad

s T

ot

Act

En

l Na

t

70000

60000

50000

40000

30000

20000

10000

Internet Spend

Int CPL

4000003000002000001000000

Co

nt

Act

En

l Na

t

1600

1400

1200

1000

800

600

Internet CPL

Each of these variables contribute to lead generation at > 85% CL

2020

Influence on Leads per $ Spend

Spending Category Leads per $100,000 Sig. LevelInternet CPL 3329 83%Internet 1817 86%Direct Marketing (Total) 1239 91%Radio 1072 98%Television 277 97%Total Media Spend 209Out of Home -1880 90%Emerging NSMedia Event NSPrint NSInternet Search NS

Of the variables that have a positive correlation with leads, Internet (specifically CPL), Direct Marketing, Radio and TV have the highest rate of return per an additional $100K of spending

2121

Positive Impact on Contracts

Spending non-Production

80000006000000400000020000000-2000000

Co

nt

Act

En

l Na

t

1600

1400

1200

1000

800

600

400

200

Total Spend

Cat Television

500000040000003000000200000010000000-1000000

Co

nt

Act

En

l N

at

1600

1400

1200

1000

800

600

400

200

TV Spend

Cat Media Events

4000003000002000001000000-100000

Co

nt

Act

En

l Na

t

1600

1400

1200

1000

800

600

400

200

Cat Media Events

4000003000002000001000000-100000

Co

nt

Act

En

l Na

t

1600

1400

1200

1000

800

600

400

200

Media Events

DM Total

120000010000008000006000004000002000000

Co

nt

Act

En

l Na

t

1600

1400

1200

1000

800

600

DM Total

120000010000008000006000004000002000000

Co

nt

Act

En

l Na

t

1600

1400

1200

1000

800

600

Direct Marketing

Int CPL

4000003000002000001000000-100000

Co

nt

Act

En

l Na

t

1600

1400

1200

1000

800

600

400

200

Int CPL

4000003000002000001000000-100000

Co

nt

Act

En

l Na

t

1600

1400

1200

1000

800

600

400

200

Internet CPL

Each of these variables contribute to contracts at > 85% CL

2222

Contracts

Total media spend

Print

Media Events

War Handling

Media Events

Media Events work by themselves, but also act as catalysts to Print and Direct Marketing

2323

Contracts: Influence per $ Spend

Spending Category Contracts per $100,000 Sig. LevelInternet Search 200 80%Internet CPL 107 97%Media Event 32Internet 29Direct Marketing (Total) 29 94%Print 21Television 3Total Media Spend 2.4 80%Radio NSEmerging NSOut of Home Negative 99%

Of the variables that have positive correlation with number of contracts, Internet (specifically CPL and Search), Direct Marketing and Media Events have the highest rate of return per additional $100K spent

2424

Implication: Media Influences

LEADS

CONTRACTS

Ove

rall

Spe

ndin

gG

ener

al M

arke

t Spe

nd

Radio

TV

Internet CPL

Direct Marketing

Media Events

25

Discussion

Data requirements –– As many variables as you are able to identify– OG supplements with secondary data sources and

GIS database information

Time periods – Ideally 2-3 years of back-data depending on

category, and if it is monthly/weekly

Data format– It doesn’t matter …we do the Extraction,

Transformation and data Loading (ETL) work

2626

In Summary…

Marketing Analytics…not just “market research”– Primary data + secondary data– Integrate and synthesize data from throughout the

company – not just research data but also sales, marketing plan investments, etc.

Using proven tools and templates… – We frame data and convert to actionable information

2727

Intersecting marketing, science and technology™