in-depth workshop: digital forecasting and...

JPK

Gro

upBusiness Forecasting and Analytics Forum

September 19-20 • Chicago, IL

In-Depth Workshop:Digital Forecasting and Analytics

September 20, 1:15pm

Widely considered one of the leading digital measurement experts in the world, Garyleads EY’s Digital Analytics Practice. EY acquired Gary’s previous company –

Semphonic – in March of 2013. As Semphonic’s President and co-Founder, Gary ledSemphonic’s growth over a 15 year period from a 2-person practice to the one of the

leading digital analytics practices in the United States. Voted the most InfluentialIndustry Contributor by the Digital Analytics Association in 2012, Gary writes an

influential blog (http://semphonic.blogs.com/semangel), has published more thantwenty whitepapers on advanced digital analytics practice and is a frequent speaker

at industry events.

View presentation online at:https://jpkgroupsummits.com/attendee5

Gary Angel – Ernst & Young

Examine the role of exogenous variables & discusshow/whether to include them in a forecast

Introduction to Digital Forecasting November 2015

Introduction to Digital

Forecasting

with Gary Angel

Introduction to Digital Forecasting

Overview

Today’s Agenda:

Introduction Forecasting Techniques

Matching Technique to Problem

Digital Analytics Model

Historical Forecasting Basics

Advanced Elements

Predictive Models

Sample Conceptual Models

Summary


But have you really thought about how it works?

You know what forecasting is…


The three primary types of forecast

Opinion-Based

Time-Series

Predictive Models

Ask the Experts

Avaraging, Trending

and

Smoothing

Model the System


With their common sub-methods

Opinion Based Methods

Market Research

Surveys

Expert Panels

Time Series

Averages & Moving Averages

Smoothing Methods

Models Regression

Econometric

Demand Signals

Simulation


Opinion-Based Methods

Best when information is limited (new channels, markets, products)

Difficult to replicate and fairly low accuracy


Time-Series Methods

Best when historical data points exist and are stable

Doesn’t capture levers of change


Predictive Models

Provides causal insight & allows for what-if analysis

Harder to build and needs enough variation in historical data plus knowledge of exogenous factors


Building a Digital Analytics Forecast


Building a digital forecast

► There are countless problems in digital that might require

a forecast. For our workshop, we’re going to focus on just

one:

Forecasting Website Traffic


Mathematical Techniques for Forecasting

► Outline

► Stability

► Moving Average

► Weighted Average

► Smoothing

► Break-outs


Stability

► All forecasting is based on the assumption that the future

will resemble the past.

► The simplest forecast (which we use far more than we

ought) is that things will remain exactly the same:

Actual Visits in

Jan. 2016

1,763,226

Forecast Visits

Feb. 2016

1,763,226


But stability from what/when?

► Even when stability is our forecast, it’s fair to ask

“Compared to what?”

► Here’s three different stable forecasts:

Actual Visits in

Jan. 2016

1,763,226

Forecast Visits

Feb. 2016

1,763,226

Actual Visits in

Feb. 2015

1,676,079

Forecast Visits

Feb. 2016

1,676,079

Visits per day in

Jan. 2016

1,676,079

Forecast Visits

Feb. 2016

1,592,591


Moving Average

► But we know things don’t remain the same. There’s often

lots of noise and variation in any given measurement.

► So one common technique is to try and eliminate noise by

taking the average of our past data points.

Forecast Visits

Feb. 2016

Month Visits1/1/2015 1,919,431

2/1/2015 1,676,079

3/1/2015 1,755,637

4/1/2015 1,842,057

5/1/2015 1,790,122

6/1/2015 1,822,100

7/1/2015 1,756,784

8/1/2015 1,842,306

9/1/2015 1,744,129

10/1/2015 1,691,928

11/1/2015 1,570,476

12/1/2015 1,507,756

1/1/2016 1,763,226

1,744,772


Moving Average – Can make for a slow learner

► When using an average, the number of periods

determines how quickly the measure detects changes

AND how much it is influenced by outliers:

1,200,000

1,400,000

1,600,000

1,800,000

2,000,000

Visit 12 month Average

1,200,000

1,400,000

1,600,000

1,800,000

2,000,000

Visit 12 month Average

3 Month Average


Weighted Moving Average

► A weighted average is designed to provide stability AND

rapid tracking.

► Each period is given a different weight – usually biasing

the average toward more recent periods.

Forecast Visits

Feb. 2016

Month Visits1/1/2015 1,919,431

2/1/2015 1,676,079

3/1/2015 1,755,637

4/1/2015 1,842,057

5/1/2015 1,790,122

6/1/2015 1,822,100

7/1/2015 1,756,784

8/1/2015 1,842,306

9/1/2015 1,744,129

10/1/2015 1,691,928

11/1/2015 1,570,476

12/1/2015 1,507,756

1/1/2016 1,763,226

1,721,567


Weighted average is designed to blend stability and sensitivity

► The stability and sensitivity of the forecast are functions of

the size of the window (# of periods used in the average)

and the weighting:

1,200,000

1,400,000

1,600,000

1,800,000

2,000,000

1/1

/20

15

2/1

/20

15

3/1

/20

15

4/1

/20

15

5/1

/20

15

6/1

/20

15

7/1

/20

15

8/1

/20

15

9/1

/20

15

10

/1/2

015

11

/1/2

015

12

/1/2

015

1/1

/20

16

Actual Weighted

1,200,000

1,400,000

1,600,000

1,800,000

2,000,000

1/1

/20

15

2/1

/20

15

3/1

/20

15

4/1

/20

15

5/1

/20

15

6/1

/20

15

7/1

/20

15

8/1

/20

15

9/1

/20

15

10

/1/2

015

11

/1/2

015

12

/1/2

015

1/1

/20

16

Actual Weighted

0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.1 0.1 0.1 0.3 0 0 0 0 0 0 0 0 0.1 0.2 0.3 0.4


Exponential smoothing

► Exponential Smoothing applies a weight to each iteration,

constantly adjusting the forecast score against the

difference between the previous forecast/actual. The

weight determines the amount of adjustment:

Forecast Visits

Feb. 2016

1,671,7261,200,000

1,400,000

1,600,000

1,800,000

2,000,000

4/1

/20

14

6/1

/20

14

8/1

/20

14

10

/1/2

014

12

/1/2

014

2/1

/20

15

4/1

/20

15

6/1

/20

15

8/1

/20

15

10

/1/2

015

12

/1/2

015

Actual Exponential Forecast

1671726

.5 Weight to Actual and .5 Weight to Forecast


Exponential Smoothing in Excel

► Excel includes a simple Exponential Smoothing function in

the Data Analysis Toolpack (an add-in):


Double exponential smoothing (Holt)

► Double exponential smoothing adjusts both the predicted

value and the trend (weighting) with each new value.

► This makes it much better for matching trends than Single

Exponential Smoothing.

Forecast Visits

Feb. 2016

1,790,276 1,200,000

1,400,000

1,600,000

1,800,000

2,000,000

1/1

/20

15

2/1

/20

15

3/1

/20

15

4/1

/20

15

5/1

/20

15

6/1

/20

15

7/1

/20

15

8/1

/20

15

9/1

/20

15

10

/1/2

015

11

/1/2

015

12

/1/2

015

1/1

/20

16

Visit Double Exp Forecast

Alpha 0.9

Beta 0.2


Double Exponential Smoothing in Excel

► Here’s the process for double exponential smoothing in

Excel:

1 Set your alpha (exponential smoothing value) and beta (trend value) – between 0-1

2You will create three new values: Exponential Forecast, Trend Forecast, and DESmoothed Forecast (this last one is the real forecast)

3 DESmoothed Forecast is always equal to the Exponential Forecast + Trend Forecast

4For the 1st Period, the Exponential Forecast is equal to the actual value for that period and the Trend Forecast is zero.

5For every other period, the Exponential Forecast is equal to the Previous Exponential Forecast plus the alpha value times the difference between the Previous Actual and the Previous DESmoothed Forecast.

6For every other period, the Trend Forecast is equal to the Previous Trend Forecast plus the beta value times the difference between the current Exponential Forecast and the Previous DESmoothed Forecast.



► It looks like this:1

2

3

3

4

5

5



► It looks like this:6

6


Triple exponential smoothing (Holt-Winters)

► Triple exponential smoothing adds a weight for a seasonal

cycle. The predicted value and trend value are updated

identically to the Holt method except that they are first

adjusted seasonally. The seasonal parameter can be

tuned or pre-determined.

1,200,000

1,400,000

1,600,000

1,800,000

2,000,000

Actual Forecast

► It should only be used

when your data has a

significant seasonal

component.


About those pesky parameters

► You can use tools (like Excel’s Solver) to find the best

values for the parameters.

► You do this by calculating the MSE (average of the

forecast errors after squaring) and then optimizing the

parms to that value.


Break-outs and Banding

► Break-outs and banding aren’t a separate forecasting

technique – they are tools for understanding whether a

movement is interesting.

► All processes have a certain amount of variation. Banding

is used to draw a band of fairly natural variation around

the trend. When an actual “breaks” the band, the variation

is usually significant.

1,200,000

1,400,000

1,600,000

1,800,000

2,000,000

2/1

/20

15

3/1

/20

15

4/1

/20

15

5/1

/20

15

6/1

/20

15

7/1

/20

15

8/1

/20

15

9/1

/20

15

10

/1/2

015

11

/1/2

015

12

/1/2

015

1/1

/20

16

Actual Upper Band Lower Band


Whew…Let’s take a break and then tackle Modeling


Model-based Forecasting

► Outline

► Time-Series vs. Models

► Identifying key variables

► Source, Season

► Building a Conceptual Model

► Sample Conceptual Models

► Quick Discussion of other Models


Time-Series vs. Model

Forecast Model


Building a Model

► The first (and maybe most important) step in building a

model is deciding what variables you might use.

► Keep in mind that there is no one right answer. The level

of variables you use needs to match the operational level

you want to understand.

► For example, if you’re trying to optimize channel

marketing decisions, it doesn’t work to use Total Marketing

Spend as your marketing variable.


Throwing variables at a wall

► Don’t just toss variables into a modelling blender.


Building a conceptual model

Website Traffic = Visits from last month * Repeat Visit Rate( ) + Avg. New Visits

Website Traffic = Search Visit per Dollar * Exp. Search Spend( ) +

Display Visit per Dollar * Exp. Display Spend( ) +

Avg. Direct Visits

Website Traffic = Current High Frequency Customers * Avg. Visit Propensity( ) +

Avg. New Visits

Current Med. Frequency Customers * Avg. Visit Propensity( ) +

Current Low Frequency Customers * Avg. Visit Propensity( ) +


SEO

Page Ranks

Keyword Volumes

Outcomes

Open Sessions

Repeat Rates

Satisfaction

Exogenous

Econometrics

Brand Awareness

Web Growth

Device Shifts

Seasonality

Sample systems and variables

Marketing

Total Spend

Digital Spend

Channel Spend

Mix

User-Base

Active Users

Users by Cohort

New Users Last Period

Segmentation

User Types

Visit Types


Deepening a conceptual model

Website Traffic = Visits sourced by marketing +

Visits sourced by our user-base +

Visits sourced by Web “Flow”

Visits sourced by marketing = Total Marketing Spend * constant

Visits sourced by marketing = Digital Marketing Spend * constant

Mass Marketing Spend * constant

+( )

( )

Visits sourced by marketing = PPC Marketing Spend * constant


+( )

( )

Display Marketing Spend * constant +( )


Deepening a conceptual model II

Visits sourced by marketing = PPC Marketing Spend * constant


+( )

( )

Display Marketing Spend * constant +( )

► Implies that the impact of increasing PPC marketing

spend will be constant. That’s rarely the case for any

variable except over a narrow band.

► We could further break out PPC Marketing spend into

categories (like brand, non-brand) but eventually we’ll run

into the same problem.


Modeling change in a variable

PPC Marketing Visits Sourced = PPC Marketing Spend * function(saturation)( )

► Media buying typically

decays both in terms of

impact per incremental

buy after a saturation

point and in terms of

time. 0 10 20 30 40 50 60 70 80 90 100

Constant

PPC Marketing Visits Sourced = PPC Marketing Spend * function(saturation)( ) +

Prior PPC Marketing Spend * function(saturation) * function(timedecay)( )


Paid Marketing: Outline Model

► Break-downs by Channel

► Inside Channel by Ad Group / Campaign

► National vs. Local

► Web / Mobile

► Make sure (at minimum) that brand and non-brand are separated

► Build decay model to capture time lag of spending

► Build saturation model to capture reduced incrementality of spend

by channel

► Think about integrating a full mix/attribution model

► Consider treating as a separate customer type or sub-type within

each customer segment


SEO: Outline Model

► Break-out Brand vs. Non-Brand

► Break out Long-Tail as a single entity

► For short-tail

► Track positional impact for major keywords

► Consider treating as a separate customer type or sub-type

within each customer segment


Mass Media: Outline Model

► Spend at highest level of temporal granularity possible

(Monthly / Weekly / Daily)

► Channel and GRPs (Planned or Delivered)

► National / Local

► Think about integrating a full mix model

► Consider use of pre-qualification survey to evaluate mass-

media source quality


Repeat Visitors: Return Model

► Two (potentially complementary) approaches:

► Customer-based by expected return frequency

► Visit-based by previous visit type

Advisors

Plan Managers

High-Wealth Investors

General Investors

1.8

1.5

1.3

1.2

1.3

1.2

1.3

1.1

1.6

1.4

1.8

1.3

1.9

1.4

1.6

1.3

Research SupportSpecific

FundGeneral

Fund


Oh the places you’ll go

► Outline – Other kinds of digital models

► Conversion Rate

► Site Alerting (Problem Identification)

► Marketing Spend

► Content Performance

► Brand impact


Take another deep breath and we’ll wrap up


When do we use what

► Factors in selecting an approach:

► Do you have historical data?

► Is the situation inherently unpredictable or chaotic?

► Do you need to understand WHY a system changed or WHAT you

can adjust? (you don’t always)

► How much data do you actually have?

► How much work are you willing to do?


Thank you

► [email protected]

► +1 415 894 8255

► LinkedIn:

www.linkedin.com/pub/gary-

angel/0/176/a43/

Gary Angel

in-depth workshop: digital forecasting and...

Documents