Dynamic Linear Models
FISH 507 – Applied Time Series Analysis
Mark Scheuerell
5 Feb 2019
Topics for today
Univariate response
Multivariate response
Stochastic level & growth
Dynamic Regression
Dynamic Regression with fixed season
Forecasting with a DLM
Model diagnostics
·
·
·
·
·
2/73
Simple linear regression
Let's begin with a linear regression model
The index has no explicit meaning in that shuffling ( ) pairs has noeffect on parameter estimation
3/73
Simple linear regression
We can write the model in matrix form
4/73
Simple linear regression
We can write the model in matrix form
with
and
5/73
Dynamic linear model (DLM)
In a dynamic linear model, the regression parameters change overtime, so we write
as
6/73
Dynamic linear model (DLM)
There are 2 important points here:
1. Subscript explicitly acknowledges implicit info in the time orderingof the data in
7/73
Dynamic linear model (DLM)
There are 2 important points here:
1. Subscript explicitly acknowledges implicit info in the time orderingof the data in
2. The relationship between and is unique for every
8/73
Constraining a DLM
Close examination of the DLM reveals an apparent problem forparameter estimation
9/73
Constraining a DLM
Close examination of the DLM reveals an apparent problem forparameter estimation
We only have 1 data point per time step (ie, is a scalar)
Thus, we can only estimate 1 parameter (with no uncertainty)!
10/73
Constraining a DLM
To address this issue, we'll constrain the regression parameters to bedependent from to
11/73
Constraining a DLM
In practice, we often make time invariant
or assume is an identity matrix
In the latter case, the parameters follow a random walk over time
12/73
DLM in state-space form
Observation model relates covariates to data
State model determines how parameters "evolve" over time
13/73
DLM in MARSS notation
Full state-space form
14/73
Contrast in covariate effects
Note: DLMs include covariate effect in the observation eqn muchdifferently than other forms of MARSS models
DLM: is covariates, is parameters
Others: is covariates, is parameters
15/73
Other forms of DLMs
The regression model is but one type
Others include:
stochastic "level" (intercept)
stochastic "growth" (trend, bias)
seasonal effects (fixed, harmonic)
·
·
·
16/73
The most simple DLM
Stochastic level
17/73
The most simple DLM
Stochastic level = random walk with obs error
18/73
Ex of stochastic level model
19/73
Ex of stochastic level model
20/73
Univariate DLM for level & growth
Stochastic "level" with deterministic "growth"
21/73
Univariate DLM for level & growth
Stochastic "level" with deterministic "growth"
This is just a random walk with bias
22/73
Univariate DLM for level & growth
Stochastic "level" with stochastic "growth"
Now the "growth" term evolves as well
23/73
Univariate DLM for level & growth
Evolution of and
24/73
Univariate DLM for level & growth
Evolution of and
25/73
Univariate DLM for level & growth
Observation model for stochastic "level" & stochastic "growth"
26/73
Univariate DLM for regression
Stochastic intercept and slope
27/73
Univariate DLM for regression
Parameter evolution follows a random walk
28/73
Univariate DLM with seasonal effect
Dynamic linear regression with fixed seasonal effect
29/73
Univariate DLM with seasonal effect
30/73
Univariate DLM with seasonal effect
How should we model the fixed effect of ?
31/73
Univariate DLM with seasonal effect
We don't want the effect of quarter to evolve
32/73
Univariate DLM with seasonal effect
OK, but how do we select the right quarterly effect?
33/73
Univariate DLM with seasonal effect
Let's separate out the quarterly effects
But how do we select only the current quarter?
34/73
Univariate DLM with seasonal effect
We can set some values in to 0 ( = 1)
35/73
Univariate DLM with seasonal effect
We can set some values in to 0 ( = 2)
36/73
Univariate DLM with seasonal effect
But how would we set the correct 0/1 values?
37/73
Univariate DLM with seasonal effect
We could instead reorder the within ( = 1)
38/73
Univariate DLM with seasonal effect
We could instead reorder the within ( = 2)
39/73
Univariate DLM with seasonal effect
But how would we shift the within ?
40/73
Example of non-diagonal
We can use a non-diagonal to get the correct quarter effect
41/73
Evolving parameters
42/73
Evolving parameters
43/73
Forecasting with a DLM
DLMs are often used in a forecasting context where we want aprediction for time based on the data up through time
44/73
Forecasting with a DLM
Pseudo-code
1. get estimate of today's parameters from yesterday's
2. make prediction based on today's parameters & covariates
3. get observation for today
4. update parameters and repeat
45/73
Forecasting with a DLM
1. Define the parameters at time
46/73
Forecasting with a DLM
1. Define the parameters at time
47/73
Forecasting with a DLM
1. Define the parameters at time
48/73
Forecasting with a DLM
1. Define the parameters at time
49/73
Forecasting with a DLM
1. Make a forecast of at time
50/73
Forecasting with a DLM
Putting it all together
51/73
Forecasting with a DLM
Putting it all together
Using MARSS() will make this easy to do
52/73
Diagnostics for DLMs
Just as with other models, we'd like to know if our fitted DLM meets itsunderlying assumptions
We can calcuate the forecast error as
and check if
with a QQ-plot (1) and an ACF (2)
53/73
MULTIVARIATE DLMs
54/73
The most simple multivariate DLM
Multiple observations of a stochastic level
with
55/73
Another simple multivariate DLM
Multiple observations of multiple levels
with
56/73
Multivariate DLMs
Regression model
Our univariate model
becomes
57/73
Kronecker products
If is an matrix and is a matrix
then will be an matrix
58/73
Kronecker products
For example
so
59/73
Multivariate DLMs
Regression model with
60/73
Multivariate DLMs
61/73
Multivariate DLMs
Covariance of observation errors
62/73
Multivariate DLMs
Parameter evolution
becomes
63/73
Multivariate DLMs
Parameter evolution
If we have 2 regression parameters and , then
64/73
Multivariate DLMs
Parameter evolution
65/73
Multivariate DLMs
66/73
Multivariate DLMs
Parameter evolution
67/73
Multivariate DLMs
Evolution variance
What form should we choose for ?
68/73
Multivariate DLMs
Evolution variance
Diagonal and equal (IID)
69/73
Multivariate DLMs
Evolution variance
Diagonal and unequal
70/73
Multivariate DLMs
Evolution variance
Unconstrained
71/73
Multivariate DLMs
Evolution variance
In practice, keep as simple as possible
72/73
Topics for today
Univariate response
Multivariate response
Stochastic level & growth
Dynamic Regression
Dynamic Regression with fixed season
Forecasting with a DLM
Model diagnostics
·
·
·
·
·
73/73