prediction of a nonlinear time series with feedforward neural networks
DESCRIPTION
Prediction of a nonlinear time series with feedforward neural networks. Mats Nikus Process Control Laboratory. The time series. A closer look. Another look. Studying the time series. Some features seem to reapeat themselves over and over, but not totally ”deterministically” - PowerPoint PPT PresentationTRANSCRIPT
Prediction of a nonlinear time series with feedforward neural networks
Mats NikusProcess Control Laboratory
The time series
0 100 200 300 400 500 600 700 800 900 1000-1.5
-1
-0.5
0
0.5
1
1.5
2
k
y(k)
A closer look
250 260 270 280 290 300 310 320 330
-1
-0.5
0
0.5
1
1.5
k
y(k)
Another look
520 530 540 550 560 570
-1
-0.5
0
0.5
1
1.5
k
y(k)
Studying the time series
Some features seem to reapeat themselves over and over, but not totally ”deterministically”Lets study the autocovariance function
The autocovariance function
0 10 20 30 40 50 60 70 80 90 100-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
l (lag)
(l)
Studying the time series
The autocovariance function tells the same: There are certainly some dynamics in the dataLets now make a phaseplot of the dataIn a phaseplot the signal is plotted against itself with some lagWith one lag we get
Phase plot
-1.5 -1 -0.5 0 0.5 1 1.5 2-1.5
-1
-0.5
0
0.5
1
1.5
2
y(k)
y(k+
1)
3D phase plot
-2-1
01
2
-2
-1
0
1
2-2
-1
0
1
2
y(k-2)y(k-1)
y(k)
The phase plots tell
Use two lagged valuesThe first lagged value describes a parabolaLets make a neural network for prediction of the timeseries based on the findings.
The neural network
y(k+1)^
y(k) y(k-1)
Lets try with 3 hidden nodes2 for the ”parabola”and one for the ”rest”
Prediction results
0 5 10 15 20 25 30 35 40 45 50-1.5
-1
-0.5
0
0.5
1
1.5
2
sample #
y pred
ict y
Residuals (on test data)
0 50 100 150 200 250 300 350 400 450 500-0.1
-0.08
-0.06
-0.04
-0.02
0
0.02
0.04
0.06
0.08
0.1
sample #
y pred
ict-y
A more difficult case
If the time series is time variant (i.e. the dynamic behaviour changes over time) and the measurement data is noisy, the prediction task becomes more challenging.
Phase plot for a noisy timevariant case
-1.5 -1 -0.5 0 0.5 1 1.5 2-1.5
-1
-0.5
0
0.5
1
1.5
2
y(k-1)
y(k)
Residuals with the model
0 500 1000 1500 2000 2500-0.1
-0.08
-0.06
-0.04
-0.02
0
0.02
0.04
0.06
0.08
0.1
sample #
y pred
ict-y
Use a Kalman-filter to update the weights
We can improve the predictions by using a Kalman-filterAssume that the process we want to predict is described by
11
1
,
kkkk
kkk
vufy
w
Kalman-filter
Use the following recursive equations
kkk uNNy ,ˆˆ 1
1ˆˆ
k
kk
yC
111
RCPCCPK T
kkkTkkk
kkkkk yyK ˆˆˆ1
QPCKIP kkkk 1
The gradient needed inCk is fairly simple to calculate for a sigmoidalnetwork
Residuals
0 500 1000 1500 2000 2500-0.1
-0.08
-0.06
-0.04
-0.02
0
0.02
0.04
0.06
0.08
0.1
sample #
y pred
ict-y
Neural network parameters
0 500 1000 1500 2000 2500-30.5
-30
-29.5
-29
sample #
w10
0 500 1000 1500 2000 250035
40
45
50
sample #
w11
Henon series
The timeseries is actually described by
20
312
21
.b
.a
byyay kkk