Intelligent Database Systems Lab
國立雲林科技大學National Yunlin University of Science and Technology
1
SOM time series clustering and prediction with recurrent neural networks
Aymen Cherif , Hubert Cardot , Romuald Bone2011, Necurocomputing
Presented by Chien-Hao Kung2011/11/3
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
2
Outlines· Motivation· Objectives· Methodology· Experiments· Conclusions· Comments
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
3
Motivation
· Local models for regression have been the focus of a great deal of attention in the recent years.
· Many models have been proposed to cluster time series and they have been combined with several predictors
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
4
Objectives
· This paper presents an extension for recurrent neural networks applied to local models and a discussion about the obtained results.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Methodology
· From global models to local models─ Step1: The time series is embedded into M-dimensional
space vectors
─ Step2:The time series is clustered into sublearning sets.
─ Step3:Local predictions are performed on each subset.
5
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Methodology
· Multi-layer perceptron─ A multilayer perceptron (MLP) is a feedforward
artificial neural network model
Step1: The time series is embedded into M-dimensional space vectors
6
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Methodology· VQ
─ VQ was a method used for reducing a large volume of vectors to a smaller number of distribution.
· Self-Organizing Maps(SOM)─ The SOM has the advantage of being easy to use─ However, since the original Self-Organizing Maps
algorithm does not take into account the temporal sequence processing.
7
Step2: The time series is clustered into sublearning sets
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Methodology
8
· Alternative clustering way
- Type1:the use of recurrent processing of time signal with recurrent BMU computation Temporal Kohonen Map(TKM) Recursive Self-Organizing Maps(RSOM)
- Type2:consists in mapping the temporal dependencies to spatial correlation. Mege Self-Organizing Maps(MSOM) The SOM with Temporal Activity Diffusion(SOMTAD)
Step2: The time series is clustered into sublearning sets
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Methodology
9
Step3:Local predictions are performed on each subset.
· MLP as local predictors─ The use of a temporal windows which is precisely the
same as the one used in the clustering step.─ The feedforward nature of the MLP network─ The output calculation and the weights modification
are done at the same time step as the learning process.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Methodology
10
Step3:Local predictions are performed on each subset.
· RNN as local predictors─ Original RNN
─ Back-Propagation Through Time(BPTT)
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Methodology
11
Step3:Local predictions are performed on each subset.
· RNN as local predictors
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Methodology
12
Step3:Local predictions are performed on each subset.
· RNN as local predictors
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
· Time series─ Sunspots time series
─ Laser time series
─ The Mackey-Glass(MG)-17
Experiments
13
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
· Sunspots time series
Experiments
14
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Experiments
15
· Laser time series
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Experiments
16
· MG-17 time series
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Experiments
17
· Experiments on sunspot
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Experiments
18
· Experiments on Laser time series
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Experiments
19
· MG-17 time series
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
20
Conclusions· This paper preferred to use the original SOM
algorithm in order to demonstrate the contribution of RNN as a local model.
· However, this paper saw that the performance of the model depends on the clustering and also on the nature of the time series.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
21
Comments
· Drawback─ The paper is useful for time series
Application─ Time sereis