[italian] enea seminar - computational intelligence and energy systems: intelligent solutions for...
DESCRIPTION
Computational Intelligence and Energy Systems: intelligent solutions for complex problems 31/05/2011 (in Italian)TRANSCRIPT
Computational Intelligenceand Energy Systems: intelligent solutions for
complex problems
Matteo De FeliceUnità Modellistica Energetica Ambientale
UTMEA - ENEA
1Tuesday, May 31, 2011
Sommario
Cos’è la Computational Intelligence (CI)?
Quali sono le applicazioni della CI ai sistemi complessi?
2Tuesday, May 31, 2011
CI: paradigmi
NN
EC FS
SI AIS
Soft Computing
Computational Intelligence
IA?
3Tuesday, May 31, 2011
Visione d’insieme
NN
EC FS
SI AIS
Temi principali
4Tuesday, May 31, 2011
CI e letteratura
1994 1996 1998 2000 2002 2004 2006 2008 20100
1
2
3
4
5x 10 3
year
Evolutionary ComputationSwarm IntelligenceArtificial Neural Networks
Dati dalla Thomson Reuters ISI considerando Computer Science & Technology (Gennaio 2010)
Due journals sulla CI nei primi 10 in CS (IF 2009)
5Tuesday, May 31, 2011
La diffusione della CI
Problemi sempre più complessi
Più potenza di calcolo disponibile
6Tuesday, May 31, 2011
ma...Assenza di una teoria consolidata
Frammentazione degli algoritmi
Approccio poco sistematico e confronti poco “robusti”
PSO APSO CPSO DPSO EPSO FPSO GPSO HPSO IPSO LPSO MPSO NPSO OPSO PPSO QPSO RPSO SPSO TPSO UPSO VPSO WPSO GA AGA BGA CGA DGA EGA FGA HGA IGA KGA LGA MGA OGA PGA QGA RGA SGA VGA ...
7Tuesday, May 31, 2011
Applicazioni Principali
1) Modellazione & Forecasting
2) Ottimizzazione
CalcoloEvolutivo
Reti Neurali
& Logica Fuzzy
8Tuesday, May 31, 2011
Quadro generale
Reti Neurali Evolutive
Ensemble
Reti neurali evolutive con topologia a rete complessa
Artificial Neural Networks and Support Vector Machines ensembling: a comparison
2008Evolving predictive neural models for complex processes
Evolving Complex Neural Networks
2009 9Tuesday, May 31, 2011
2009Modellazione temperature con NN
Ambient temperature modelling with soft computing techniques
Combining Back-Propagation and Genetic Algorithms to Train Neural Networks for Ambient Temperature Modeling in Italy
2010Ottimizzazione dello start-upcentrale a ciclo combinato
Combining Back-Propagation and Genetic Algorithms to Train Neural Networks for Ambient Temperature Modeling in Italy
10Tuesday, May 31, 2011
2011Reti Neurali e Load Forecast
Climate Variables in Energy Modeling
Short-Term Load Forecasting with Neural Network Ensembles: a Comparative Study
11Tuesday, May 31, 2011
Altri Progetti
IdentificazioneStructural System
in Ing. Sismica
Reti Neurali Evolutive e
Applicazioni alla Finanza
Algoritmi Evolutivi Spazialmente
Strutturati
Ottimizzazione traiettorie missioni
interplanetarie
12Tuesday, May 31, 2011
Ottimizzazione13Tuesday, May 31, 2011
Process Optimization
Come migliorare la ‘performance’ di un processo tramite i suoi parametri?
ProcessProcess
Parameters (X)
Environment
Measurement
14Tuesday, May 31, 2011
Ottimizzazione tradizionale
Metodi Line-search and trust-region (serve l’Hessiano!)
Metodi Quasi-newton (Hessiano approssimato)
Metodi Derivative-free
15Tuesday, May 31, 2011
...ma il real-world è:
1) ‘Rumoroso’
2) Dinamico
3) Difficile da esaminare
16Tuesday, May 31, 2011
Evolutionary Computation (EC)
Ottimizzazione Black-box
Singolo e Multi-Obiettivo
Anche funzioni discontinue e non differenziabili
Meta-euristica Population-based
17Tuesday, May 31, 2011
Metaeuristica
Ottimizzazione Stocastica
Algoritmi usati per trovare soluzioni a problemi “difficili”
Esempio: Hill-Climbing, Tabu Search, Simulated-Annealing
18Tuesday, May 31, 2011
Real-World problems
19Tuesday, May 31, 2011
Metodi di ottimizzazioneLipschitzian Optimization
DIRECT AlgorithmApplications
Taxonomy of Methods
Yves Brise Lipschitzian Optimization, DIRECT Algorithm, and Applications20Tuesday, May 31, 2011
ApplicazioneOttimizzazione dello startup di una centrale a ciclo combinato (CCPP)
Minimizzazione del tempo di avvio, consumi, emissioni e stress termico
Massimizzazione della produzione di energiaM. De Felice, I. Bertini, A. Pannicelli, and S. Pizzuti, "Soft Computing based optimisation of combined cycled power plant start-up operation with fitness approximation methods," Applied Soft Computing, 2011.
I. Bertini, M. De Felice, F. Moretti, and S. Pizzuti, "Start-Up Optimisation of a Combined Cycle Power Plant with Multiobjective Evolutionary Algorithms," in Applications of Evolutionary Computation, 2010, pp. 151-160.
21Tuesday, May 31, 2011
Procedura1. Definizione di un indice di
performance
2. Impostazione simulatore sw
3. Algoritmo EC tramite simulatore
22Tuesday, May 31, 2011
Indice Performance
Informazioni dagli esperti di processo
Knowledge modeling con funzioni fuzzy
0 0.5 1 1.5 2 2.5 3x 104
0
0.5
1
F1
0 0.5 1 1.5 2 2.5 3x 105
0
0.5
1
F2
0 5 10 15x 109
0
0.5
1
F3
0 5 10 15 20 25 300
0.5
1
F4
0 50 100 150 200 250 3000
0.5
1
F5
23Tuesday, May 31, 2011
Singolo-obiettivo
Algoritmo Genetico
operazione di mutazione Gaussiano
Funzione di fitness approssimata per velocizzare l’ottimizzazione(da 2070 a 36 ore/CPU)
24Tuesday, May 31, 2011
Risultati
Tempo avvio Consumi Prod.
Energia Emissioni Stress Termico
Esperti 21070 143557 2.5•109 25 10
GA 16569 115070 1.86•109 18.8 78.4
Var. Norm. -25% -16% -16% -30% 2%
25Tuesday, May 31, 2011
Multi-obiettivo
3.9 4 4.1 4.2 4.3 4.4 4.5 4.6x 109
12.2
12.25
12.3
12.35
12.4
12.45
12.5
12.55
12.6
12.65
Energy Production (KJ)
Emis
sion
s (m
g s
/ N m
3 )
RealNSGA 2WSGARAND
26Tuesday, May 31, 2011
Modellazione & Forecasting
27Tuesday, May 31, 2011
Modellazione con NNs||F (x)− f(x)|| < �, ∀x
6.50 1 2 3 4 5 6
1.2
-1.2
-0.6
0
0.6
X Axis
Y A
xis
y = sin(x)NN(x)
28Tuesday, May 31, 2011
Modellazione con NNs
Errore (MSE)
Metodi empirici per decidere la topologia della rete
System
NeuralNetwork
Input u(k) Output y(k)
Disturbances
29Tuesday, May 31, 2011
Regressione con NN
Si una una NN per fare regressione non-lineare
30Tuesday, May 31, 2011
Time Series Forecasting
Possiamo fare una previsione dei dati futuri usando quelli osservati
Altre informazioni utili (!)
31Tuesday, May 31, 2011
Approcci per le NN
NeuralNetwork
Input attime t
y(t+1)y(t+2)
...
y(t+N)
Direct Method
NeuralNetwork
Input attime t output t+1
delay
output tIterative Method
NeuralNetwork
Input attime t output t+1
delay
output tIterative Method
32Tuesday, May 31, 2011
Short-Term Load Forecasting
Dati Orari
Obiettivo: predizione del carico fino a 24 ore
0 200 400 600 800 1000 1200 1400 1600 1800 20000
20
40
60
hours
kW
33Tuesday, May 31, 2011
Modelli Seasonal
ΦP (Bs)φ(B)∇D
s ∇dxt = α+ΘQ(Bs)θ(B)et
0 10 20 30 40 500.5
0
0.5
1
Implementazione in R
34Tuesday, May 31, 2011
Modello NN
Campioni passati
Informazioni aggiuntive
Rete Neurale
Previsione
35Tuesday, May 31, 2011
Rete Neurale
Pesi wi Pesi wo
Funzioni di attivazione f differenziabili
36Tuesday, May 31, 2011
Backpropagation
[Werbos, 1974]
Forward phase: il segnale si propaga “in avanti”
Backward phase: si calcola l’errore e lo si propaga “all’indietro”, modificando i pesi
37Tuesday, May 31, 2011
Modello NN
240 2 4 6 8 10 12 14 16 18 20 22
36
10
15
20
25
30
X Axis
Y A
xis
y(k+1)
y(k)
y(k-1)
Come scegliere i lags?
38Tuesday, May 31, 2011
Data Analysis1. ACF
2. Distribution
3. Multivariate analysis
0 10 20 30 40 500.5
0
0.5
1
hourkW
1 5 9 13 17 21 24
5
10
15
20
25
30
35
40
45
50
0
0.05
0.1
0.15
0.2
0.25
0 20 40 60 80 1000
10
20
30
40
50
60
occupancy
load
(kW
)
y = 0.0013*x2 + 0.26*x + 12
39Tuesday, May 31, 2011
Domanda...
Come ridurre la varianza delle reti neurali?
40Tuesday, May 31, 2011
Ensembling
41Tuesday, May 31, 2011
Ensembling
1. Calibrazione del modello usando sottoinsiemi dei dati (Bagging)
2. Uso dei dati pesato per importanza (Adaboosting)
3. Interazione e cooperazione tra gli stimatori
42Tuesday, May 31, 2011
Ensembling
[Hansen & Salomon, 1990]
Majority voting (classificazione)
Combinazione lineare (regressione)
F (x,D) =1
N
N�
i=1
Fi(x,D)
43Tuesday, May 31, 2011
Ensembling
Media
44Tuesday, May 31, 2011
ApplicazioniSTLF dell’edificio ENEA Casaccia (C59)
Presentato al IEEE Symposium on CI Applications in Smart Grid
M. De Felice and X. Yao, "Neural Networks Ensembles for Short-Term Load Forecasting," in IEEE Symposium Series in Computational Intelligence 2011 (SSCI 2011), 2011
45Tuesday, May 31, 2011
Tecniche
Predittore naive:
modello SARIMA (Seasonal ARIMA):
Reti Neurali (NN)
NN Ensembles
ΦP (Bs)φ(B)∇D
s ∇dxt = α+ΘQ(Bs)θ(B)et
46Tuesday, May 31, 2011
Dati misurati da Settembre a Novembre 2009
Training (13 settimane) e testing (una settimana divisa in T1 e T2)
20582010 2013 2016 2019 2022 2025 2028 2031 2034 2037 2040 2043 2046 2049 2052 2055
40
10
15
20
25
30
35
hours
kW
24 hours
training part
Metodologia
47Tuesday, May 31, 2011
Misure d’errore
Errore Assoluto (MAE e MSE)
Error Percentuale (MAPE)
Scaled Error (MASE)
48Tuesday, May 31, 2011
Negative Correlation Learning
[Liu & Yao, 1999]
Modifica alla funzione di backpropagation
Penalty term λ
ei =M�
n=1
(Fi(xn)− yn)2 + λpi
49Tuesday, May 31, 2011
Regularized NCL
[Chen & Yao, 2009]
NCL con Regolarizzazione
ei =1
N
M�
n=1
(Fi(xn)− yn)2 − 1
N
M�
n=1
(Fi(x)− F (xn))2+
+αiwTi wi
50Tuesday, May 31, 2011
ErroriMAE MSE
NN (Media) 2.34 (0.79)2.49 (1.47)
10.9 (17.88)21.67 (59.29)
NN Ensemble 1.381.09
2.952.4
RNCL 1.471.07
3.342.82
Naive 2.112.28
7.616.4
SARIMA 1.891.24
5.522.17
51Tuesday, May 31, 2011
Dati AggiuntiviInformazioni aggiuntive: occupanti edificio, ora del giorno, giorno della settimana, giorni lavorativi.
NN: input aggiuntivi
SARIMA: termine lineare addizionale
52Tuesday, May 31, 2011
Dati Aggiuntivi
0 20 40 60 80 100 120 1400
1
2
3
4
Forecasting window
Abso
lute
erro
r
SARIMA external dataSARIMA
0 20 40 60 80 100 120 1400
1
2
3
4
forecast window
abso
lute
erro
r
MLP Ensemble external dataMLP Ensemble
0 20 40 60 80 100 120 1400
1
2
3
4
forecast window
abso
lute
erro
r
53Tuesday, May 31, 2011
Errori – dati aggiuntiviMAE MSE
NN (Media) 2.46 (0.83)2.34 (1.00)
12.13 (16.80)11.61 (10.61)
NN Ensemble 1.420.75
3.301.27
RNCL 1.330.92
2.71.62
Naive 2.112.28
7.616.4
SARIMA 1.911.20
5.612.07
54Tuesday, May 31, 2011
Errori giornalieri
5 10 15 200
1
2
3
(a) SARIMA T1
5 10 15 200
1
2
3
(b) MLP Ensembling T1
5 10 15 200
1
2
3
(c) RNCL T1
5 10 15 200
1
2
3
(d) SARIMA T2
5 10 15 200
1
2
3
(e) MLP Ensembling T2
5 10 15 200
1
2
3
(f) RNCL T2
Fig. 6. Univariate approach: 24-hours ahead forecasting absolute errors on both T1 and T2. In light grey the area between the 1st and the 3rd quartiles.
hour of the day
1 5 9 13 17 21 24
20
40
60
80
100
120
140
1
2
3
4
5
6
7
8
(a) SARIMA
hour of the day
1 5 9 13 17 21 24
20
40
60
80
100
120
140
1
2
3
4
5
6
7
8
(b) MLP Ensembling
hour of the day
1 5 9 13 17 21 24
20
40
60
80
100
120
140
1
2
3
4
5
6
7
8
(c) RNCL
Fig. 7. Absolute errors (in kW) made during testing parts T1 and T2 for the univariate approach models. On the Y axis there are the various forecastingwindows and on the X axis the hour of the day of each of the 24 prediction errors. Note that the color scale is not the same in each plot.
TABLE IIMODELS PERFORMANCES: APPROACH WITH OCCUPANCY, HOUR OF THE DAY AND WORKDAY FLAG (ALL THE ERROR VALUES ARE ROUNDED TO TWO
DECIMALS). IN BRACKETS THE STANDARD DEVIATION WHERE NEEDED AND IN BOLD THE BEST MODEL ERROR FOR THE TESTING PART.
Training Testing T1 Testing T2Model MAE MSE MAE MSE Max MAE MSE Max
Naive (no ext. data) 2.45 14.97 2.11 7.61 7.35 2.28 6.4 6.36SARIMA 1.13 4.31 1.91 5.61 8.00 1.20 2.07 5.18
ANN MLP best training 0.36 0.70 3.51 20.28 18.00 2.20 11.83 24.53Average ANN MLP 1.20 (0.31) 3.25 (1.52) 2.46 (0.83) 12.13 (16.80) 13.84 (16.62) 2.34 (1.00) 11.61 (10.61) 13.00 (6.01)MLP Ensemble 0.74 1.47 1.42 3.30 7.98 0.75 1.27 4.79
ANN RBF best training 1.06 2.48 1.36 3.03 6.43 0.88 1.61 7.05Average ANN RBF 1.65 (0.86) 7.71 (10.61) 1.97 (1.01) 7.99 (13.19) 8.22 (5.71) 1.77 (1.39) 8.98 (21.24) 10.74 (6.18)
RNCL 1.15 3.35 1.33 2.71 5.37 0.92 1.62 4.52
information is provided in Figure 9, where, after a comparisonwith Figure 7, we can see in which part of the datasetadditional data has reduced the error.
VII. DISCUSSION
In Figure 10 testing absolute errors are shown, arranged inascending order, for all the MLP and RBF networks on testing
set T1 (we omitted T2 for sake of clearness). It’s evident howneural network ensembles exhibits an error lower or at leastequal than the best network, both for MLP and RBFs. Thismeans that ensembling allows, thank to the exploitation ofall the information ‘contained’ within the trained networks,to achieve an effective forecasting overcoming the drawbacksof neural networks: overfitting and the high variability of the
55Tuesday, May 31, 2011
Errori giornalieri
5 10 15 200
1
2
3
(a) SARIMA T1
5 10 15 200
1
2
3
(b) MLP Ensembling T1
5 10 15 200
1
2
3
(c) RNCL T1
5 10 15 200
1
2
3
(d) SARIMA T2
5 10 15 200
1
2
3
(e) MLP Ensembling T2
5 10 15 200
1
2
3
(f) RNCL T2
Fig. 6. Univariate approach: 24-hours ahead forecasting absolute errors on both T1 and T2. In light grey the area between the 1st and the 3rd quartiles.
hour of the day
1 5 9 13 17 21 24
20
40
60
80
100
120
140
1
2
3
4
5
6
7
8
(a) SARIMA
hour of the day
1 5 9 13 17 21 24
20
40
60
80
100
120
140
1
2
3
4
5
6
7
8
(b) MLP Ensembling
hour of the day
1 5 9 13 17 21 24
20
40
60
80
100
120
140
1
2
3
4
5
6
7
8
(c) RNCL
Fig. 7. Absolute errors (in kW) made during testing parts T1 and T2 for the univariate approach models. On the Y axis there are the various forecastingwindows and on the X axis the hour of the day of each of the 24 prediction errors. Note that the color scale is not the same in each plot.
TABLE IIMODELS PERFORMANCES: APPROACH WITH OCCUPANCY, HOUR OF THE DAY AND WORKDAY FLAG (ALL THE ERROR VALUES ARE ROUNDED TO TWO
DECIMALS). IN BRACKETS THE STANDARD DEVIATION WHERE NEEDED AND IN BOLD THE BEST MODEL ERROR FOR THE TESTING PART.
Training Testing T1 Testing T2Model MAE MSE MAE MSE Max MAE MSE Max
Naive (no ext. data) 2.45 14.97 2.11 7.61 7.35 2.28 6.4 6.36SARIMA 1.13 4.31 1.91 5.61 8.00 1.20 2.07 5.18
ANN MLP best training 0.36 0.70 3.51 20.28 18.00 2.20 11.83 24.53Average ANN MLP 1.20 (0.31) 3.25 (1.52) 2.46 (0.83) 12.13 (16.80) 13.84 (16.62) 2.34 (1.00) 11.61 (10.61) 13.00 (6.01)MLP Ensemble 0.74 1.47 1.42 3.30 7.98 0.75 1.27 4.79
ANN RBF best training 1.06 2.48 1.36 3.03 6.43 0.88 1.61 7.05Average ANN RBF 1.65 (0.86) 7.71 (10.61) 1.97 (1.01) 7.99 (13.19) 8.22 (5.71) 1.77 (1.39) 8.98 (21.24) 10.74 (6.18)
RNCL 1.15 3.35 1.33 2.71 5.37 0.92 1.62 4.52
information is provided in Figure 9, where, after a comparisonwith Figure 7, we can see in which part of the datasetadditional data has reduced the error.
VII. DISCUSSION
In Figure 10 testing absolute errors are shown, arranged inascending order, for all the MLP and RBF networks on testing
set T1 (we omitted T2 for sake of clearness). It’s evident howneural network ensembles exhibits an error lower or at leastequal than the best network, both for MLP and RBFs. Thismeans that ensembling allows, thank to the exploitation ofall the information ‘contained’ within the trained networks,to achieve an effective forecasting overcoming the drawbacksof neural networks: overfitting and the high variability of the
56Tuesday, May 31, 2011
Ensemble: altro esempio
60 80 100 120 140 160 180 200 220
20
40
60
80
100
testing hours
kW
57Tuesday, May 31, 2011
TO-DO
Ensemble: usare tutte le stime per creare una pdf
Ibridizzazione con metodi statistici classici: analisi multivariate, modelli stagionali, Holt-Winters
58Tuesday, May 31, 2011
The Big View
Forecasting & Modeling
59Tuesday, May 31, 2011
Passi principali
1. Definizione target (short-term, medium-term, seasonal)
2. Raccolta dati e analisi
3. Definizione e comparazione tecniche
4. Valutazione
5. Simulazione
Statistical Analysys
High-dimensionalityData Mining
Time Series MethodsNNsHybrid Methods
Cost AnalysisPerformance Measures
Software Simulator
Multi-Agent Systems
60Tuesday, May 31, 2011
PPSN 2012
12th International Conference on “Parallel Problem Solving From Nature”, Taormina
Paper submission: 15 Marzo 2012 (Proceedings Springer)
http://www.dmi.unict.it/ppsn2012/
61Tuesday, May 31, 2011
http://matteodefelice.name/research
62Tuesday, May 31, 2011