online learning in estimation of distribution algorithms for dynamic environments
TRANSCRIPT
Departamento de Engenharia de Computação e Automação
Industrial
Faculdade de Engenharia Elétrica e de Computação
Unicamp
Online learning in estimation of distribution algorithms for dynamic environments
André Ricardo GonçalvesFernando J. Von Zuben
2
Optimization in dynamic environments
Estimation of distribution algorithms
Mixture model and online learning
Proposed method: EDAOGMM
Experimental results
Concluding remarks and future works
References
Outline
3
Optimization in dynamic environments
Estimation of distribution algorithms
Mixture model and online learning
Proposed method: EDAOGMM
Experimental results
Concluding remarks and future works
References
Outline
4
Optimization in dynamic environments World is dynamic!
New events arrive and others canceled at a scheduling problem;
Vehicles must reroute around heavy trac and road repairs;
Machine breakdown occurs during a production run.
5
Optimization in dynamic environments Dynamic optimization algorithm should be able to react to the new
environment, updating the internal model and generating new candidate solutions;
Evolutionary algorithms (EAs) appear as promising approaches, since they maintain a population of solutions that can be adapted by means of a balance between exploration and exploitation of the search space;
EAs approaches: GA, PSO, AIS, EDAs, among others;
However, to be applied in dynamic environments, they must be adapted.
6
Optimization in dynamic environments
Estimation of distribution algorithms
Mixture model and online learning
Proposed method: EDAOGMM
Experimental results
Concluding remarks and future works
References
Outline
7
Estimation of distribution algorithms Estimation of distribution algorithms (EDA) are
evolutionary methods that use estimation of distribution techniques, instead of genetic operators.
The key aspect in EDAs is how to estimate the true distribution of promising solutions. Dependence trees, Bayesian networks, mixture models, etc.
Classication of EDAs based on complexity of probabilistic model.
8
Estimation of distribution algorithms
9
Optimization in dynamic environments
Estimation of distribution algorithms
Mixture model and online learning
Proposed method: EDAOGMM
Experimental results
Concluding remarks and future works
References
Outline
10
Mixture model and online learning Mixture models are flexible estimators;
In optimization, they are able to capture the multimodality of the search space;
Learning methods, such as Expectation-Maximization (EM), are computationally efficient;
In optimization of dynamic environments, the model tends to change constantly;
EM with online learning appear as a promising approach to model dynamic environments.
11
Mixture model and online learning Online learning
Fast adaptation model to the new data coming from the environment;
Approach proposed by (Nowlan,1991) stores the relevant information in a vector of sufficient statistics;
Exponential decay (γ) of the data importance to the model.
12
Optimization in dynamic environments
Estimation of distribution algorithms
Mixture model and online learning
Proposed method: EDAOGMM
Experimental results
Concluding remarks and future works
References
Outline
13
Proposed method: EDAOGMM
EDA with online Gaussian mixture model (EDAOGMM)
Employs an incremental and constructive mixture model (low computational cost);
Self-adjusts the components number by means of BIC;
Model tends to adapt to the multimodality of search space;
Employs a “random immigrants” approach to promote population diversity;
14
Proposed method: EDAOGMM
15
Proposed method: EDAOGMM Selection method:
Stochastic selection aids to preserve the population diversity; η parameter defines the balance between exploration and explotation.
Diversity control: Stochastic selection; Random immigrants; Controlled reinitializations (δ parameter).
Components number control: Incremental and constructive approach; Removal of overlapped components (ε parameter).
16
Proposed method: EDAOGMM New population is composed by 3 subpopulations (dependent
of the η parameter): Sampled by the mixture model; Best individuals; Random immigrants.
Overlapped components is a redundant representation of a promising region Remove the component with lower mixture coefficient; Check the overlap using the ε parameter.
17
Optimization in dynamic environments
Estimation of distribution algorithms
Mixture model and online learning
Proposed method: EDAOGMM
Experimental results
Concluding remarks and future works
References
Outline
18
Experimental results Moving Peaks benchmark (MPB) generator plus a rotation method (Li &
Yang, 2008);
Fitness surface are composed by a set of peaks that changes your positions, heights and widths over time;
Maximization problem in a continuous space;
Seven types of change (T1-T7): small step, large step, random, chaotic, recurrent, recurrent with noise and random with dimensional changes;
There are parameters to control the multimodality of the search space, severity of changes and the dynamism of the environment;
Range of search space: [-5,5];
Problem dimensions: 10 and [5-15].
19
Experimental results Six dynamic environments settings were considered:
20
Experimental results Contenders proposed algorithms in the literature:
Improved Univariate Marginal Distribution Algorithm - IUMDA (Liu et al., 2008);
Tri-EDAG (Yuan et al., 2008); Hypermutation Genetic Algorithm - HGA (Cobb,1990).
Two EDAs and a GA developed for dynamic environments.
21
Experimental results Free parameters settings:
22
Experimental results Comparison metrics:
Offline error Average of the absolute error between the best solution found
so far and the global optimum (known) at each time step t.
23
Experimental results Scenarios 1 and 2
24
Experimental results Scenarios 3 and 4
25
Experimental results Scenarios 5 and 6
26
Optimization in dynamic environments
Estimation of distribution algorithms
Mixture model and online learning
Proposed method: EDAOGMM
Experimental results
Concluding remarks and future works
References
Outline
27
Concluding remarks and future works EDAOGMM outperforms all the contenders, particularly in high-
frequency changing environments (Scenarios 1 and 2);
EDAOGMM has a fast convergence because it can explore several peaks simultaneously;
We can detect a less prominent performance in low frequency scenarios (5 and 6), indicating that, once converged, the EDAOGMM reduces its exploration power;
So, a continued control to avoid premature convergence is desirable.
28
Concluding remarks and future works Future works:
Incorporate a continued convergence control mechanism;
Compare EDAOGMM with other algorithms designed to deal with dynamic environments;
Increment the experimental tests aiming at investigating scalability and other aspects related to the relative performance of the proposed algorithm;
Performs a parameter sensitivity analisys.
29
Optimization in dynamic environments
Estimation of distribution algorithms
Mixture model and online learning
Proposed method: EDAOGMM
Experimental results
Concluding remarks and future works
References
Outline
30
References S. Nowlan, “Soft competitive adaptation: neural network learning
algorithms based on fitting statistical mixtures,” Ph.D. dissertation, Carnegie Mellon University, Pittsburgh, PA, USA, 1991.
C. Li and S. Yang, “A generalized approach to construct benchmark problems for dynamic optimization,” in Proc. of the 7th Int. Conf. on Simulated Evolution and Learning, 2008.
X. Liu, Y. Wu, and J. Ye, “An Improved Estimation of Distribution Algorithmin Dynamic Environments,” in Fourth International Conference on Natural Computation. IEEE Computer Society, 2008, pp. 269–272.
B. Yuan, M. Orlowska, and S. Sadiq, “Extending a class of continuous estimation of distribution algorithms to dynamic problems,” Optimization Letters, vol. 2, no. 3, pp. 433–443, 2008.
H. Cobb, “An investigation into the use of hypermutation as an adaptive operator in genetic algorithms having continuous, time-dependent nonstationary environments,” Naval Research Laboratory, Tech. Rep., 1990.
Departamento de Engenharia de Computação e Automação
Industrial
Faculdade de Engenharia Elétrica e de Computação
Unicamp
Online learning in estimation of distribution algorithms for dynamic environments
André Ricardo GonçalvesFernando J. Von Zuben