parallel simulated annealing with adaptive neighborhood determined by ga

2003 IEEE International Conference on Systems, Man & Cybernetics

Doshisha University, Kyoto, Japan

Parallel Simulated Annealing with Adaptive Neighborhood determined by GA

　 Mitsunori MIKI　 Tomoyuki HIROYASU○ Toshihiko FUSHIMI


2003.10.06

Introduction Optimization problems become more complicated and larger.

•Simulated Annealing (SA)based on the simulation of the physical process “annealing”.

Important mattersImportant matters

1. Parallelization2. Adaptive parameter tuning

Heuristic search

•GA, CA, NN etc.


2003.10.06

Algorithm of Simulated Annealing

AlgorithmAlgorithm

Ene

rgy

low

1. Generation

2. Judge Transition

3. Cooling

Design spaceDesign space

high

( E = Enext - Enow)⊿

good acceptancegood acceptance

bad acceptancebad acceptance

1

Exp( 　　　 )- E⊿Temperature

Metropolis probability


2003.10.06

Neighborhood range

•Can’t search optimum effectively.

•Often trapped in a local minimum.

Too large neighborhood range

Too small neighborhood range

The neighborhood range in the continuous Euclid space is the extent for generating next solution.

Global optimumGlobal optimum

•The range has to be small.

•The range has to be large.


2003.10.06

Background

For the control of the neighborhood range, some method are proposed.

These methods control the neighborhood range using an appropriate acceptance ratio.

• The adaptive neighborhood mechanism. [Corana 1987]• The advanced adaptive neighborhood mechanism. [Miki 2002]

This type of adaptive neighborhood method is very effective and useful, but the target acceptance ratio should be determined experimentally.

Propose a new adaptive neighborhood mechanismPropose a new adaptive neighborhood mechanism


2003.10.06

Purpose

Parallel Simulated Annealing with Adaptive NeighborhoodParallel Simulated Annealing with Adaptive Neighborhooddetermined by Genetic Algorithm (PSA/ANGA)determined by Genetic Algorithm (PSA/ANGA)

Controlling the neighborhood range adaptively during the search.

•This method is Parallel model.•This method parallels neighborhood ranges on each processes.•This neighborhood range is controlled by GA.

CharacteristicsCharacteristics


2003.10.06

Effect of Neighborhood RangesN

eig

hb

orh

oo

d ra

nge

large

small

Fixed neighborhood rangeFixed neighborhood range Search spaceSearch space

•The neighborhood range has a significant effect on the accuracy of the solution.•In order to verify this effect, some numerical experiments were carried out with various fixed neighborhood ranges.

Compare the qualities of the solutions.

Obtain the effect of the neighborhood ranges.


2003.10.06

Test problems

Rastrigin Griewangk Rosenbrock

MathematicalMathematicaltest functionstest functions

Griewangk function

Rastrigin function

Rosenbrock function


2003.10.06

Appropriate neighborhood range

AppropriateAppropriateneighborhood rangeneighborhood range

Rastrigin

The neighborhood range has a significant effect on the performance of SA.

Good solutionGood solution


2003.10.06



Griewangk




2003.10.06



Rosenbrock




2003.10.06

Concept of PSA/ANGA

The neighborhood range determined adaptively by GA.

PSA searches the solution with various neighborhood range.

•The appropriate neighborhood ranges depend on problems.•It is difficult to find the appropriate neighborhood ranges in advance.

There are the appropriate neighborhood ranges in SA when solving the continuous optimization problems.

Parallel Simulated Annealing with Adaptive NeighborhoodParallel Simulated Annealing with Adaptive Neighborhooddetermined by Genetic Algorithm (PSA/ANGA)determined by Genetic Algorithm (PSA/ANGA)


2003.10.06

Algorithms of PSA/ANGAN

eig

hb

orh

oo

d ra

nge

large

small

Multiple SA processes searches the solution with various neighborhood range.

GA operators are applied on neighborhood ranges.Fitness =

1

Energy


2003.10.06

Abstract of numerical experiments

Parallel SA with Fixed Neighborhood (PSA/FN)Parallel SA with Fixed Neighborhood (PSA/FN)

Optimum fixed neighborhood range

Comparative methodComparative method

Use the optimum fixed neighborhood range determined by preliminary numerical experiments.

PSA/ANGA is compared with a parallel SA with optimum fixed neighborhood range, PSA/FN.


2003.10.06

Parameters usedFunctions Rastrigin Griewangk Rosenbrock

Max temperature 10 20 1

Min temperature 0.01 0.001 0.001

Markov length 102400 307200 3072

No. of variable 3 3 3

Cooling rate 0.8 0.7 0.8

Optimum fixed neighborhood range

1.0 5.5 0.3

No. of processes 32


2003.10.06

Performance of the proposed method

Proposed method

The proposed method, PSA/ANGA, provides better The proposed method, PSA/ANGA, provides better performance than PSA/FN in all problems.performance than PSA/FN in all problems.


2003.10.06

History of Neighborhood range

Rastrigin

•History of the neighborhood ranges in 32 SA processes.

•The appropriate neighborhood range varies dynamically during the search.

The appropriate neighborhood range is automatically The appropriate neighborhood range is automatically determined using GA. determined using GA.


2003.10.06

History of energy (Rastrigin)

•The proposed method, PSA/ANGA, shows fast convergence of the energy and obtains lower energy than PSA/FN.

•Accuracy of the solution improves because the neighborhood ranges were changed adaptively.


2003.10.06

ConclusionsA new Parallel Simulated Annealing method with adaptive neighborhood range mechanism is proposed.

Parallel SA with Adaptive NeighborhoodParallel SA with Adaptive Neighborhood determined by Genetic Algorithm (PSA/ANGA)determined by Genetic Algorithm (PSA/ANGA)

PSA/ANGA shows good performance on the some test functions.

The appropriate neighborhood range varies according to the condition of the search.

The proposed method adapts to these appropriate The proposed method adapts to these appropriate neighborhood ranges.neighborhood ranges.

The method is effective in SA for continuous optimization The method is effective in SA for continuous optimization problems.problems.


2003.10.06

questions and answers

Thank you for your kind attention.

parallel simulated annealing with adaptive neighborhood determined by ga

Documents