![Page 1: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/1.jpg)
Regrouping Particle Swarm Optimization: A New Global Optimization Algorithm with
Improved Performance Consistency Across Benchmarks
George I. EversAdvisor: Dr. Mounir Ben Ghalia
Electrical Engineering DepartmentThe University of Texas – Pan American
1
![Page 2: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/2.jpg)
OutlineI. From Physics to PSOII. Visual Illustration of Stagnation
& the Regrouping MethodIII. RegPSO FormulationIV. Graph of Solution QualityV. Statistical Comparison with Basic PSOVI. SummaryVII. Future Work
2
![Page 3: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/3.jpg)
How PSO Derives from Standard Physics Equations
I. From Physics to PSO
3
![Page 4: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/4.jpg)
From Physics to PSO
Displacement Formula of Physics:
20 0
12
x x v t at
assuming constant acceleration over the time period
4
![Page 5: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/5.jpg)
From Physics to PSO
Iterative Version:
Using 1 time unit between iterations:• t = (k + 1) – k = 1 iteration per update• t2 = 1 iteration2 per update• For practical purposes, t drops out of theequation.
5
1( 1) ( ) ( ) ( )2
x k x k v k a k
![Page 6: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/6.jpg)
From Physics to PSO
Subscript “i” Used for Particle Index:
(All particles follow the same rule.)
6
1( 1) ( ) ( ) ( ).2i i i ix k x k v k a k
![Page 7: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/7.jpg)
From Physics to PSO
Particles are physical conceptualizations accelerating according to social andcognitive influences.
7
![Page 8: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/8.jpg)
From Physics to PSO
Cognitive AccelerationThe cognitive acceleration is proportional to(i) the distance, , of a particle from its personal best, and (ii) the cognitive acceleration coefficient, .
8
( ) ( )i ip k x k
1c
![Page 9: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/9.jpg)
From Physics to PSO
Social AccelerationThe social acceleration is proportional to(i) the distance, , of a particle from its global best, and (ii) the social acceleration coefficient, .
9
( ) ( )ig k x k
2c
![Page 10: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/10.jpg)
From Physics to PSO
Total AccelerationThe overall acceleration can therefore be
written as
Substitution then leads from
to
10
1 2( ) c ( ) ( ) c ( ) ( ) .i i i ia k p k x k g k x k
1 21 1( 1) ( ) ( ) c ( ) ( ) c ( ) ( ) .2 2i i i i i ix k x k v k p k x k g k x k
1( 1) ( ) ( ) ( )2i i i ix k x k v k a k
![Page 11: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/11.jpg)
From Physics to PSO
Total AccelerationIn place of constant , a pseudo-random
number with an expected value of is generated per dimension to add anelement of stochasm to the algorithm.
In this manner
becomes
11
1 1 2 2( 1) ( ) ( ) c ( ) ( ) c ( ) ( ) .i i i i i i i ix k x k v k r p k x k r g k x k
1 21 1( 1) ( ) ( ) c ( ) ( ) c ( ) ( )2 2i i i i i ix k x k v k p k x k g k x k
12 1
2
![Page 12: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/12.jpg)
From Physics to PSO
Simulating FrictionTo prevent velocities from growing out of control, only
a fraction of the velocity is carried over to the next iteration. This is accomplished by introducing an inertia weight, , which is set less than 1.
In this manner
becomes
12 1 1 2 2( 1) ( ) ( ) c ( ) ( ) c ( ) ( ) .i i i i i i i ix k x k v k r p k x k r g k x k
1 21 1( 1) ( ) ( ) c ( ) ( ) c ( ) ( )2 2i i i i i ix k x k v k p k x k g k x k
![Page 13: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/13.jpg)
From Physics to PSO
Velocity and Position Updates
The previous equation is separated into two more succinct equations, allowing velocities and positions to be recorded and analyzed separately.
1 1 2 2( 1) ( ) c ( ) ( ) ( ) c ( ) ( ) ( )
( 1) ( ) ( 1).
i ii i i i i
i i i
v k v k r k p k x k r k g k x k
x k x k v k
Velocity UpdateEquation
PositionUpdateEquation
13
![Page 14: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/14.jpg)
The Main Obstacle: Premature Convergence/
Stagnation
II. Visual Example of Stagnation& The Regrouping Method
14
![Page 15: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/15.jpg)
Rastrigin BenchmarkUsed to Illustrate Stagnation
15
![Page 16: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/16.jpg)
Swarm Initialization
Particles 1 and 3 are selected to visually
illustrate how velocities and positions are
updated.
16
![Page 17: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/17.jpg)
First Velocity Updates
17
![Page 18: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/18.jpg)
First Position Updates
Particle 1 found a new personal best, but particle 3 did
not.
18
![Page 19: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/19.jpg)
Second Velocity Updates
19
![Page 20: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/20.jpg)
Second Position Updates
Particle 3 found a new personal best, while particle 1 did
not.
20
![Page 21: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/21.jpg)
Swarm Snapshots
Having seen how particles iteratively update their positions, the following slides show the swarm state each 10 iterations to track the progression from initialization to eventual solution.
21
![Page 22: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/22.jpg)
Swarm Initialization at Iteration 0
Particles are randomly initialized within the original initialization space.
22
![Page 23: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/23.jpg)
Swarm Collapsing at Iteration 10
Particles are converging to a local minimizer near [2,0] via their attraction to the global best in that vicinity.
23
![Page 24: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/24.jpg)
Exploratory Momenta at Iteration 20
Momenta and cognitive accelerations keep particles searching prior to settling down.
24
![Page 25: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/25.jpg)
Convergence in Progressat Iteration 30
Personal bests move closer to the global best and momenta wane as no better global best is found. Particles continue converging to the local minimizer near [2,0].
25
![Page 26: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/26.jpg)
Momenta Waning at Iteration 40
Momenta continue to wane as particles are repeatedly pulled toward (a) the global best very near [2,0] and (b) their own personal bests in the same vicinity.
26
![Page 27: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/27.jpg)
Mostly Converged at Iteration 50
Most particles are improving their approximation of the local minimizer found, while two particles still have some momenta.
27
![Page 28: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/28.jpg)
Momenta Waning at Iteration 60
The final two particles are collapsing upon the global best while the remaining particles are refining the solution.
28
![Page 29: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/29.jpg)
Momenta Waning at Iteration 70
All particles are in the same general vicinity.
29
![Page 30: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/30.jpg)
Cognitive Acceleration at Iteration 80
At least one particle still has some exploratory momentum.
30
![Page 31: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/31.jpg)
Premature Convergence Detectedat Iteration 102
All particles have converged to within 0.011% of the diameter of the initialization space. It is important to allow particles to refine each solution before regrouping since they have no prior knowledge of which solution is the global minimizer.
31
![Page 32: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/32.jpg)
Options for Dealing with Stagnation
• Terminate the search rather than wasting computations while stagnated.
• Allow the search to continue and hope for solution refinement.
• Restart particles from new positions and look for a better solution.
• Somehow flag solutions already found so that each restart finds new solutions, and continue restarting until no better solutions are found.
• Reinvigorate the swarm with diversity to continue the current search for the global minimizer.
32
![Page 33: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/33.jpg)
“Regrouping” Definition
Regroup: “to reorganize (as after a setback)for renewed activity”
– Merriam Webster’s online dictionary
33
![Page 34: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/34.jpg)
Regrouping at Iteration 103
Regrouping is more efficient than restarting on the original initialization space.
34
![Page 35: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/35.jpg)
Exploration at Iteration 113
“Gbest” PSO continues as usual within the new regrouping space. Particles move toward the global best with new momenta, personal bests, and positions/perspectives.
35
![Page 36: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/36.jpg)
Swarm Migration at Iteration 123
The swarm is migrating toward a better region discovered by an exploring particle near [1,0].
36
![Page 37: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/37.jpg)
Differences of Opinion at Iteration 133
Some particles are refining a local minimizer near [1,0] while others continue exploring in the vicinity.
37
![Page 38: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/38.jpg)
Solution Comparison at Iteration 143
Cognition pulls some particles back to the local well containing a local minimizer near [1, 0].
38
![Page 39: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/39.jpg)
Solution Comparison at Iteration 153
Cognition and momenta keep particles moving as momenta wane.
39
![Page 40: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/40.jpg)
Unconvinced of Optimality on Horizontal Dimension
at Iteration 163
There is still some uncertainty on the horizontal dimension.
40
![Page 41: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/41.jpg)
New Well Agreed Uponat Iteration 173
All particles agree that the new well is better than the previous.
41
![Page 42: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/42.jpg)
Waning Momenta at Iteration 183
Momenta wane.
42
![Page 43: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/43.jpg)
Premature Convergence Detected Again at Iteration 219
Regrouping improved the function value from approximately 4 to approximately 1, and premature convergence is detected again.
43
![Page 44: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/44.jpg)
Swarm Regrouped Againat Iteration 220
The swarm is regrouped a second time.
44
![Page 45: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/45.jpg)
Best Well Foundat Iteration 230
The well containing the global minimizer is discovered.
45
![Page 46: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/46.jpg)
Swarm Migrationat Iteration 240
The swarm migrates to the newly found well.
46
![Page 47: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/47.jpg)
Convergence at Iteration 250
Particles swarm to the newly found well due to its higher quality minimizer.
47
![Page 48: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/48.jpg)
Cognition at Iteration 260
Momenta carry particles beyond the well.
48
![Page 49: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/49.jpg)
Convergence at Iteration 270
Solution refinement of the global minimizer is in progress.
49
![Page 50: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/50.jpg)
Regrouping PSO (RegPSO) Formulation
III. RegPSO Formula
50
![Page 51: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/51.jpg)
Regrouping PSO (RegPSO)Detection of Premature Convergence
1 2
1, ,
, ,...,
( ) max ( ) ( )
r r r rd
r r
ii s
range range range range
diam range
k x k g k
Range of theSearch Space
Diameter of theSearch Space
Maximum Euclidean Distance from Global Best
TerminateWh
norm( )( )r
kdiam
en Maximum Distance from Global Best is Less Thana User - Specified Percentage of the Diameter of the Search Space
representsthe search space forregrouping index .
r
r
51
![Page 52: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/52.jpg)
Regrouping PSO (RegPSO)Regrouping the Swarm
,1, ,
0
1 2
max
( ) min ( ),
, ,...,
1
j i j ji s
rj j j
r r r rd
i i
x k g k
range range
range range range range
x k g k r ra
Uncertainty per Dimension
Range of New Search Space
New Search Space Centered at Global Best
1 2
1( ) ( )2
where , ,...,
with each (0,1) randomly selected.i i i
i
r r
i d
j
nge range
r r r r
r U
52
![Page 53: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/53.jpg)
Regrouping PSO (RegPSO)High-Level Pseudo Code
DoRun Gbest PSO until premature convergence.Regroup the swarm.Re-calculate the velocity clamping value based on
the range of the new initialization space.Re-initialize velocities.Re-initialize personal bests.Remember the global best.
Until Search Termination
53
![Page 54: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/54.jpg)
Effectiveness of RegPSO Demonstrated Graphically
IV. Graphical Comparison of Mean Function Values
54
![Page 55: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/55.jpg)
Mean Behavior on 30D Rastrigin A swarm size of 20 suffices for RegPSO to approximate the global minimizer of the 30-D Rastrigin and reduce the cost function to approximately true minimum across all 50 trials.
55
![Page 56: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/56.jpg)
Effectiveness of RegPSO Demonstrated Statistically
V. Statistical Comparison
56
![Page 57: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/57.jpg)
Regrouping PSO (RegPSO)Compared to Gbest, Lbest PSO
RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size 2 s = 20, c1 = c2 = 1.49618, 50 trial sets, 800,000 function evaluations RegPSO used 4 11.1 10 ; 1.2 ; 100,000 evaluations max per grouping. Benchmark d Gbest PSO
0.5,0.72984
Gbest PSO 0.15,0.9 to 0.4
Lbest PSO 0.5,0.72984
Lbest PSO 0.15,0.9 to 0.4
RegPSO 0.5,0.72984
Ackley 30 Mean:
3.6524 1.1191e-014
0.046206 1.0623e-014 5.2345e-007
Griewangk 30 Mean:
0.055008 0.022023 9.1051e-003 0.012538 0.013861
Quadric 30 Mean:
4.1822e-75 2.3189e-014 3.4340e-012 5.9577e-022 3.1351e-010
Quartic with noise
30 Mean:
0.0039438 0.0015241 1.2630e-002 0.0025417 0.00064366
Rastrigin 30 Mean:
71.63686 25.252 52.812 31.2746 2.6824e-011
Rosenbrock 30 Mean:
2.06915 18.859 2.6106 1.0713 0.0039351
Schaffer’s f6 2 Mean:
0.0033034 0 1.2025e-003 0 0
Sphere 30 Mean:
2.4703e-323 1.0834e-094 2.0146e-160 2.1967e-215 9.2696e-015
Weighted Sphere 30 Mean:
1.0869e-321 4.4182e-093 6.5519e-158 1.2102e-225 9.8177e-014
57
![Page 58: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/58.jpg)
Summary
By regrouping the swarm within an efficiently sized regrouping space when premature convergence is detected, RegPSO considerably improves performance consistency, as demonstrated with a suite of popular benchmarks.
58
![Page 59: Regrouping Particle Swarm Optimization: A New … Algorithm with Improved Performance Consistency Across Benchmarks ... RegPSO Compared to Gbest PSO & Lbest PSO of neighborhood size](https://reader031.vdocuments.site/reader031/viewer/2022030422/5aaa36c87f8b9a95188dd215/html5/thumbnails/59.jpg)
Future Work
Theoretical Improvements• Give the algorithm the ability to progress from regrouping to a solution refinement phase.Testing• NP hard problems• Applications to real-world problems
59