Topological Crossover for the Permutation Representation

Alberto Moraglio & Riccardo Poli{amoragn,rpoli}@essex.ac.uk

GECCO 2005

Topological Crossover

Abstract Geometric Crossover

Sorry… Name Change!

ContentsI. Abstract Geometric Operators

II. Geometric Crossover for Permutations

III. Geometric Crossover for TSP

IV. Conclusions

I. Abstract Geometric Operators

What is crossover?

CrossoverIs there anycommon

aspect ?

Is it possible to give arepresentation-

independent definitionof crossover and

mutation?

100000011101000

100111100011100

100110011101000

100001100011100

Binary StringsPermutations

Real Vectors

Syntactic Trees

Shortest Path Crossover011001

010001 011101 011011

010101 011111

010011

010111

D0 : P1

D2 : P2

D1

Parent1: 011101

Parent2: 010111

Children: 01*1*1

Crossover in the Neighbourhood: offspring between parents

Mask-based crossover: children are on shortest paths

Hamming Neighbourhood Structure

From graphs to geometry• Neighbourhood Structure=Metric Space • The distance in the neighbourhood is the

length of the shortest path connecting two solutions

• Mutation Direct neighbourhood Ball• Crossover All shortest paths Line

Segment

Balls & SegmentsIn a metric space (S, d) the closed ball is the set of the form

where x belongs to S and r is a positive real number called the radius of the ball.

In a metric space (S, d) the line segment or closed interval is the set of the form

where x and y belong to S and are called extremes of the segment and identify the segment.

}),(|{);( ryxdSyrxB

)},(),(),(|{];[ yxdyzdzxdSzyx

Squared balls & Chunky segments

33

000 001

010 011

100 101

111

110

B(000; 1)Hamming space

3

B((3, 3); 1)Euclidean space

3

B((3, 3); 1)Manhattan space

Balls

1

2

1

2

000 001

010 011

100 101

111

110

[000; 011] = [001; 010]2 geodesics

Hamming space

1 3

[(1, 1); (3, 2)]1 geodesic

Euclidean space

1 3

[(1, 1); (3, 2)] = [(1, 2); (3, 1)]infinitely many geodesics

Manhattan space

Line segments

Uniform Mutation & Uniform Crossover

Uniform topological crossover:

Uniform topological ε-mutation:

|],[|]),[(}2,1|Pr{),|(

yxyxzyPxPzUXyxzfUX

],[}0),|(|{)],(Im[ yxyxzfSzyxUX UX

|),(|)),((}|Pr{)|(

xBxBzxPzUMxzfUM

),(}0)|(|{)](Im[ xBxzfSzxUM M

Genetic operators have a geometric nature

Representation-independentand rigorous definition of

crossover and mutation in the neighbourhood seen as a

geometric space

So what? Claims at Gecco 2004

(i) EAs Unification: most pre-existing genetic operators for main representations are geometric

(ii) Simplification & Clarification: crossover as function of classical neighbourhood structure simplifies the established notion of crossover landscape (hyper-neighbourhood) as function of crossover

(iii) General theory: formal representation-independent definitions allow for a general theory

(iv) Crossover principled design: specifying the formal definition of crossover for a specific representation and distance one gets automatically a specific crossover

II. Geometric Crossover

for Permutations

Many Distances Dilemma

Many Distances Dilemma

WHAT IS A GOOD DISTANCE? WHAT IS THE RIGTH CROSSOVER?

Representation Binary Strings Permutations

Distance One distance =Hamming distance Many distances

Geometric Crossover Mask-based crossover Many types of crossover

Geometric Uniform Crossover Uniform crossover Many uniform crossovers

What is a good distance?– IN PRINCIPLE: abstract genetic operators are well-

defined for any distance. However:– IMPLEMENTATION: a distance not rooted in the

solution syntax does not tell how to implement crossover– PROBLEM KNOWLEDGE: a problem-independent

distance does not put any problem knowledge in the search

– A GOOD DISTANCE: – (i) suggests how to implement crossover– (ii) embeds problem knowledge in the algorithm

Crossover Implementation & Edit

Distances

Mutations/Edit moves for Permutations

• Reversal: (A B C D E F) (A E D C B F)

• Insert: (A B C D E F) (A C D E B F)

• Swap: (A B C D E F) (A D C B E F)

• Adj.Swap: (A B C D E F) (A C B D E F)

Edit Distance = minimum number of edit moves to transform one permutation into the other

Permutation+Edit Move = Neighbourhood Structure

Shortest path distance = edit distance

abc

bac acb

bca cab

cba

B(abc; 1)Adjacent swap space

abc

bac acb

bca cab

cba

[abc; bca]1 geodesic

Adjacent swap space

B(abc; 1)Swap space & Reversal space

abc

bac acb

bca cab

cba

abc

bac acb

bca cab

cba

[abc; bca]3 geodesics

Swap space & Reversal space

B(abc; 1)Insertion space

[abc; bca]1 geodesic

Insertion space

abc

bac acb

bca cab

cba

abc

bac acb

bca cab

cba

Line segment in the neighbourhood structure = all shortest paths connecting two nodes

Neighbourhood/syntax duality

• NEIGHBOURHOOD: Picking offspring on shortest path connecting two nodes

• SYNTAX: picking offspring on minimal

sorting trajectory between parent permutations using the edit move as sort move (minimal sorting by x)

Many sorting algorithms do minimal sorting by X

Ordinary Sorting Algorithm

Minimal Sorting by X

Bubble Sort Adj. Swap

Insertion Sort Insert

Selection Sort Swap

Quick Sort No Fix Move!

Geometric Crossovers = Sorting Crossovers!

• Sorting Crossover by X:– sorting one parent permutation toward the other

using X sort move– stop the sorting at random and return the

partially sorted permutation as offspring• Bubble Sort Crossover = Geometric

Crossover under adj. swap edit distance

EmbeddingProblem Knowledge

Edit Distances & Problem Knowledge

How can we pick an edit distance that embeds problem knowledge?

• Minimal fitness change: pick the edit distance whose edit move corresponds to a minimal fitness change

• Good mutation, Good crossover: pick the edit distance whose edit move corresponds to a good mutation for the problem at hand

• Good neighbourhood, Good crossover: pick the edit distance whose edit move induces a neighbourhood structure that is known to be good for the problem

N-queens - mutations

0

5

10

15

20

25

30

35

40

45

1 40 79 118 157 196 235 274 313 352 391 430 469

swpadj_swpins

N-queens - crossovers

0

5

10

15

20

25

30

35

40

45

1 37 73 109 145 181 217 253 289 325 361 397 433 469

pmxss1xssuxbs1xbsuxis1xisux

Crossover Rank vs. Mutation Rank

1. Selection Sort Uniform 1. Swap2. PMX -3. Selection Sort 1-point 1. Swap4. Insertion Sort Uniform 2. Insertion5. Insertion Sort 1-point 2. Insertion

6. Bubble Sort Uniform 3. Adj. Swap

7. Bubble Sort 1-point 3. Adj. Swap

Good mutation, good crossover heuristic holds!

Uniform crossovers are better than 1-point crossovers

III. Geometric Crossover

for TSP

Geometric Crossover for TSP

• A good neighbourhood structure for TSP is 2opt structure = space of circular permutations endowed with reversal edit distance

• Geometric crossover for TSP =picking offspring on the minimal sorting trajectories by sorting one parent circular permutation toward the other parent by reversals (sorting circular permutations by reversals)

Approximated Geometric Crossover

• BAD NEWS: sorting circular permutations by reversals is NP-Hard!

• GOOD NEWS: there are approximation algorithms that sort within a bounded error to optimality (used in genetics)

• A 2-approximation algorithm sorts by reversals using sorting trajectories that are at most twice the length of the minimal sorting trajectories

• Approximation algorithms can be used to build approximated geometric crossovers for TSP

Experiments - ParametersTest-bed• TSPLIB: eil51, gr96, eil101, lin105, d198, kroA200, lin318, pcb442Crossovers• PMX: partially matched crossover• ERX: edge recombination• SBRX: sorting by reversal crossover (limitations: no circular permutation,

uniform on one fixed geodesic, 2-approxiamtion) Parameter Setting• BIG POPULATION: Population Size = Instance Size * 20• Until Population Convergence• No Mutation• Runs=30 (average of bests in population)• No Fine Tuning. The settings have been chosen to allow the best crossover to

reach a near optimal solution before convergence.

Results for eil51 (small)

0

200

400

600

800

1000

1200

1400

1600

1 13 25 37 49 61 73 85 97 109 121 133

PMXERXSBRX

Results for lin105 (medium)

0

20000

40000

60000

80000

100000

120000

1 28 55 82 109 136 163 190 217 244 271 298 325

PMXERXSBRX

Results for kroA200 (medium-big)

0

50000

100000

150000

200000

250000

300000

350000

1 22 43 64 85 106 127 148 169 190 211 232

PMXERXSBRX

Good results & lot of room for improvement

• SBRX better than ERX for bigger instances• good empirical results based only on theoretical

considerations • Possible improvements:

– Fine parameter tuning– Better approximation algorithm– Non-deterministic approx algorithm (uniform

crossover)– Circular Permutations instead of Linear Permutations

IV. Conclusions

ConclusionsPermutations & Many Distances

– Many types of geometric crossovers!– What is a good distance?

Implementation & Edit Distance: – Edit Distances are good– For permutations: geometric crossovers = sorting algorithms!

Problem Knowledge and Edit Move: – Good mutation, good crossover heuristics– For permutations: good mutation, good crossover holds for the N-

queen problem using sorting crossovers Geometric Crossover for TSP

– Sorting circular permutation by reversals (NP-Hard)– 2-approximation algorithm for approximated geometric crossover– Good empirical results based only on theory!

Thank you for your attention… Questions?

N-queens - parametersProblem size 100

Population size 5000

Mutation probability 0.1 (0)

Crossover probability (0) 1

Generation 500

Selection tournament size 5

Statistics Average 30 runs