Introduction to Genetic Algorithms

2

Abstract

An introduction to emulating the problem solving according to nature's method: via evolution, selective breeding, "survival of the fittest.” We will present the fundamental algorithms and present several examples, especially some problems that are hard to solve otherwise.

3

Dealing with Hard (NP-Complete) Problems

Some Problems We Just Don't Know How to Solve!. . . but we do know how to critique a “solution”.

Coloring graphs is hard, but counting colors and violations is easy

(a violation is two adjacent vertices with the same color).

Finding the shortest salesman's path is hard, but measuring a path is easy.

Scheduling examinations or assigning teachers to classes is hard, but

counting the conflicts (ideally there are none) is easy.

Computer programs are hard to write, but counting bugs is easy.

4

GAs Emulate Selective Breeding

Designing tender chickens is hard; taste-testing them is easy.

Designing thick-skinned tomatoes is hard; dropping is easy.

So, the breeders iterate:• Selection: Cull their population of the inferior

members.• Crossover: Let the better members breed.• Mutation: X-ray them.

5

Darwin’s Theory of Evolution

During reproduction, traits found in parents are passed on to their offspring

Variations (mutations) are present naturally in all species producing new traits.

A process called natural selection, ‘selects’ individuals best adapted to the environment

Over long periods of time, variations can accumulate and produce new species.

6

Natural Selection

Those fittest survive longestCharacteristics, encoded in genes are transmitted to

offspring and tend to propagate into new generations

In sexual reproduction, the chromosomes of offspring are a mix of their parents

An offspring’s characteristics are partially inherited from parents and partly the result of new genes created during the reproduction process

7

Applications I Have Known

Choosing among 1,500 features for OCR.

Scheduling the Chili, NY, annual soccer invitational.

N Queens, Graphs, Salesmen, etc., etc.

8

Subsetting 1,500 OCR Features

The polynet OCR engine trains and executes rapidly.

Performance was competitive.

We wanted to embed it in hardware, but it used 1,500 features.

We could deal with 300 features.

So, we bred high-performance feature subsets.

9

Soccer Scheduling

Bill Gustafson's MS Project, May, 1998

The Chili Soccer Association hosts an annual soccer tournament.

131 teams, 209 games, 14 fields, 17 game times.

a long weekend for a group of schedulers,

. . . . . and then some teams back out. . .

10

Soccer Scheduling Hard Constraints

A field can have one game at a time.

A team can only play one game at a time.

Teams must play on appropriate size fields.

Late games must be played on lighted fields.

A team must rest one game period (two is better) between games.

Teams can only play when they can be present (some cannot come Friday evening.

11

Soccer Scheduling Soft Constraints

A team's games should be distributed evenly over the playing days.

Teams should play in at most two playing areas.

Each team should play at least once in the main playing area.

Teams should play in areas where they have a preference.

Games should finish as early as possible on Sunday.

Etc...

12

Placing N Non-Attacking Queens

Found by my genetic algorithm!

13

Placing N Non-Attacking Queens

Queens attack on chess-board rows, columns, and diagonals.

Any permutation in N rows avoids row & column attacks.

Exhaustive search works for N 10, but N! grows rapidly.

A GA can place 1,000 Queens in 1,344 fitness evaluations.

(This is not an NP-complete problem.)

14

100 Non-Attacking Queens in 130 Fitness Evaluations

15

Graph Coloring: Edge Ends Get Different Colors

16

Graph Coloring = Map Coloring

17

Graph Coloring = Map Coloring

18

Graph Coloring = Map Coloring

19

Traveling Salesman Route Min.

Another classic, hard, NP-complete problem.

We tried cities on a HW grid, so best distance is known.

Perfection is hard to achieve.

A clever algorithm costs O(cities2) to evaluate a fittness.

But, we get pretty good answers.

20

Some GA Application Types

Domain Application Types

Control gas pipeline, pole balancing, missile evasion, pursuit

Design semiconductor layout, aircraft design, keyboardconfiguration, communication networks

Scheduling manufacturing, facility scheduling, resource allocation

Robotics trajectory planning

Machine Learning designing neural networks, improving classificationalgorithms, classifier systems

Signal Processing filter design

Game Playing poker, checkers, prisoner’s dilemma

CombinatorialOptimization

set covering, travelling salesman, routing, bin packing,graph colouring and partitioning

21

History

HollandBagleyCavicchioHollstienDe JongGoldbergMedical image processingPrisoner’s dilemmaHollandOthers

22

Classes of Search Techniques

F inonacci N ew ton

D irect methods Indirect methods

C alculus-based techniques

E volutionary strategies

C en tralized D istrib u ted

P arallel

S tead y-state G en eration al

S eq u en tia l

G en etic a lgorith m s

E volu tion ary a lgorith m s S imulated annealing

G u id ed ran d om search tech n iq u es

D ynamic programming

E numerative techniques

S earch tech n iq u es

23

Comments

Stochastic algorithmrandomness has an essential role in genetic algorithmsboth selection and reproduction needs random

proceduresConsider population of solutions

evaluates more than a single solution at each iterationassortment, amenable for parallelisation

RobustnessAbility to perform consistently well on a broad range of

problem typesno particular requirements on the problems before using

GAs

24

Benefits of Genetic Algorithms

Concept is easy to understand

Modular, separate from application

Supports multi-objective optimization

Good for “noisy” environments

Always an answer; answer gets better with time

Inherently parallel; easily distributed

25

Benefits of Genetic Algorithms (cont.)

Many ways to speed up and improve a GA-based application as knowledge about problem domain is gained

Easy to exploit previous or alternate solutions

Flexible building blocks for hybrid applications

Substantial history and range of use

26

Uses of GAs

GAs (and SAs): the algorithms of despair. Use a GA when

you have no idea how to reasonably solve a problem

calculus doesn't apply

generation of all solutions is impractical

but, you can evaluate posed solutions

27

Outline of This GA Course

The basic frame of genetic algorithm.

More about representation and operators

Theory analysis of GA

Combinatorial Problem

Constraint optimization

28

Outline of This GA Course

Genetic algorithm and Artificial neural network

Parallel GA

Genetic programming

29

Famous Problems & Concepts

N Queens

Traveling salesman

Knight's tour

Bin packing

Scheduling

Function optimization

Graph coloring, Ramsey problems

Satisfiability