Lecture 2. Co-Evolution (II)

학습목표

공진화와 관련된 다양한 방법론을 이해하고 , 응용예를 통한 실제 적용가능성을 점검한다 .

Outline

Review of the last lecture

Diploid gene representation

Parallel evolutionary algorithms (fine-grained): Local selection, Recombination

Different types of co-evolution

Inter-population co-evolution: An example in design, Knowledge discovery (data mining), Interactive evolution

Intra-population co-evolution: Co-evolving a backgammon player, Taking two interwound spirals apart by co-evolution, Iterated prisoner’s dilemma

Summary and overviews of other related work

Why co-evolutionary learning

More examples and open issues

Different Types of Co-Evolution

Based on the number of population involved:

Inter-population co-evolution

Two or more populations

Intra-population co-evolution

Within a single population

Based on the relationship among individuals

Competitive co-evolution

Individuals compete for higher fitness to solve a problem (often a dynamic problem)

Cooperative co-evolution

Individuals cooperate with other in order to solve a problem

Co-Evolution in Design (1)

Many design problems do not have a fixed goal or a fixed set of specifications

People do change minds

CurrentProblem

k

CurrentProblem

k+1

CurrentSolution

k

CurrentSolution

k+1

…evolve

providefitness

providefitness

providefitness

…evolve

…

…

Co-Evolution in Design (2)

Task: floor plan design

A candidate solution

Adjacency graph (representing requirements)

kitchendiningroom

bed1

bed2bed3lounge hall

corridorwc

ensuite

kitchen corridor

dining

lounge

wc bed1 ensuite

hall bed3bed2

Co-Evolution in Design (3)

Representation

Solution space

Problem (specification) space

genotype phenotype

(pen movement) (floor plan)

1 2 3 4 5 45mapping mapping

123

4 5

genotype

A B C D E BC CD

(adjacency)

mappingA

B

CD

E

A B E

C D

Co-Evolution in Design (4)Fitness evaluation

Solution space / population

Fitness = (Basic initial requirements) + (current best problem)

Problem space: Fitness = (current best solution)

What does “best” mean in this case?Answer: How well an individual matches a floor plan (or adjacency graph)

solutions(floor plans)

best

best

problems(adjacency graphs)

providefitness

providefitness

Knowledge Discovery (Data Mining)One application: fraud detection

We want to discover all kinds of frauds, but we do not really know what they are (what kind of patterns they have)

In general, we want to find something interesting without knowing what “interesting” actually means

corporatedatabase rules

evolving

top performingrules

newinterestingness

analysis human ranking

Interestingness is co-evolving with interesting rules

Interactive Evolution

Evolution with a human being in the loop

Often used in creative design or creative problem solving

May be time-consuming

Case1: no co-evolution

Case 2: co-evolution

population

replacement fitness evaluationby human

genetic operation

computerprograms

humanbeings

fitnessevaluation

Intra-Population Co-Evolution

There is only one population

However, fitness of one individual depends on other individuals in the population

Example 1: Playing backgammon

TD-Gammon

a grand master level computer program based on NN

It learned by self-playing

Self-playing = Co-evolution

Is it because machine learning algorithms or co-evolution?

Co-Evolving Backgammon Players (1)

Simple neural network without any fancy learning algorithm except for hill-climbing

Simple EA with population size 1, hardly an EA!

Simple mutation with Gaussian noise

No recombination at all

Task = evolve an NN that plays backgammon

Co-Evolving Backgammon Players (2)

Task = evolve an NN that plays backgammon

197-20-1 feed-forward fully connected NN

Initial weights were 0’s

1. Let the initial NN be NNk, k 0

2. Generate a mutant challenger of NNk

w’ij = wij + G(0, )

3. If NN’k is beaten by NNk, NNk+1 = NNk

Else NNk+1 = NNk*0.95 + 0.05*NN’k

4. k k+1, goto step 2

Performance: Winning 40% of the games against PUBEVAL after 100,000 generations

strong program trained by experts

Separating Interwound Two Spirals

Task: Given 194 training points, learn to separate two spirals

Very tough problem for machine learning algorithms, e.g., decision trees, neural networks, …

There was an attempt to evolve a solution to it. But the solution generalized poorly

Competitive co-evolution based on covering really helps

Fitness evaluation without co-evolution: number of test cases correctly classified

Fitness evaluation with co-evolution: based on pair-wise competition, it depends only on the number of test cases correctly classified but NOT covered by its opponent

Iterated Prisoner’s DilemmaNon-zero sum, non-cooperative games

The 2 player version

The purpose here is not to find the optimal solution for some simplified conditions, but to study how to find it

Fitness evaluation

Entirely determined by the total payoff obtained through playing against each other

The initial population was generated at random

Player A

Player B

C D

C

D

33

11

0

0

5

5

Why Co-Evolution

We do not know the fitness function

There are too many cases to test in order to obtain a fitness value. Co-evolution can be used to FOCUS search in the most important area

The problem is inherently changing in time

Increase and maintain diversity

Self-learning

More Examples of Co-Evolution

Discovering CA rules (using coverage again)

Computer-aided learning (students + software tutors)

Robot morphology and control

Character recognition

Chess playing

International coffee market prices

…

Open IssueForgetting (also known as the Red Queen effect): co-evolution does not have a good memory at present

Mediocre stable state: individuals learn to co-exist with each other and do not want to explore the search space any more

Incremental evolution: continuous improvement without forgetting

References

J. Poon and M.L. Maher, “Emergent behavior in co-evolutionary design,” Artificial Intelligence in Design’96, J. Gero (ed.), Kluwer Academic.

J.B. Pollack, A.D. Blair & M. Lund, “Coevolution of a backgammon player,” Proc. Of the Fifth Alife, May 1996.

H. Juille and J.B. Pollack, “Dynamics of co-evolutionary learning,” Proc. Of the 4th Int. Conf. on Simulation of Adaptive Behaviro, Sept. 1996, MIT Press, pp. 526~534

Homework #1

주제 : Diversity 유지를 위한 speciation 방법 구현 및 실험

마감일 : 9/30

내용 :

5 가지 평가함수 (De Jong function 1~5) 에 2 가지 종분화 방법 (Fitness sharing, Crowding) 을 적용하여 진화동안의 diversity 변화를 조사한다

Bonus

Sorting algorithm/problem 에 공진화와 종분화를 이용하기

Othello 게임에 공진화와 종분화를 이용하기