Using Free Cloud Storage Services For DistributedEvolutionary Algorithms

Maribel García-Arenas,

Juan-J. Merelo,

Antonio M. Mora,

Pedro Castillo

2

Outline

1) Idea and how to test it

2) Dropbox features

3) Putting in practice with Evolutionary Computation

4) File-individuals

5) Island Algorithm

6) Goals

7) Problems

8) Results

3

IDEA

• What do you know about cloud storage services?

• Why not use them for computing?• How can we use all our computers to

make a multicomputer?– Desktop computer– Portable computer– Home computer– Any other computers...

4

How to test the idea• Look for some free storage

services and test them: What are their features and what is the availability for storing, sharing and synchronizing information

• After that, We have selected Dropbox

5

Dropbox TM features

• It is free up to a certain level of use (measured in traffic and usage)

• It is popular, so many people use it, and we may found many volunteers for computation

• It monitors the local filesystem and uploads information asynchronously

• It looks like a local directory

6

Putting in practice with Evolutionary Computation

• What do we need to build Evolutionary Distributed Algorithms?

– Exchange individuals among populations: Phenotype and Genotype

• We can exchange this information using files. So the name of the file represents the phenotype and genotype and all connected PCs share it with Dropbox

7

Let's go

• File distribution via Dropbox • It synchronizes the file-individuals with

other computers• Each computer evolves an island• Dropbox folder contains a pool of

individuals and each computer adds and gets file-individuals from it

8

Let's go (II)

• Each computer connected or synchronized by Dropbox is part of a multi-computer

• Each Island-computer evolves a population of individuals and exchanges with the pool file-individuals when the migration process must be done

9

File-individuals

• How to include phenotype and genotype into a file– As the contents of the file? It is not a good

idea because we have to open and close files and Dropbox has to synchonize them.

– Into the filesystem attributes? Dropbox is working on that and we will be testing in the future

– Into the filename? It is our approach

10

File-individuals (II)

• The filename problem– How many gens can we include into the

name?– We have to code the genotype into base 32– Ex: 00000 → 0, 00001-> 1, 01010->A ...

111111->V

• The filename includes: Fitness, genotypeBase32codification and the id of the computer which generates the individual

11

Island Algorithm1.Creates and evaluates the initial population

2.Until to reach a number of evaluations into the multi-computer

• Breed the population • Evaluate• Generational replacement with 1-elitism • After a fixed number of generations, Immigrate

(gets one file-individual from the pool and incorporates it to the population)

• After a fixed number of generations, Migrate (adds the best or a random file-individual to the pool)

3.Adds the best individual to the pool

12

Control of the number of evaluations

• Each computer creates a file whose name is the number of evaluations performed and its identification (random initial seed)

• Each computer looks for this kind of file within the Dropbox folder and adds the total of evaluations.

• When the sum of this evaluations is greater than the fixed minimum, the evolution of this island ends.

13

Goals

• What do we want to test?– We want test if we save time when use the

multi-computer for computing a fixed number of evaluations.

• How can we test it?– Making a distributed evolutionary algorithm

based on pool and testing that the time for reaching the fixed evaluations decreases when you add new nodes to our multi-computer linked by Dropbox.

14

Problems: MMDP

• Multimodal Deceptive Problem

• It is composed of k (k=80) subproblems of 6 bits each one called s

i for i=0 to 79.

• Depending of the number of ones s

i takes the values

detailed into the table

Fitness individual=∑i=1

k fitness si

ones fitness

0 or 6 1

5 or 1 0

2 or 4 0,360384

3 0,640576

15

Problems: TRAP

• It is defined for the unitation function (number of ones in a binary string) using the following function.

• For our problem, the trap is defined for l=4, a=3, b=4 and z = 3

• With 30 traps

into the genome

trap u x=az z−u x , if u x zbl−z ux−z , otherwise{

16

Parameters• We use as multi-computer one, two or four

heterogeneous computers so we use one, two, three or four island

• Population size: 1000 individuals

• Selection: Tournament

• Crossover: uniform

• Mutation: bit-flit

• Replacement: Generational with 1-elitism

• Stop criteria: minimum number of evaluations for the multi-computer

• WiFi with WPA/Enterprise encryption.

17

Results for MMDP

18

Results for TRAP

19

Conclusions

• The Dropbox File-storage and sharing system, can be used as a migration device for distributed evolutionary computation experiments without needing to acquire or set up complicated cloud or grid infrastructure.

• With this approach everyone can use a multicomputer running an evolutionary algorithm with a good scaling behavior.

20

Others results for MMDP

100 200 400

0

20

40

60

80

100

120

Success Rate

124

Migration frecuency

Per

cent

age

1 2 4

0

50000

100000

150000

200000

250000

300000

Time to find the solution

MMDP Problem

100200400

Islands

Tim

e(m

ilise

cond

s)

21

Questions