9781420035063%2ech36

CHAPMAN: C4754_C036 2005/8/6 14:14 page 649 #1

36Parallel HybridMultiobjective

Metaheuristics onP2P Systems

N. MelabEl-Ghazali TalbiM. MezmazB. Wei

36.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .36-64936.2 Parallel Hybrid MOMs and P2P Computing . . . . . . . . . .36-650

Multiobjective Optimization Parallelism andHybridization P2P Computing for Parallel MOOptimization

36.3 A Model for P2P Coordination . . . . . . . . . . . . . . . . . . . . . . . . .36-652Model Description Implementation on Top of XtremWeb

36.4 Application to BPFSP and Experimentation . . . . . . . . . . .36-655Problem Formulation A GeneticMimetic Algorithm forSolving BPFSP Parallel Hybrid AGMA Deployment andExperimentation Deployment and Fault Tolerance Experimental Results

36.5 Conclusion and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . .36-661References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .36-663

36.1 Introduction

Metaheuristics allow to provide near-optimal solutions of NP-hard complex problems in a reasonable time.They fall into two complementary categories: evolutionary algorithms (EAs) that have a good explorationpower, and local searches (LSs) characterized by better intensification capabilities. The hybridization ofthe two categories permits to improve the effectiveness (quality of provided solutions) and the robustnessof the metaheuristics [11]. Nevertheless, as it is CPU time consuming it is not often fully exploited inpractice. Indeed, experiments with hybrid metaheuristics are often stopped before the convergence isreached. Nowadays, Peer-to-Peer (P2P) computing [8] and grid computing [5] are two powerful waysto achieve high performance on long-running scientific applications. Parallel hybrid metaheuristics usedfor solving real-world multiobjective problems (MOPs) are good challenges for P2P and grid computing.However, to the best of our knowledge no research work has been published on that topic.

36-649

2006 by Taylor & Francis Group, LLC

CHAPMAN: C4754_C036 2005/8/6 14:14 page 650 #2

36-650 Handbook of Bioinspired Algorithms and Applications

In this chapter, we contribute with the first results on parallel hybrid multiobjective metaheuristicson P2P systems. The design and deployment of these optimization methods require a middleware thatallows cooperation between parallel tasks. In addition, the traditional parallel models and hybridizationmechanisms have to be re-thinked and adapted to be scaled up. Moreover, these require to be fault-tolerantto allow long-running problem resolutions. We particularly focus here on the island model and themultistart model.

Recently, few middlewares [1,4,13] allowing to exploit P2P systems have emerged. These middlewaresare well suited for embarrassingly parallel applications such as multi parameter simulations. However,they are limited regarding the parallelism as they do not allow direct cross-peer (or cross-task) commu-nication. Our contribution is to propose a Linda-like [7] coordination model and its implementationon top of XtremWeb [4]. This is a Dispatcher/Worker oriented middleware, in which the Dispatcherdistributes application tasks submitted by clients to volunteer worker peers at their request. In addition,the considered middleware provides fault-tolerance mechanisms that are costly in a highly volatile P2Penvironment. Indeed, a work unit is re-started from scratch each time it fails. Another contribution of thischapter is to deal with the fault-tolerance issue at application level. We propose a check-pointing approachfor the two parallel models quoted above.

To be validated the proposed approaches have been experimented on the Bi-criterion PermutationFlow-Shop Problem (BPFSP) [12]. The problem consists roughly to find a schedule of a set of jobson a set of machines that minimizes the makespan and the total tardiness. Jobs must be scheduled inthe same order on all machines, and each machine cannot be simultaneously assigned to two jobs.In Reference 2, a hybrid MultiObjective Metaheuristic (MOM) has been proposed to solve this problem.In this chapter, we extend this work with two P2P-based fault-tolerant parallel models: the island andmultistart models. Our extended version allows to fully exploit the hybridization and provides clearlybetter results. This constitutes another contribution of this chapter.

This chapter is organized as follows: Section 36.2 presents briefly parallel hybrid multiobjective optimiz-ation (MOO). Section 36.3 highlights the requirements of MOO and describes the proposed coordinationmodel and its implementation on top of XtremWeb. Section 36.4 presents the experimentation of themodel and its implementation through a parallel hybrid metaheuristic applied to the BPFSP, and analyzesthe preliminary experimental results. Finally, Section 36.5 concludes the chapter.

36.2 Parallel Hybrid MOMs and P2P Computing

36.2.1 Multiobjective Optimization

An MOP consists generally in optimizing a vector of nbobj objective functions F(x) = ( f1(x), . . . , fnbobj(x)),where x is an d-dimensional decision vector x = (x1, . . . , xd) from some universe called decision space.The space the objective vector belongs to is called the objective space. F can be defined as a cost functionfrom the decision space to the objective space that evaluates the quality of each solution (x1, . . . , xd) byassigning it an objective vector (y1, . . . , ynbobj), called the fitness.

Unlike single-objective optimization problems, an MOP may have a set of solutions known as the Paretooptimal set rather than a unique optimal solution. The image of this set in the objective space is denoted asPareto front. Graphically, a solution x is Pareto optimal if there is no other solution x such that the pointF(x ) is in the dominance cone of F(x). This dominance cone is the box defined by F(x), its projections

36.2.2 Parallelism and Hybridization

In Reference 3, different parallel models have been distinguished: those associated with LSs and thosededicated to EAs. Three major parallel models for EAs are presented: the island (a)synchronous cooperativemodel, the parallel evaluation of the population, and the distributed evaluation of a single solution.The parallel models for LSs are mainly: the parallel exploration of neighboring candidate solutions and


on the axes and the origin (Figure 36.1).Dow

nloa

ded

by [I

ndian

Insti

tute o

f Tec

hnolo

gy, K

anpu

r], [T

ATHA

GATA

BHA

DURI

] at 0

8:34 0

1 Jun

e 201

5

CHAPMAN: C4754_C036 2005/8/6 14:14 page 651 #3

Parallel Hybrid Multiobjective Metaheuristics on P2P Systems 36-651

Pareto solutionDominated solution

f 2

f1

FIGURE 36.1 Example of nondominated solutions.

the multistart model. In this chapter, we focus only on the coarse-grained models: the island model and themultistart model. Due to the communication delays, fine-grained models are often inefficient when theyare deployed in a large-scale network.

In the island (a)synchronous cooperative model, different EAs are simultaneously deployed and cooper-ate for computing better and robust solutions. They exchange, in an asynchronous way, the genetic stuffto diversify the search. The objective is to allow to delay the global convergence, especially when the EAsare heterogeneous regarding the variation operators. The migration of individuals follows a policy definedby few parameters: the migration decision criterion, the exchange topology, the number of emigrants, theemigrants selection policy, and the replacement/integration policy.

The multistart model consists in simultaneously launching several local searches. They may be hetero-geneous, but no information is exchanged between them. The results would be identical as if the algorithmswere sequentially run. Very often deterministic algorithms differ by the supplied initial solution and/orsome other parameters. This trivial model is convenient for low-speed networks of workstations.

Combinations of different metaheuristics often provide very powerful search methods. In Reference 11,two levels and two modes of hybridization are distinguished: Low and High levels, and Relay and Cooper-ative modes. The low-level hybridization consists in replacing an internal function (e.g., an operator) ofa given metaheuristic by another metaheuristic. In high-level hybrid algorithms, the different metaheur-istics are self-containing, meaning no direct relationship to their internal working is considered. Relayhybridization means a set of metaheuristics is applied in a pipeline way. The output of a metaheuristic(except the last) is the input of the following one (except the first). Conversely, teamwork hybridization is acooperative optimization model. Each metaheuristic performs a search in a solution space, and exchangessolutions with others. In this chapter, we address the high-level hybridization mechanism in the relay andcooperative modes.

36.2.3 P2P Computing for Parallel MO Optimization

In this chapter, we focus on Dispatcher/Worker-oriented P2P middlewares such as XtremWeb [4] andSETI@Home [1]. In such systems, clients can submit their jobs to the Dispatcher. A set of volatile workers(peers) request the jobs from the Dispatcher according to the cycle stealing model. Then, they execute thejobs and return the results to the Dispatcher to be collected by the clients. In these middlewares, even acentral server (the Dispatcher) is required for controlling the peers (workers) they are considered as P2Psoftware environments. Indeed, an important part of these systems is executed on these peers with a highautonomy.

One of the major limitations of P2P computing environments is that they are well suited for embarrass-ingly parallel (e.g., multiparameter) applications with independent tasks. In this case, no communicationis required between the tasks, and thus peers. The deployment of parallel hybrid metaheuristics thatneed cross-peer/task communication is not straightforward. The programmer has the burden to manage


Dow

nloa

ded

by [I

ndian

Insti

tute o

f Tec

hnolo

gy, K

anpu

r], [T

ATHA

GATA

BHA

DURI

] at 0

8:34 0

1 Jun

e 201

5

CHAPMAN: C4754_C036 2005/8/6 14:14 page 652 #4


and control the complex coordination between the workers. To deal with such problem existing middle-wares must be extended with a software layer that implements a coordination model. Several interestingcoordination models have been proposed in the literature [6,9]. In this chapter, we focus only on one ofthe most popular of them, that is, Linda [7], as our proposed model is an extension of this model.

In the Linda model, the coordination is performed through generative communications. Processesshare a virtual memory space called a tuple-space (set of tuples). The fundamental data unit, a tuple,is an ordered vector of typed values. Processes communicate by reading, writing, and consuming thesetuples. The eval operation is particularly useful in a P2P environment as it allows to spawn tasks to beexecuted on volunteer peers. A small set of four simple operations allows highly complex communicationand synchronization schemes:

Out(tuple): puts tuple into tuple-space. In(pattern): removes a (often the first) tuple matching pattern from tuple-space. rd(pattern): is the same as in(pattern), but does not remove the tuple from tuple-space. Eval(expression): puts expression in tuple-space for evaluation. The evaluation result is a tuple left

in tuple-space.

Nevertheless, Linda has several limitations regarding the design and deployment of parallel hybridmetaheuristics for P2P systems. First, it does not allow rewriting operations on the tuple space. Due tothe high communication delays in a P2P system, tuple rewriting is very important as it allows to reducethe number of communications and the synchronization cost. Indeed, in Linda a rewriting operationis performed as an in or rd operation followed by a local modification and an out operation.The operations in/rd and out involve two communications and a heavy synchronization. Therefore,the model needs to be extended with a rewriting operation. Furthermore, the model does not supportgroup operations that are useful for efficiently writing/reading Pareto sets in/from the tuple-space. Finally,nonblocking operations that are very important in a P2P context are not supported in Linda. In the nextsection, we propose an extension of the Linda model that allows to meet these requirements.

36.3 A Model for P2P Coordination

36.3.1 Model Description

Designing a coordination model for parallel MOO requires the specification of the content of the tuple-space, a set of coordination operations and a pattern matching mechanism. The tuple-space may becomposed of a set of Pareto optimal solutions and their corresponding solutions in the objective space.For the parallel island model of the multiobjective metaheuristics, the tuple-space contains a collection of(parts of) Pareto optimal sets deposited by the islands for migration. The mathematical formulation ofthe tuple-space (Pareto Space or PS) is the following:

PS =

PO, with PO = {(x , F(x)), x is Pareto optimal}.

In addition to the operations provided in Linda, parallel P2P multiobjective optimization needs otheroperations. These operations fall into two categories: group operations and nonblocking operations. Groupoperations are useful to manage multiple Pareto optimal solutions. Nonblocking operations are necessaryto take into account the volatile nature of P2P systems. In our model, the coordination primitives aredefined as follows:

in, rd, out and eval : These operations are the same as those of Linda defined in Section 36.2.3. ing(pattern): Withdraws from PS all the solutions matching the specified pattern. rdg(pattern): Reads from PS a copy of all the solutions matching the specified pattern. outg(setOfSolutions): Inserts multiple solutions in PS.


Dow

nloa

ded

by [I

ndian

Insti

tute o

f Tec

hnolo

gy, K

anpu

r], [T

ATHA

GATA

BHA

DURI

] at 0

8:34 0

1 Jun

e 201

5

CHAPMAN: C4754_C036 2005/8/6 14:14 page 653 #5


update(pattern, expression): Updates all the solutions matching the specified pattern by the solutionsresulting from the evaluation of expression.

inIfExist, rdIfExist, ingIfExist, and rdgIfExist : These operations have the same syntax thanrespectively in, rd, ing, and rdg but they are non blocking probe operations.

The update operation allows to locally update the PS, and so to reduce the communication and syn-chronization cost. The pattern matching mechanism depends strongly on how the model is implemented,and in particular on how the tuple-space is stored and accessed. For instance, if the tuple-space is storedin a database the mechanism can be the request mechanism used by the database management system.More details on the pattern matching mechanism of our model are given in the next section.

36.3.2 Implementation on Top of XtremWeb

XtremWeb [4] is a Java P2P project developed at Paris-Sud University. It is intended to distribute applica-tions over a set of peers, and is dedicated to multiparameter applications that have to be computed severaltimes with different inputs. XtremWeb manages tasks following the Dispatcher/Worker paradigm (seeFigure 36.2). Tasks are scheduled by the Dispatcher to workers only on their specific demand since theymay adaptively appear (connect to the Dispatcher) and disappear (disconnect from the Dispatcher). Thetasks are submitted by either a client or a worker, and in the latter case, the tasks are dynamically generatedfor parallel execution. The final or intermediate results returned by the workers are stored in a MySQLdatabase. These results can be requested later by either the clients or the workers. The database stores alsodifferent information related to the workers and the deployed application tasks.

XtremWeb is well suited for embarrassingly parallel applications where no cross-peer communicationoccurs between workers, and these can only communicate with the Dispatcher. Yet, many parallel dis-tributed applications particularly parallel MOMs need cooperation between workers. In order to freethe user from the burden of managing himself/herself such cooperation we propose an extension of themiddleware with a software layer.

a coordination API and its implementation at the worker level and a coordination request broker (CRB).The PS is a part of the MySQL database associated with the Dispatcher. Each tuple or solution of the PSis stored as a record in the database.

...

...

XtremWeb workers

XtremWeb dispatcher

XtremWeb clients

Internet

Get results

Submit work

Get a work unitSend results

FIGURE 36.2 Global architecture of XtremWeb.


The software layer is an implementation of the proposed model composed of two parts (see Figure. 36.3):

Dow

nloa

ded

by [I

ndian

Insti

tute o

f Tec

hnolo

gy, K

anpu

r], [T

ATHA

GATA

BHA

DURI

] at 0

8:34 0

1 Jun

e 201

5

CHAPMAN: C4754_C036 2005/8/6 14:14 page 654 #6


XtremWeb worker

XtremWeb dispatcher

XtremWeb databaseinformation

Paretospace (PS)

execlp(..., CRB_Stub,ING, ARGS_FILE, ...);

Work unit

CRB_Stub

CRB_Skeleton

ParetoSpace

Manager

switch(){}

RMI callof ing

switch(OP) {...

case ING:RMI call of ing(pattern);...

};

...

...

s=ing(pattern);

switch(OP) { ...

case ING: local call of ing(pattern); ...

};

switch(){}...

...

ing(pattern){}...

...

SELECT * FROM PSWHERE pattern is

matched

FIGURE 36.3 Implementation of the coordination model on top of XtremWeb.

From the worker side the coordination API is implemented in Java and in C/C++. The C/C++version allows the deployment and execution of C/C++ applications with XtremWeb (written in Java).The coordination library must be included in these programmer applications. From the Dispatcher side,the coordination API is implemented in Java as a PS manager. The CRB is a software broker allowing theworkers to transport their coordination operations calls to the Dispatcher, and has two components: onefor the worker (CRB stub) and another for the Dispatcher (CRB skeleton). The role of the CRB stub is totransform the local calls to the coordination operations performed by the tasks executed by the worker intoRMI calls. The role of the CRB skeleton is to transform these RMI calls into local calls to the coordinationoperations performed by the PS Manager. These local calls are translated into MySQL requests addressedto the PS.

To illustrate the implementation of the coordination layer on top of XtremWeb, let us consider the scen-ario presented in Figure 36.3. The work unit performed by an XtremWeb worker calls the ing (template)coordination operation. In the C++ version of the coordination API, the implementation of each coordin-ation operation makes the system call execlp() with appropriate parameters to plug in the CRB_Stub Javaobject. In our scenario, the major parameters are the number ING designating the operation and the fileARGS_FILE containing the arguments specified in the template parameter. CRB_Stub translates the inglocal call into an RMI call to the CRB_Skeleton Java object. This latter translates the RMI call into a localcall to the ing operation implemented in the PS Manager class. The implementation of the coordinationoperation consists in a MySQL select request addressed to the PS part of the XtremWeb informationdatabase.

Note that the method declarations for the coordination operations in the PS Manager class contain theJava synchronized keyword. Hence, the system associates a unique lock with the instance of the PS Managerclass. Whenever control enters a synchronized coordination operation, other calls to a synchronizedcooperation method are blocked until the PS Manager object is unlocked. In the next section, the proposedcoordination model is applied to parallel hybrid MOMs.


Dow

nloa

ded

by [I

ndian

Insti

tute o

f Tec

hnolo

gy, K

anpu

r], [T

ATHA

GATA

BHA

DURI

] at 0

8:34 0

1 Jun

e 201

5

CHAPMAN: C4754_C036 2005/8/6 14:14 page 655 #7


M3

M2

M1 J2

J2

J4 J5 J1 J3J6

J5 J1

J1

J4 J6 J3

J2 J4 J5 J6 J3

FIGURE 36.4 Example of permutation flow-shop with 6 jobs and 3 machines.

36.4 Application to BPFSP and Experimentation

36.4.1 Problem Formulation

The Flow-Shop problem is a scheduling problem [12] that has received a great attention given its import-ance in many industrial areas. The problem can be formulated as a set of N jobs J1, J2, . . . , JN to bescheduled on M machines. The machines are critical resources as each machine cannot be simultaneouslyassigned to two jobs. Each job Ji is composed of M consecutive tasks ti1, . . . , tiM , where tij represents thejth task of the job Ji requiring the machine mj . To each task tij is associated a processing time pij , and eachjob Ji must be achieved before a due date di .

In this chapter, we focus on the BPFSP where jobs must be scheduled in the same order on all themachines (see Figure 36.4). Therefore, two objectives have to be minimized:

Cmax: Makespan (Total completion time) T : Total tardiness

The task tij being scheduled at time sij , the two objectives can be formulated as follows:

f1 = Cmax = max{siM + piM |i [1, . . . , N ]},f2 = T =

Ni=1[max(0, siM + piM di)].

The Pareto front PF associated with BPFSP may be formulated as follows:

y , x PF , (m(x) m(y)) or (t (x) t (y)),

where x and y are solutions of the MOP, and m(x) (respectively t (x)) is the value of x corresponding tothe makespan (respectively tardiness) criterion.

36.4.2 A GeneticMimetic Algorithm for Solving BPFSP

In single objective optimization, it is well known that GAs provide better results when they are hybridizedwith LS algorithms. Indeed, the GA convergence is too slow to be really effective without any hybridization[10]. In Reference 2, a hybrid GeneticMimetic algorithm named AGMA has been proposed for solving

AGMA combines a genetic algorithm (GA) and a mimetic algorithm (MA). In this chapter, we do notgive the details and parameters of the two algorithms, and if needs be, the reader is referred to Reference 2.The GA uses mainly two parameters: an archive (Pareto Front) PO of nondominated solutions, and aprogression ratio PPO of PO. At each generation, these two parameters are updated. If no significantprogression is noticed (PPO < , where is a fixed threshold), an intensified search process is triggered.

generation. The application of MA returns a Pareto Front PO that serves to update the Pareto Front POof the GA.


Figure 36.5.

The intensification consists in applying MA (see Algorithm 36.2) to the current population during one

BPFSP. The simplified pseudocode of the algorithm is presented in Algorithm 36.1 and illustrated in

Dow

nloa

ded

by [I

ndian

Insti

tute o

f Tec

hnolo

gy, K

anpu

r], [T

ATHA

GATA

BHA

DURI

] at 0

8:34 0

1 Jun

e 201

5

CHAPMAN: C4754_C036 2005/8/6 14:14 page 656 #8


Algorithm 36.1 AGMA Algorithm

Create an initial populationwhile run time not reached do

Perform a GA generation with adaptive mutationUpdate PO and PPOif PPO < then

Perform a generation of MA on the population (Algorithm 2)Update PO and PPO

end ifUpdate selection probability of each mutation operator

end while

POP

Genetic algorithm Mimetic algorithm

POPCrossover

PO*

PO* POP

Ppo* < a

Neighbors

FIGURE 36.5 Illustration of AGMA.

Algorithm 36.2 MA algorithm

while MA run time not reached doSelect randomly a set P of solutions from the current populationApply the crossover on P to generate a set P of new solutionsCompute the nondominated set PO from P while New solutions found do

Create the neighborhood N of each solution of POLet PO be the nondominated set of N

PO

end whileend while

Mimetic algorithm consists in selecting randomly a set of solutions from the current population ofthe GA. A crossover operator is then applied to these solutions and new solutions are generated. Amongthese new solutions only nondominated ones are maintained to constitute a new Pareto Front PO. AnLS is then applied to each solution of PO to compute its neighborhood. The nondominated solutionsbelonging to the neighborhood are inserted into PO.

36.4.3 Parallel Hybrid AGMA

Different parallel models have been sketched and analyzed in Section 36.2. The fine-grained parallel modelscould not be exploited efficiently in a P2P environment due to the communication delays. In BPFSP, themodel based on parallel evaluation of each solution is fine-grained and is not likely to lead to better


Dow

nloa

ded

by [I

ndian

Insti

tute o

f Tec

hnolo

gy, K

anpu

r], [T

ATHA

GATA

BHA

DURI

] at 0

8:34 0

1 Jun

e 201

5

CHAPMAN: C4754_C036 2005/8/6 14:14 page 657 #9


performance. Indeed, the evaluation of each objective has a low cost. Therefore, it is useless to evaluatein parallel the two objectives and evaluate each of them in parallel. Conversely, it is useful to exploit thefollowing parallel models: (1) the island model that consists in performing in parallel several cooperativeAGMAs; (2) the parallel evaluation of the population of each AGMA; (3) the multistart model that consistsin applying in parallel an LS on each solution of the Pareto Front PO in MA. The parallel evaluation ofthe neighborhood of each solution could not be efficient for the same reason as the parallel evaluation ofeach solution.

We have limited our implementation to the coarse-grained parallel models, that is, the island modeland the multistart model. Figure 36.6 illustrates the parallel hybrid AGMA exploiting these two models.

The island model : Due to its exorbitant cost in terms of CPU time on large-size instances of BPFSP,the island model has not been exploited in Reference 2. Indeed, the exploitation of the model onlarge-size BPFSP is possible only on large-scale P2P networks or grids. In our implementation(see Figure 36.6), the parameters of the model are the following: The different cooperative AGMA

POP POPCrossover

Neighbors

PO*

PO* POP

Ppo* < a

PO*

XtremWeb workers

XtremWebinterface

Multistartmodel

Islandmodel

Geneticalgorithm Mimetic

algorithm

FIGURE 36.6 Illustration of parallel AGMA.


Dow

nloa

ded

by [I

ndian

Insti

tute o

f Tec

hnolo

gy, K

anpu

r], [T

ATHA

GATA

BHA

DURI

] at 0

8:34 0

1 Jun

e 201

5

CHAPMAN: C4754_C036 2005/8/6 14:14 page 658 #10


exchange their whole archives PO, and the number of emigrants is dynamic. At its arrival, theimmigrant archive is merged with the local one. Migrations occur periodically (each a fixed numberof iterations). The migration topology is the random one, meaning the destination island is selectedrandomly.

The multistart model : The multistart model is exploited during the execution of MA. Each solutionof the Pareto Front PO computed by the algorithm represents the initial solution of an LS methodthat calculates its neighborhood. The different LSs are executed in parallel according to the MasterSlave model. The master, that is, the algorithm MA merges with PO the neighborhoods returnedby the different slaves and computes the new PO that contains the nondominated solutions.

36.4.4 Deployment and Experimentation

In this section, we present the deployment scheme of the different parallel hybrid models on the XtremWebarchitecture, and the preliminary experimental results obtained on the application presented above.

36.4.5 Deployment and Fault Tolerance

A deployment scheme may be defined as a function that consists in embedding the different componentsof the parallel models on the different components of the P2P architecture. Different deployment schemaof the island and multistart models on XtremWeb are possible. Indeed, the AGMA algorithms of the islandmodel can be deployed either as XtremWeb clients or workers. For the multistart model, the master canbe either a client or a worker, and the slaves are necessarily deployed as workers. For our experimentation,the deployment scheme is illustrated in Figure 36.7.

The island model is deployed on three XtremWeb clients, and each client runs the AGMA algorithm.During the hybridization phase (execution of MA), the LSs initiated on the Pareto Front PO are submittedas tasks to the Dispatcher that launches them on the workers at their request. The multistart model is thusdeployed on a client and a set of workers.

In XtremWeb, the fault tolerance issue is tackled at Worker and Dispatcher levels. When a worker failsthe work unit being executed is re-started from scratch. If the Dispatcher crashes it is re-started usingits information database. The problem with such solution is that in a highly volatile environment a large

Dispatcher

LSNeighbors

MySQL

LS11 LS25 LSm4

LSNeighbors

Worker1 Worker2 WorkerN

Client1 Client2 Clientm

LSNeighbors

...

PO*PO*

FIGURE 36.7 Deployment schema of parallel hybrid AGMA on top of XtremWeb.


Dow

nloa

ded

by [I

ndian

Insti

tute o

f Tec

hnolo

gy, K

anpu

r], [T

ATHA

GATA

BHA

DURI

] at 0

8:34 0

1 Jun

e 201

5

CHAPMAN: C4754_C036 2005/8/6 14:14 page 659 #11


amount of CPU time is wasted as the system spends its time in re-starting work units performed by theworkers. Therefore, we propose a check-pointing approach at the client level that allows to solve moreefficiently the fault-tolerance problem. Indeed, the problem data and intermediate results are periodicallystored. If the Dispatcher fails the application is restored and re-started from the last checkpoint. In caseof worker failure the work unit is re-started using the intermediate results. The check-pointing operation(storing) is performed after each LS and/or every 100 generations. The second condition is necessary whenno LS has been launched during the last 100 generations. This means that a significant progression of thePareto Front has been observed at each generation, what excludes any resort to the hybridization.

36.4.6 Experimental Results

In our experiments, we consider the BPFSP instance 200 jobs on 10 machines. The parameters of theisland model are fixed as the following: migrations occur every 10 generations, the number of emigrantsis fixed at each migration operation to 20 the size of the archive PO is upper than 20, and the wholearchive otherwise, and the population size of each AGMA is 100.

The application has been deployed during working days (nondedicated environment) on the educationnetwork of the PolytechLille engineering school. The experimentation hardware platform is composed of120 heterogeneous Linux Debian PCs. The characteristics of these PCs are presented in Table 36.1.

Three parallel hybrid versions of AGMA are experimented, evaluated and compared:

Version 1 is that proposed in Reference 2, and exploits only the multistart model. This meansthat the GA is executed on a single machine and the hybridization phase is deployed on a parallelmachine (IBM-SP2) according to the MasterSlave model (Push mode, i.e., work distribution isinitiated by the master).

Version 2 is the same as Version 1 except that the hybridization is deployed in a distributed wayon a set of XtremWeb workers according to the cycle stealing paradigm (Pull mode, i.e., workdistribution is initiated by the workers).

Version 3 is not considered in Reference 2, and is a combination of the multistart and island models.

according to the island model. Each AGMA is an implementation of Version 2.

are approximately the same, but Version 2 has the advantage to be fault tolerant.The execution of Version 1 is stopped after 80 LSs as it is not fault tolerant. Conversely, with Version 2

350 LSs. The execution lasted one week, and 10 failures have been observed and as many check-pointingoperations have been performed. As a result, the Pareto Front obtained with 350 LSs is clearly better thanthat obtained with 80 LSs using Version 1 or Version 2. One has to note that such results are possible onlywith a scalable and fault tolerant version of the algorithm.

TABLE 36.1 Experimentation Hardware Platform

Processor Number

AMD Duron(tm) Processor 14Celero (Coppermine) 14Intel(R) Celeron(R) CPU 2.00 GHz 8Intel(R) Celeron(R) CPU 2.20 GHz 28Intel(R) Celeron(R) CPU 2.40 GHz 21Intel(R) Celeron(R) CPU 1400 MHz 7Pentium III (Katmai) 28

Total 120


As illustrated in Figure 36.7, three AGMA algorithms are deployed on client machines and cooperate

Figure 36.8 illustrates the Pareto Fronts obtained with the versions 1 and 2 after 80 LSs. The two fronts

long-lasting executions are possible. For instance, Figure 36.9 shows that the execution goes on up to

Dow

nloa

ded

by [I

ndian

Insti

tute o

f Tec

hnolo

gy, K

anpu

r], [T

ATHA

GATA

BHA

DURI

] at 0

8:34 0

1 Jun

e 201

5

CHAPMAN: C4754_C036 2005/8/6 14:14 page 660 #12


40000 45000 50000 55000 60000 65000 7000010850

10900

10950

11000

11050

11100

11150

11200

11250M

akesp

an

Version 1 (80 LS)Version 2 (80 LS)

Tardiness

FIGURE 36.8 Pareto Fronts obtained with Version 1 and Version 2 (80 LSs).

35000 40000 45000 50000 55000 60000 65000 7000010850

10900

10950

11050

11000

11100

11200

11150

11250

11300

11350

Mak

esp

an


Tardiness

FIGURE 36.9 Pareto Fronts with Version 1 (80 LSs) and Version 2 (350 LSs).


Dow

nloa

ded

by [I

ndian

Insti

tute o

f Tec

hnolo

gy, K

anpu

r], [T

ATHA

GATA

BHA

DURI

] at 0

8:34 0

1 Jun

e 201

5

CHAPMAN: C4754_C036 2005/8/6 14:14 page 661 #13


42000 44000 46000 48000 50000 52000 54000 56000 58000 60000 62000 6400010850

10900

10950

11000

11050

11100

11150

11200

11250M

akesp

an


Tardiness

FIGURE 36.10 Pareto Fronts with Version 2 (80 LSs) and Version 3 (80 LSs).

Figure 36.10 allows to compare the Pareto Fronts obtained with Version 2 and Version 3 and todemonstrate the contribution of the island model to the effectiveness. With 80 LSs, the Pareto Frontobtained using Version 3 is better than that obtained using Version 2. More experiments with more LSsare in progress.

Version 3. Figure 36.11 (Part B) is a zoom up of Figure 36.11 (Part A) on the 50,000 first time units. Itshows that the MA (thus LS) is frequently solicited and this lasts long. Figure 36.11 (Part C) illustrates theevolution in time of the number of deployed workers at the beginning of the execution (zoom out on thefirst 350 time-units). The maximum number of workers is 60 because during the starting phase the ParetoFront contains a small number of solutions. The number of workers decrease to 0 when the GA succeedto improve the Pareto Front without calling MA.

One has to note that the spectrum blackens with the time (from left to right). This means that the GAsolicits more and more the MA, that is, the LS because it never enhances again the Pareto Front, in otherwords the GA converges. On the other hand, the local search lasts less and less time. Therefore, even theintensification (by LS) does not contribute to enhance the effectiveness, meaning that the AGMA con-verges. Through this experimentation, we have learned more on the convergence of the AGMA algorithm.Therefore, one can note that P2P computing allows topush farthe limits in terms of computing resourcesto better evaluate the contribution of the hybridization but also its limitations.

36.5 Conclusion and Future Work

The hybridization of metaheuristics having complementary behaviors allows to enhance the effectivenessand robustness in combinatorial optimization [11]. However, its exploitation on industrial applications ispossible only by using a great computing power. Large-scale parallelism based on the use of computationalgrids and/or P2P systems is recently revealed to be a good way to get at hand such computing power andexploit hybridization. To our best of knowledge, no research work has been published on parallel hybrid


Figure 36.11 (Part A) shows the oscillation between GA and MA (or LS) over time obtained with

Dow

nloa

ded

by [I

ndian

Insti

tute o

f Tec

hnolo

gy, K

anpu

r], [T

ATHA

GATA

BHA

DURI

] at 0

8:34 0

1 Jun

e 201

5

CHAPMAN: C4754_C036 2005/8/6 14:14 page 662 #14


.

TimeMA(LS)

GA

Time

Time

(Part A)

(Part C)

5000040000100005000 3000020000 45000350002500015000

500000450000350000 400000300000000mps

0

(Part B)

0 50 100 150 200 250 300 350

60

50

40

30

20

10

0

Number of workers

FIGURE 36.11 Oscillation spectrum between the GA and MA (thus LS).

metaheuristics on P2P systems. Nowadays, existing P2P computing middlewares are inadequate for thedeployment of parallel cooperative applications. Indeed, these need to be extended with a software layerto support the cooperation. In this chapter, we have proposed a Linda-like cooperation model that hasbeen implemented on top of XtremWeb.

In Reference 2, a hybrid metaheuristic (AGMA) has been proposed and experimented on BPFSP.The performed experiments on large-size instances such as 200 jobs on 10 machines are often stoppedwithout the convergence is reached. The full exploitation of the hybridization needs a large amount ofcomputational resources and the management of the fault-tolerance issue. We have proposed a fault-tolerant hybrid parallel design of the AGMA combining two parallel models: the multistart model and theisland model. The algorithm has been implemented on our extended version of XtremWeb.

The first experiments have been performed on the education network of the PolytechLille engineeringschool. The network is composed of 120 heterogeneous Linux PCs. The preliminary results, obtainedafter several execution days, demonstrate that the use of P2P computing allows to fully exploit the benefitsof hybridization. Indeed, the obtained Pareto Front is clearly better than that obtained in Reference 2.


Dow

nloa

ded

by [I

ndian

Insti

tute o

f Tec

hnolo

gy, K

anpu

r], [T

ATHA

GATA

BHA

DURI

] at 0

8:34 0

1 Jun

e 201

5

CHAPMAN: C4754_C036 2005/8/6 14:14 page 663 #15


On the other hand, the deployment of the island model allows to improve the effectiveness. Beyond theimprovement of the effectiveness, the parallelism on P2P systems allows to push far the limits in termsof computational resources. As a consequence, it permits to better evaluate the benefits and limitationsof the hybridization. Such result has to be confirmed again on a larger P2P network and larger instancesof the problem.

References

[1] D.P. Anderson, J. Cobb, E. Korpela, M. Lepofsky, and D. Werthimer. SETI@home: An experimentin public-resource computing. Communications of the ACM, 45: 5661, 2002.

[2] M. Basseur, F. Seynhaeve, and E.-G. Talbi. Adaptive mechanisms for multi-objective evolution-ary algorithms. In Congress on Engineering in System Application CESA 03, Lille, France, 2003,pp. 7286.

[3] S. Cahon, N. Melab, and E.-G. Talbi. ParadisEO: A framework for the reusable design of paralleland distributed metaheuristics. Journal of Heuristics, 10: 353376, 2004.

[4] G. Fedak, C. Germain, V. Neri, and F. Cappello. XtremWeb: Building an experimental platformfor Global Computing. Workshop on Global Computing on Personal Devices (CCGRID2001), IEEEPress, May 2001.

[5] I. Foster and C. Kesselman. The Grid: Blueprint for a New Computing Infrastructure. MorganKaufmann, San Fransisco, CA, 1999.

[6] D. Gelernter and N. Carriero. Coordination languages and their significance. Communications ofthe ACM, 35: 97107, 1992.

[7] D. Gelernter. Generative communication in Linda. ACM Transactions on Programming Languagesand Systems, 7: 80112, 1985.

[8] A. Oram. Peer-to-Peer: Harnessing the Power of Disruptive Technologies. OReilly & Associates, 2001.[9] G.A. Papadopoulos and F. Arbab. Coordination models and languages. Advances in Computers:

The Engineering of Large Systems, Academic Press, 1998, p. 46.[10] E.-G. Talbi, M. Rahoual, M.-H. Mabed, and C. Dhaenens. A hybrid evolutionary approach

for multicriteria optimization problems: Application to the Flow Shop. In E. Zitzler et al.,Eds., Evolutionary Multi-Criterion Optimization, Vol. 1993 of Lecture Notes in Computer Science,Springer-Verlag, Heidelberg, 2001, pp. 416428.

[11] E.-G. Talbi. A Taxonomy of Hybrid Metaheuristics. Journal of Heuristics, 8: 541564, 2002.[12] V. Tkindt and J.-C. Billaut. Multicriteria Scheduling Theory, Models and Algorithms. Springer-

Verlag, Heidelberg, 2002.[13] J. Verbeke, N. Nadgir, G. Ruetsch, and I. Sharapov. Framework for peer-to-peer distributed com-

puting in a heterogeneous, decentralized environment. In Proceedings of the Third InternationalWorkshop on Grid Computing (GRID 2002), Baltimore, MD, January 2002, pp. 112.


Dow

nloa

ded

by [I

ndian

Insti

tute o

f Tec

hnolo

gy, K

anpu

r], [T

ATHA

GATA

BHA

DURI

] at 0

8:34 0

1 Jun

e 201

5

Chapter 36: Parallel Hybrid Multiobjective Metaheuristics on P2P Systems36.1 Introduction36.2 Parallel Hybrid MOMs and P2P Computing36.2.1 Multiobjective Optimization36.2.2 Parallelism and Hybridization36.2.3 P2P Computing for Parallel MO Optimization

36.3 A Model for P2P Coordination36.3.1 Model Description36.3.2 Implementation on Top of XtremWeb

36.4 Application to BPFSP and Experimentation36.4.1 Problem Formulation36.4.2 A GeneticMimetic Algorithm for Solving BPFSP36.4.3 Parallel Hybrid AGMA36.4.4 Deployment and Experimentation36.4.5 Deployment and Fault Tolerance36.4.6 Experimental Results

36.5 Conclusion and Future WorkReferences

9781420035063%2ech36

Documents

parallel applications

parallel tasks

parallel mooptimization36

parallel hybrid moms

peer p2p computing

p2p coordination

traditional parallel

grid computing