May 2002 Parallel GRASP with PR for the 2-path network design problem
1/35 PAREO’2002
PAREO 2002Guadeloupe, May 20-24, 2002
A Parallel GRASP with Path-Relinking Heuristic for the 2-Path Network Design Problem
Celso C. RIBEIRO Isabel ROSSETI
Catholic University of Rio de Janeiro BrazilGosier, Guadeloupe
May 2002 Parallel GRASP with PR for the 2-path network design problem
2/35 PAREO’2002
Summary• Problem formulation• GRASP with path-relinking heuristic
– Construction phase– Local search phase– Path-relinking
• Parallel implementation– Independent strategy– Cooperative strategy
• Computational results• Concluding remarks
May 2002 Parallel GRASP with PR for the 2-path network design problem
3/35 PAREO’2002
2-path network design problem
• Graph G = (V,E)V: node setE: edge setweights we associated with each edge e E
• k-path between nodes s,t V: sequence of at most k edges connecting s and t
• D: set of demands (origin-destination pairs)
May 2002 Parallel GRASP with PR for the 2-path network design problem
4/35 PAREO’2002
2-path network design problem
• 2-path network design problem (2PNDP): Find a minimum weighted subset of edges E’ E containing a 2-path in G between the extremities of every origin-destination pair in D
• Applications: design of communication networks, in which paths with few edges are sought to enforce high reliability and small delays
May 2002 Parallel GRASP with PR for the 2-path network design problem
5/35 PAREO’2002
2-path network design problem
• Dahl & Johannessen (2000): – Decision version of 2PNDP is NP-complete.– Approximate algorithm– Exact cutting plane algorithm
• Balakrishnan & Altinkemer (1992): – Integer programming formulation for kPNDP– See also LeBlanc, Chifflet & Mahey (1999).
• Generalizations: k-hop minimum spanning tree, k-hop minimum Steiner tree
May 2002 Parallel GRASP with PR for the 2-path network design problem
6/35 PAREO’2002
GRASP with path-relinking• GRASP:
– Multistart metaheuristic: Feo & Resende (1989)• Path-relinking:
– Intensification strategy: Glover (1996)• Repeat for Max_Iterations:
– Construct a greedy randomized solution– Use local search to improve the constructed
solution– Apply path-relinking to further improve the
solution– Update the pool of elite solutions– Update the best solution found
May 2002 Parallel GRASP with PR for the 2-path network design problem
7/35 PAREO’2002
GRASP with path-relinking
• GRASP– Construction phase
1. Set the modified weights equal to the original weights.
2. Randomly select an origin-destination pair (a,b) D.3. Compute a shortest 2-path between a and b using
the modified weights.4. Set to 0 the modified weights of the edges in this
path.5. Remove (a,b) from D.6. If D is empty stop, otherwise go back to step 2.
May 2002 Parallel GRASP with PR for the 2-path network design problem
8/35 PAREO’2002
GRASP with path-relinking• GRASP
– Local search phase1. Generate a circular random permutation of the pairs in
D.2. Select the next origin-destination pair (a,b) D.3. Tentatively replace the shortest 2-path between a and
b:• Weights of edges used by other 2-paths are temporarilly set to
0.• Compute a new shortest 2-path between a and b.• Update the current solution if it is improved by the new 2-path.• Restore all original edge weights.
4. If |D| paths have been investigated without improvement stop, otherwise go back to step 2.
May 2002 Parallel GRASP with PR for the 2-path network design problem
9/35 PAREO’2002
GRASP with path-relinking
• Path-relinking: introduced in the context of tabu search by Glover (1996)– Intensification strategy using set of
elite solutions
• Consists in exploring trajectories that connect high quality solutions.initial
solution
guidingsolution
path in neighborhood of solutions
May 2002 Parallel GRASP with PR for the 2-path network design problem
10/35 PAREO’2002
GRASP with path-relinking• Path is generated by selecting
moves that introduce in the initial solution attributes of the guiding solution.
• At each step, all moves that incorporate attributes of the guiding solution are evaluated and the best move is taken:
Initialsolution
guiding solution
May 2002 Parallel GRASP with PR for the 2-path network design problem
11/35 PAREO’2002
Elite solutions x and y(x,y): symmetric difference
between x and y while ( |(x,y)| > 0 ) {
evaluate moves corresponding in (x,y) make best move
update (x,y)}
GRASP with path-relinking
May 2002 Parallel GRASP with PR for the 2-path network design problem
12/35 PAREO’2002
GRASP with path-relinking
• Maintain an elite set of solutions found during GRASP iterations.
• After each GRASP iteration (construction and local search):– Select an elite solution at random:
guiding solution.– Use GRASP solution as initial solution.– Perform path-relinking between these
two solutions.
May 2002 Parallel GRASP with PR for the 2-path network design problem
13/35 PAREO’2002
GRASP with path-relinking• Successful applications:
– Prize-collecting Steiner tree problem: Canuto, Resende & Ribeiro (2001)
– Minimum Steiner tree problem: Ribeiro, Uchoa & Werneck (2002) (e.g., best known results for open problems in series dv640 of the SteinLib)
– Three-index assignment problem: Aiex et al. (2000)
– Capacitated minimum spanning tree:Souza, Duhamel & Ribeiro (2002) (e.g., best known results for largest problems with 160 nodes)
May 2002 Parallel GRASP with PR for the 2-path network design problem
14/35 PAREO’2002
GRASP with path-relinking
• P is a set of elite solutions.• Each iteration of first |P| GRASP
iterations adds one solution to P (if different from others).
• After that: solution x is promoted to P if:– x is better than best solution in P.– x is not better than best solution in P, but is
better than worst and is sufficiently different from all solutions in P.
May 2002 Parallel GRASP with PR for the 2-path network design problem
16/35 PAREO’2002
Parallel implementation• Main interest of parallel implementations of
metaheuristics: robustnessCung, Martins, Ribeiro & Roucairol (2001)
• Multiple-walk independent-thread strategy: – p processors available– Iterations evenly distributed over the p processors– Each processor keeps a copy of the algorithm and
data– One processor acts as the master (data, seeds,
iterations), but also performs GRASP iterations– Each processor performs Max_Iterations/p
iterations
May 2002 Parallel GRASP with PR for the 2-path network design problem
17/35 PAREO’2002
Parallel implementation: independent
seed(1) seed(2) seed(3) seed(4) seed(p-1)
Best solution is sent to the master
1 2 3 4 p-1Elite Elite Elite Elite Elite
Elite
pseed(p)
May 2002 Parallel GRASP with PR for the 2-path network design problem
18/35 PAREO’2002
Parallel implementation• Main interest of parallel implementations of
metaheuristics: robustnessCung, Martins, Ribeiro & Roucairol (2001)
• Multiple-walk cooperative-thread strategy: – p processors available– Iterations evenly distributed over p-1 processors– Each processor keeps a copy of the algorithm
and data– One processor acts as the master (data, seeds,
iterations) and controls the pool of elite solutions
– Each slave processor performs Max_Iterations/(p-1) iterations
May 2002 Parallel GRASP with PR for the 2-path network design problem
19/35 PAREO’2002
Parallel implementation: cooperative
2
Elite
1
p3
Elite solutions are stored in a centralized pool Master
Slave SlaveSlave
May 2002 Parallel GRASP with PR for the 2-path network design problem
20/35 PAREO’2002
Computational results
• Parallel GRASP heuristic:– Implementation in C– MPI LAM 6.3.2 for communication– Linux cluster with 32 Pentium II-400 processors
• Largest instances solved: – Larger instances solved with the GRASP
heuristic: |V|= 400, |E|= 79800, |D|= 4000(previously: |V|= 120, |E|= 7140, |D|= 60)
May 2002 Parallel GRASP with PR for the 2-path network design problem
21/35 PAREO’2002
Computational results• Effectiveness:
– 100 small instances with 70 nodes generated as in Dahl and Johannessen (2000) for comparison purposes.
– Statistical test t for unpaired observations
– Parallel GRASP finds better solutions with 40% of confidence.
Parallel
GRASP
Sample A
D&J (2000
) Sampl
e B
Size 100 30
Mean 443.7 (-
2.2%)
453.7
Std. dev.
40.6 61.6
May 2002 Parallel GRASP with PR for the 2-path network design problem
22/35 PAREO’2002
• Variants of GRASP with path-relinking:– GRASP: pure GRASP– G+PR(B): GRASP with backward PR– G+PR(F): GRASP with forward PR– G+PR(BF): GRASP with two-way PR
T: elite solution S: local search• Other strategies:
– Truncated path-relinking– Do not apply PR at every iteration
(frequency)
Variants of GRASP with path-relinking
S T
TS
S T
S T
May 2002 Parallel GRASP with PR for the 2-path network design problem
23/35 PAREO’2002
Variants of GRASP with path-relinking
• Select an instance and a target value.• For each variant of GRASP with path-
relinking:– Perform 200 runs using different seeds.– Stop when a solution value at least as good as
the target is found.– For each run, measure the time-to-target-
value.– Plot the probabilities of finding a solution at
least as good as the target value within some computation time.
May 2002 Parallel GRASP with PR for the 2-path network design problem
24/35 PAREO’2002
Variants of GRASP with path-relinking
Each variant: 200 runs for one instance of 2PNDP
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0.1 1.0 10.0 100.0 1000.0 10000.0
Pro
ba
bili
ty
Time (seconds)
G+PR(BF)G+PR(B)G+PR(F)GRASP
May 2002 Parallel GRASP with PR for the 2-path network design problem
25/35 PAREO’2002
Variants of GRASP with path-relinking
• Same computation time: probability of finding a solution at least as good as the target value increases from GRASP G+PR(F) G+PR(B) G+PR(BF)
• P(h,t) = probability that variant h finds a solution as good as the target value in time no greater than t– P(GRASP,10s) = 2% P(G+PR(F),10s) = 56%
P(G+PR(B),10s) = 75% P(G+PR(BF),10s) = 84%
• Effectiveness of path-relinking to improve and speedup the pure GRASP
May 2002 Parallel GRASP with PR for the 2-path network design problem
26/35 PAREO’2002
Independent strategy: speedups• Linear speedups: |V|= 400, 3200 iterations, G+PR(BF)
1.02.0
4.0
7.9
15.7
31.2
1 2 4 8 16 32
Sp
ee
du
p
Processors
GRASP+PR
May 2002 Parallel GRASP with PR for the 2-path network design problem
27/35 PAREO’2002
Cooperative vs. independent strategy
• Solution quality• Same instance:
15 runs with different seeds, 3200 iterations
• The pool is poorer when fewer GRASP iterations are performed
Procs.
best
avg. best avg.
1 520 525.4
- -
2 519 524.5
519 526.4
4 524 527.8
521 526.3
8 524 529.5
521 526.5
16 533 535.1
515 525.0
32 538 541.2
521 526.3
Independent Cooperative
May 2002 Parallel GRASP with PR for the 2-path network design problem
28/35 PAREO’2002
0
500
1000
1500
2000
1 2 4 8 16 32
Tim
e (
s)
Processors
independentcooperative
Cooperative vs. independent strategy
Procs.
Indep.
Coop.
1 1358.5
-
2 682.2
2192.1
4 333.0
740.4
8 165.0
312.4
16 81.6 197.9
32 41.2 182.7
May 2002 Parallel GRASP with PR for the 2-path network design problem
29/35 PAREO’2002
Cooperative vs. independent strategy
• Select an instance and a target value.• For each strategy:
– Perform 100 runs using different seeds (not all runs already performed).
– Stop when a solution value at least as good as the target is found.
– For each run, measure the time-to-target-value.
– Plot the probabilities of finding a solution at least as good as the target value within some computation time.
May 2002 Parallel GRASP with PR for the 2-path network design problem
30/35 PAREO’2002
Cooperative vs. independent strategy
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
100 1000 10000 100000
Pro
babili
ty
Time (s)
Independent vs cooperative approaches on 2 processors
cooperative (2 procs.)independent (2 procs.)
May 2002 Parallel GRASP with PR for the 2-path network design problem
31/35 PAREO’2002
Cooperative vs. independent strategy
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
10 100 1000 10000
Pro
babili
ty
Time (s)
Independent vs cooperative approaches on 4 processors
cooperative (4 procs.)independent (4 procs.)
May 2002 Parallel GRASP with PR for the 2-path network design problem
32/35 PAREO’2002
Cooperative vs. independent strategy
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
10 100 1000 10000
Pro
babili
ty
Time (s)
Independent vs cooperative approaches on 8 processors
cooperative (8 procs.)independent (8 procs.)
May 2002 Parallel GRASP with PR for the 2-path network design problem
33/35 PAREO’2002
Cooperative vs. independent strategy
• Recall that when p processors are used:– All of them perform GRASP iterations in
the independent strategy– Only p-1 processors perform GRASP
iterations in the cooperative strategy• Cooperative strategy improves w.r.t.
the independent strategy when the number of processors increases.
• Cooperative strategy is already faster for p 4 processors.
May 2002 Parallel GRASP with PR for the 2-path network design problem
34/35 PAREO’2002
Concluding remarks• New heuristic for the 2-path network design
problem.• Effectiveness of the new heuristic:
– Larger problems solved.– New heuristic finds better solutions.– Domination is stronger for harder or larger
instances.• Path-relinking adds memory and intensification
mechanisms to GRASP, systematically contributing to improve solution quality (some implementation strategies appear to be more effective than others).
• Linear speedups with the parallel implementation.
• Cooperative strategy is faster and better.
May 2002 Parallel GRASP with PR for the 2-path network design problem
35/35 PAREO’2002
Slides and publications• Slides of this talk can be downloaded from:
http://www.inf.puc-rio/~celso/talks• Paper about the parallel GRASP heuristic for
the 2-path network design problem available at:
http://www.inf.puc-rio.br/~celso/publicacoes
Isabel Rosseti