investigations on local search based hybrid metaheuristics

INVESTIGATIONS ON LOCAL SEARCHBASED HYBRID METAHEURISTICSHabilitationskolloquium

Vienna, October 1, 2014

Luca Di GasperoInstitute of Computer Graphics and Algorithms

THE RESEARCH AREA: PRACTICALMETHODS FOR COMBINATORIAL SEARCH

AND OPTIMIZATION PROBLEMS

COMBINATORIAL SEARCH AND OPTIMIZATION PROBLEMS

▶ Combinatorial Search problem:

find 𝑧 𝑠.𝑡.𝑧 ∈ ℱ

▶ Combinatorial Optimization problem:

min𝑧

𝐹( 𝑧) 𝑠.𝑡.𝑧 ∈ ℱ

3

HYBRID METAHEURISTICS

HEURISTICS ANDMETAHEURISTICS

▶ Heuristics (Pearl 1984; Pólya 1945) a practicalway to “tame”NP-[complete|hard] search and optimization problems

▶ non-systematic solution search or construction guided byexperiential problem knowledge

▶ in general, with no guarantee (efficiency vs completeness)▶ Metaheuristics a class of general-purpose heuristic methods

▶ independent (to some extent) from the specific problem(rely on an abstract/indirect problem representation)

▶ Local Searchmethods, Genetic and Evolutionary Algorithms, AntColony Optimization, Particle Swarm Optimization, …

▶ date back to the 1950s (Barricelli 1954; Robbins and Monro1951), term introduced by Glover in 1986

▶ genuinely interdisciplinary research area: AI ∩ OR ∩ CS▶ impressive success records (state-of-the-art for Routing,

Timetabling, Scheduling, SAT)

▶ empirical algorithmics (Johnson 2002; McGeoch 2012)

5

LOCAL SEARCHMETAHEURISTICS

▶ Local Search is based the following scheme:procedure LocalSearch(𝑆, 𝑁, 𝐹 )

𝑠0 ← GenerateInitialSolution()while ¬StopSearch(𝑠𝑖, 𝑖) do

𝑚 ← SelectMove(𝑠𝑖, 𝐹, 𝑁 )if AcceptableMove(𝑚, 𝑠𝑖, 𝐹 ) then

𝑠𝑖+1 ← 𝑠𝑖 ⊕ 𝑚else

𝑠𝑖+1 ← 𝑠𝑖end if𝑖 ← 𝑖 + 1

end whileend procedure

▶ the problem interface is provided by a search space 𝑆, aneighborhood relation 𝑁 and a cost function 𝐹

▶ Local search methods differ in the following components:▶ SelectMove: How to explore the neighborhood▶ AcceptableMove: When to perform the move▶ StopSearch: When to stop the search

(e.g., Hill Climbing, Simulated Annealing, Tabu Search)6

HYBRIDIZATION

▶ Hybridization of solution methods for NP-[complete|hard]problems is a promising research trend

▶ a skillful combination of components can lead to moreeffective solution methods

▶ merging complementary strengths of different solutionparadigm (milden the weaknesses)

▶ Hybrid Metaheuristics a combination of one metaheuristic withcomponents from other metaheuristics or with techniques from AIand OR (Blum et al. 2008)

7

HYBRID METAHEURISTICS CLASSIFICATION(Puchinger and Raidl 2005; Raidl 2006)

HybridMetaheuristic

controlstrategy

collaborativehomogeneity

spacedecomposition

integrative

orderof execution

parallel

interleaved

batch

level ofhybridization low-level

(strong coupling)

high-level(weak coupling)

hybridizationcomponents

other OR/AItechniques other

heuristic methods

exacttechniques

problem-specificcomponent

Metaheuristics

8

MOTIVATIONS AND PAPER SELECTION

MOTIVATIONS

▶ a skillful combination of components can lead to more effectivesolution methods

▶ merging complementary strengths of different solution paradigm(milden the weaknesses)

▶ Local Search is my area of expertise since my PhD▶ contribute the algorithmic ideas also in form of software

systems:▶ EasyLocal++ (Di Gaspero and Schaerf 2003) enhancements▶ GELATO (Cipriano, Di Gaspero, and Dovier 2013)▶ GECODE LNS

10

PAPER SELECTION GUIDELINES

1. full coverage of the different kinds of hybridization consideredin my work

2. span most the different categories reported in the HybridMetaheuristics classification

3. approaches that reached new state-of-the-art results for theproblem tackled

11

PAPER SELECTION GUIDELINES

1. full coverage of the different kinds of hybridization consideredin my work

2. span most the different categories reported in the HybridMetaheuristics classification

3. approaches that reached new state-of-the-art results for theproblem tackled

▶ a uniform temporal coveragewas not possible because ofrecent consolidation

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014←−−1

←−−2

3−−→

←−−4

5−−→

6−−→

Figure: Journal papers, Papers in edited books, Conference papers

11

SELECTED PAPERS

1. L. Di Gaspero and A. Schaerf. A composite-neighborhood tabu search approach to thetraveling tournament problem. Journal of Heuristics, 13(2):189–207, 2007.

2. M. Chiarandini, L. Di Gaspero, S. Gualandi, and A. Schaerf. The balanced academiccurriculum problem revisited. Journal of Heuristics, 18(1):119—148, 2012.

3. R. Bellio, L. Di Gaspero, and A. Schaerf. Design and statistical analysis of a hybrid localsearch algorithm for course timetabling. Journal of Scheduling, 15(1):49—61, 2012.

4. L. Di Gaspero, G. Di Tollo, A. Roli, and A. Schaerf. Hybrid metaheuristics for constrainedportfolio selection problem. Quantitative Finance, 11(10):1473-1488, 2011.

5. R. Cipriano, L. Di Gaspero, and A. Dovier. GELATO: a multi-paradigm tool for LargeNeighborhood Search. In E.-G. Talbi (ed.). Hybrid metaheuristics. volume 434 ofStudies in Computational Intelligence, pages 389-414. Springer Verlag, 2012.

6. L. Di Gaspero, A. Rendl, and T. Urli. Constraint-based approaches for Balancing BikeSharing Systems. In C. Schulte (ed.). Principles and Practice of ConstraintProgramming - 19th International Conference, CP 2013, Uppsala, Sweden,September 16-20, 2013. Proceedings. volume 8124 of Lecture Notes in ComputerScience, pages 758—773. Springer Verlag, 2013.

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014←−−1

←−−2

3−−→

←−−4

5−−→

6−−→

Figure: Journal papers, Papers in edited books, Conference papers

12

OUTLINE

▶ Local Search self-hybridization▶ Hybridization with Exact Methods

13

LOCAL SEARCH SELF-HYBRIDIZATION

NEIGHBORHOOD PORTFOLIO

▶ For a given Local Search problem encoding there might bemany natural neighborhoods

▶ Neighborhood Portfolios: LS handles multiple neighborhoodsprocedure LocalSearch(𝑆, 𝑁1, 𝑁2, … , 𝑁𝑘, 𝐹 )

𝑠0 ← GenerateInitialSolution()while ¬StopSearch(𝑠𝑖, 𝑖) do

𝑚 ← SelectMove(𝑠𝑖, 𝐹,𝑁1, 𝑁2, … , 𝑁𝑘)if AcceptableMove(𝑚, 𝑠𝑖, 𝐹 ) then

𝑠𝑖+1 ← 𝑠𝑖 ⊕ 𝑚else

𝑠𝑖+1 ← 𝑠𝑖end if𝑖 ← 𝑖 + 1


▶ Operators: union 𝑁𝑖 ∪ 𝑁𝑗, sequence 𝑁𝑖 ⊗ 𝑁𝑗, composition𝑁𝑖 ⊙ 𝑁𝑗 = (𝑁𝑖 ∪ 𝑁𝑗) ⊗ (𝑁𝑖 ∪ 𝑁𝑗)

15

NEIGHBORHOOD PORTFOLIOHybrid Metaheuristics Classification

HybridMetaheuristic

controlstrategy


spacedecomposition

integrative

orderof execution

parallel

interleaved

batch


(strong coupling)




heuristic methods

exacttechniques


Metaheuristics

16

THE TRAVELING TOURNAMENT PROBLEMA Neighborhood Portfolio Approach

▶ A combined Sport Scheduling problem▶ Find a Round-Robing tournament scheduling▶ Minimizing the travel distances (Traveling Salesman)▶ with side constraints

▶ Highly combinatorial: non-isomorphic tournament patterns|𝑃𝑛| for 𝑛 teams: |𝑃8| = 6, |𝑃12| = 526, 915, 620,|𝑃14| = 1, 132, 835, 421, 602, 062, 347

17

SOLUTION REPRESENTATION FOR THE TTP

Team 0 1 2 3 4ATL FLA NYM PIT @PHI @MONNYM @PIT @ATL @FLA MON FLAPHI @MON FLA MON ATL @PITMON PHI @PIT @PHI @NYM ATLFLA @ATL @PHI NYM PIT @NYMPIT NYM MON @ATL @FLA PHI

Team 5 6 7 8 9 DistanceATL @PIT PHI MON @NYM @FLA 4414NYM @PHI @MON PIT ATL PHI 3328PHI NYM @ATL @FLA PIT @NYM 3724MON FLA NYM @ATL @FLA PIT 3996FLA @MON @PIT PHI MON ATL 5135PIT ATL FLA @NYM @PHI @MON 3319

18

NEIGHBORHOOD STRUCTURES FOR THE TTP

▶ 𝑇 is the set {0, … , 𝑛 − 1} of teams, and 𝑅 the set{0, … , 𝑟 − 1} of rounds𝑁1, SwapHomes: ⟨𝑡1, 𝑡2⟩, swaps home/away position of the

two games between 𝑡1 and 𝑡2.𝑁2, SwapTeams: ⟨𝑡1, 𝑡2⟩, swaps 𝑡1 and 𝑡2 throughout the

whole solution𝑁3, SwapRounds: ⟨𝑟1, 𝑟2⟩, all matches assigned to 𝑟1 are

moved to 𝑟2, and vice versa𝑁4, SwapMatches: ⟨𝑟, 𝑡1, 𝑡2⟩ the opponents of 𝑡1 and 𝑡2 in 𝑟

are exchanged𝑁5: SwapMatchRound ⟨𝑡, 𝑟1, 𝑟2⟩, the opponents of 𝑡 in

rounds 𝑟1 and 𝑟2 are exchanged▶ as for 𝑁4 and 𝑁5, the execution breaks the tournament,

therefore a repair chain is needed

19

TTP NEIGHBORHOOD ANALYSIS AND SELECTION

▶ In (Di Gaspero and Schaerf 2007) we perform a thoroughanalysis of the neighborhoods, in term of the search spaceconnectedness

▶ the outcome of the analysis was the selection of the followingcomposed neighborhoods

▶ 𝐶𝑁1 = ⋃5𝑖=1 𝑁𝑖

▶ 𝐶𝑁2 = 𝑁1 ∪ 𝑁4 ∪ 𝑁5▶ 𝐶𝑁3 = 𝑁1 ∪ 𝑁≤4

4 ∪ 𝑁≤65

▶ the selected neighborhoods equipped a Tabu Search (Gloverand Laguna 1997) metaheuristic

20

EXPERIMENTAL RESULTS

CN1 CN2 CN3 CN4

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

Figure: Distribution of the ranks

CN1 CN2 CN3 CN4

0.0

00.0

50.1

00.1

5

Figure: Distribution of thenormalized costs

21

COMPARISONWITH BEST RESULTS(at the time of writing)

Instance Our Best ByNL10 59,583 59,436 [1]NL12 111,483 111,248 [2]NL14 190,174 189,766 [2]NL16 270,063 267,194 [2]CIRC10 242 242 [1]CIRC12 426 408 [3]CIRC14 668 654 [3]CIRC16 1,004 928 [3]CIRC18 1,412 1,306 [4]CIRC20 1,946 1,842 [3]

Instance Our Best ByCON10 ∗124 — —CON12 ∗181 — —CON14 ∗252 — —CON16 328 ∗327 [5]CON18 418 — —CON20 521 — —CON22 632 — —CON24 757 — —BRA24 530,043 503,158 [4]

[1] Langford, [2] Anagnostopoulos et al. 2005, [3] Lim, Rodrigues,and Zhang 2005, [4] Araùjo et al. 2005, [5] Rasmussen and Trick2005∗ optimality proven by Rasmussen and Trick 2005

22

GENERALIZED LOCAL SEARCHMACHINES(H.H. Hoos 1999; Hoos and Stützle 2004)

▶ GLSM is a formal framework for representing Local Searchmethods: it is essentially a (rich) automaton

▶ states are the basic search components▶ transitionsmodel the search control▶ a transition takes place when the search component has

finished its execution and it is labeled with a pair 𝑐/𝑎▶ 𝑐: the condition needed to select the transition▶ 𝑎: the action performed on GLSM variables

13

• C1

C2

C3

C4

•

!/c" 0

!/c"

c+

1(c<

!)/!

c# !/!

!/c"

0

T > "/!

Fig. 3 An example of GLSM with three search components.

quality. In detail, if the cost of moving to the neighbor is # f , the move is acceptedwith probability e!# f/T , where T is a time-decreasing parameter called temperature.

Among the different decreasing strategies for the temperature (cooling sched-

ules), we adopt the geometric scheme, that is, after a number of explored moves thetemperature is multiplied by a factor ! < 1 (i.e., T " !T ). The search is stopped whenthe temperature goes under a value Tmin.

Dynamic Tabu Search (DTS) At each step a subset of the neighborhood is exploredand the neighbor with the best cost value becomes the new solution independently onwhether its cost value is better or worse than the current one. The subset is inducedby the tabu list, i.e., a list of the moves recently performed, which are currently for-bidden and thus excluded from exploration. Our tabu search implementation employsa dynamic short-term tabu list (called Robust Tabu Search in Taillard 1991), so that amove is kept in the tabu list for a random number of iterations in the range [kmin,kmax].

The tabu status of a move can be overruled by the so-called aspiration criterion

which makes a move acceptable even if it is tabu. In this work, we use a basic aspi-ration criterion which states that a move is accepted if it improves on the current bestsolution.

The dynamic variant of the algorithm (for TS, not SA) is equipped with a mecha-nism that adaptively changes the shape of the cost function. In detail, the constraintswhich are satisfied for a given number of iterations will be relaxed (their penalties arereduced by a factor $ , i.e., Pi" Pi/$ ) in order to allow the exploration of regions ofthe search space where those constraints do not hold. Conversely, if some constraintis not satisfied, it is tightened (Pi" Pi ·$ ) with the aim of driving the search towardits satisfaction.

Kickers (K) We call kickers search components that perform a single move in a com-posite neighborhood (Di Gaspero and Schaerf 2006). Kickers support three strategies

23

GENERALIZED LOCAL SEARCHMACHINESHybrid Metaheuristics Classification

HybridMetaheuristic

controlstrategy


spacedecomposition

integrative

orderof execution

parallel

interleaved

batch


(strong coupling)




heuristic methods

exacttechniques


Metaheuristics

24

THE GENERALIZED BALANCED ACADEMIC CURRICULUM PROBLEM

▶ Basic features of the BACP:▶ Courses, with number of credits▶ Periods, the planning horizon divided in years and terms▶ Load limits, i.e., min and max credits per period and min and

max courses per period▶ Precedences between courses

▶ Assign periods to courses satisfying load limits and precedences▶ Generalization (GBACP):

▶ Curricula, subgroups of courses (students' selection)▶ Preferences, undesired terms for courses

▶ minimize discrepancy from average load per curriculum,minimize courses in undesired terms

25

AN EXAMPLE OF GBACP

Term 1 Term 2

Year 1

Year 2

Year 3

Curricula: 𝑄1 = {CS1, CS3, DB1, SE1},𝑄2 = {CS1, CS2, DB2, SE2}Preferences: CS3 not in term 2

Course CreditsCS1 7CS2 6CS3 5DB1 7DB2 5SE1 4SE2 5… …

PrecedencesCS1 ≺ CS2CS2 ≺ CS3CS1 ≺ DB1… ≺ …

26

AN EXAMPLE OF GBACP

Term 1 Term 2

Year 1 CS1 CS2(7) (6)

Year 2 CS3 DB1(5) (7)

Year 3 DB2, SE1 SE2(9) (5)

Curricula: 𝑄1 = {CS1, CS3, DB1, SE1},𝑄2 = {CS1, CS2, DB2, SE2}Preferences: CS3 not in term 2

Course CreditsCS1 7CS2 6CS3 5DB1 7DB2 5SE1 4SE2 5… …

PrecedencesCS1 ≺ CS2CS2 ≺ CS3CS1 ≺ DB1… ≺ …

26

THE GENERALIZED BALANCED ACADEMIC CURRICULUM PROBLEMBasic GLSMs

Basic

15

• Gr R •

(a) Basic

• Gr R •

T > !/!

(b) Multi-start

• Gr R •

T > !/!

(c) Multi-run

• Gr R •

!/r" 0

f # fbest/r" r+1

f < fbest/r" 0r > "/!

T > !/!

(d) Multi-start-multi-run

Fig. 4 The basic GLSM templates.

Table 1 Statistics on the GBACP instances.

Instance Periods Courses Curricula Courses per Courses per Prerequ. Pref.(Years $Terms)

curriculum period

csplib8 8 (4 $ 2) 46 1 46 5.75 33 0csplib10 10 (5 $ 2) 42 1 42 4.2 33 0csplib12 12 (6 $ 2) 66 1 66 5.5 65 0

UD1 9 (3 $ 3) 307 37 34.62 3.847 1383 270UD2 6 (2 $ 3) 268 20 27.8 4.633 174 158UD3 9 (3 $ 3) 236 31 29.81 3.312 1092 198UD4 6 (2 $ 3) 139 16 25.69 4.281 188 80UD5 6 (3 $ 2) 282 31 34.32 5.72 397 162UD6 4 (2 $ 2) 264 20 27.15 6.787 70 110UD7 9 (3 $ 3) 302 37 33.89 3.766 1550 249UD8 6 (2 $ 3) 208 19 22.58 3.763 149 120UD9 9 (3 $ 3) 303 37 34.08 3.787 1541 255UD10 6 (2 $ 3) 188 15 25.07 4.178 214 110

5 Computational Results

5.1 Benchmark Instances

Ten instances for the GBACP, called UD1–UD10, have been extracted from thedatabase containing historical data at the School of Engineering of the Universityof Udine. As we faced three national regulations changes in ten years, we could buildthis dataset by selecting a number of structurally different instances. Table 1 sum-marizes the main features of the instances. For reference, we also include the threeCSPLib instances.

Multi-start

15

• Gr R •

(a) Basic

• Gr R •

T > !/!

(b) Multi-start

• Gr R •

T > !/!

(c) Multi-run

• Gr R •

!/r" 0

f # fbest/r" r+1

f < fbest/r" 0r > "/!

T > !/!





curriculum period






Multi-run

15

• Gr R •

(a) Basic

• Gr R •

T > !/!

(b) Multi-start

• Gr R •

T > !/!

(c) Multi-run

• Gr R •

!/r" 0

f # fbest/r" r+1

f < fbest/r" 0r > "/!

T > !/!





curriculum period






Multi-start-multi-run

15

• Gr R •

(a) Basic

• Gr R •

T > !/!

(b) Multi-start

• Gr R •

T > !/!

(c) Multi-run

• Gr R •

!/r" 0

f # fbest/r" r+1

f < fbest/r" 0r > "/!

T > !/!





curriculum period






27

THE GENERALIZED BALANCED ACADEMIC CURRICULUM PROBLEMComposite GLSMs (with a kicker)

𝑅1 ▷ 𝐾16

• Gr R •

!/r" 0

f # fbest/r" r+1

f < fbest/r" 0r > !/!

T > "/!

(a) R1

• Gr

R1

K

•

!/r" 0

r$

!/r"

r+

1

f<

f bes

t/r"

0

T > "/!

T > "/!r > !/!

(b) R1 !K

• Gr

R1

K

•

!/r" 0

r"

r+

1

f#

f bes

t%

r$

!/

T > "/!

T > "/!r > !/!

f < fbest/r" 0

(c) R1 !K+

• Gr

R1

R2

•

!/r" 0

r$

!/r"

r+

1

f<

f bes

t/r"

0

T > "/!

T > "/!r > !/!

(d) R1 !R2

• Gr

R1

R2

K

•

!/r" 0

r$

!/r"

r+

1

f<

f bes

t/r"

0

T > "/!

T > "/!

T > "/!r > !/!

(e) R1 !R2 !K

• Gr

R1

R2

K

•

!/r" 0

r"

r+

1

f#

f bes

t%

r$

!/

T > "/!

T > "/!

T > "/!r > !/!

f < fbest/r" 0

(f) R1 !R2 !K+

Fig. 5 The composite GLSM templates (b)-(f) obtained from the multi-start-multi-run machine (a).

𝑅1 ▷ 𝐾+

16

• Gr R •

!/r" 0

f # fbest/r" r+1

f < fbest/r" 0r > !/!

T > "/!

(a) R1

• Gr

R1

K

•

!/r" 0

r$

!/r"

r+

1

f<

f bes

t/r"

0

T > "/!

T > "/!r > !/!

(b) R1 !K

• Gr

R1

K

•

!/r" 0

r"

r+

1

f#

f bes

t%

r$

!/

T > "/!

T > "/!r > !/!

f < fbest/r" 0

(c) R1 !K+

• Gr

R1

R2

•

!/r" 0

r$

!/r"

r+

1

f<

f bes

t/r"

0

T > "/!

T > "/!r > !/!

(d) R1 !R2

• Gr

R1

R2

K

•

!/r" 0

r$

!/r"

r+

1

f<

f bes

t/r"

0

T > "/!

T > "/!

T > "/!r > !/!

(e) R1 !R2 !K

• Gr

R1

R2

K

•

!/r" 0

r"

r+

1

f#

f bes

t%

r$

!/

T > "/!

T > "/!

T > "/!r > !/!

f < fbest/r" 0

(f) R1 !R2 !K+


𝑅1 ▷ 𝑅2

16

• Gr R •

!/r" 0

f # fbest/r" r+1

f < fbest/r" 0r > !/!

T > "/!

(a) R1

• Gr

R1

K

•

!/r" 0

r$

!/r"

r+

1

f<

f bes

t/r"

0

T > "/!

T > "/!r > !/!

(b) R1 !K

• Gr

R1

K

•

!/r" 0

r"

r+

1

f#

f bes

t%

r$

!/

T > "/!

T > "/!r > !/!

f < fbest/r" 0

(c) R1 !K+

• Gr

R1

R2

•

!/r" 0

r$

!/r"

r+

1

f<

f bes

t/r"

0

T > "/!

T > "/!r > !/!

(d) R1 !R2

• Gr

R1

R2

K

•

!/r" 0

r$

!/r"

r+

1

f<

f bes

t/r"

0

T > "/!

T > "/!

T > "/!r > !/!

(e) R1 !R2 !K

• Gr

R1

R2

K

•

!/r" 0

r"

r+

1

f#

f bes

t%

r$

!/

T > "/!

T > "/!

T > "/!r > !/!

f < fbest/r" 0

(f) R1 !R2 !K+


𝑅1 ▷ 𝑅2 ▷ 𝐾+

16

• Gr R •

!/r" 0

f # fbest/r" r+1

f < fbest/r" 0r > !/!

T > "/!

(a) R1

• Gr

R1

K

•

!/r" 0

r$

!/r"

r+

1

f<

f bes

t/r"

0

T > "/!

T > "/!r > !/!

(b) R1 !K

• Gr

R1

K

•

!/r" 0

r"

r+

1

f#

f bes

t%

r$

!/

T > "/!

T > "/!r > !/!

f < fbest/r" 0

(c) R1 !K+

• Gr

R1

R2

•

!/r" 0

r$

!/r"

r+

1

f<

f bes

t/r"

0

T > "/!

T > "/!r > !/!

(d) R1 !R2

• Gr

R1

R2

K

•

!/r" 0

r$

!/r"

r+

1

f<

f bes

t/r"

0T > "/!

T > "/!

T > "/!r > !/!

(e) R1 !R2 !K

• Gr

R1

R2

K

•

!/r" 0

r"

r+

1

f#

f bes

t%

r$

!/

T > "/!

T > "/!

T > "/!r > !/!

f < fbest/r" 0

(f) R1 !R2 !K+


28

ALGORITHM SETTING

▶ The GLSM where equipped with a Simulated Annealing and aDynamic Tabu Search runner

▶ DTS features an adaptive weighting strategy for the costfunction

▶ the neighborhood considered areMoveCourse (MC): move one course from its period to

another oneSwapCourses (SC): take two courses in different periods and

swap their periods▶ As for the kickerwe consider the intensification kicker relying

on the moves MC, SC and MC⊗SC.

29

CONFIGURATIONS SELECTION

▶ We first tune each algorithm in isolation▶ Afterwards we equip the GLSMs with the best tuning found in

the previous phase▶ Kickers have been tuned in conjunction with both SA and DTS

multi-run multi-start machines.

30

BEST RESULTSComparison with an exact MILP model

Table: Solutions within a timeout of 320s

MILP 𝑆𝐴 ▷ 𝐾+(𝑀𝐶 ⊗ 𝑆𝐶)𝑏Inst. LB LB2 cost (𝐹 ) (gap) best (gap) median (gap)UD1 10.54 111 952 (192.7) 276 (57.6) 284.5 (60.1)UD2 11.40 130 177 (16.7) 148 (6.7) 156.0 (9.5)UD3 10.30 106 651 (147.7) 163 (24.0) 171.5 (27.2)UD4 18.00 324 396 (10.6) 396 (10.5) 396.0 (10.5)UD5 11.50 3 771 (141.5) 219 (28.6) 230.0 (31.8)UD6 5.92 35 52 (21.8) 55 (25.3) 62.5 (33.6)UD7 10.95 120 -- (---) 214 (33.5) 240.0 (41.4)UD8 5.20 27 64 (53.9) 46 (30.5) 53.0 (40.1)UD9 11.66 136 1895 (273.3) 223 (28.0) 234.0 (31.1)UD10 6.16 38 63 (28.9) 48 (12.3) 54.50 (19.7)

31

BEST RESULTSComparison with an exact MILP model

Table: Solutions within a timeout of 3600s

MILP 𝑆𝐴 ▷ 𝐾+(𝑀𝐶 ⊗ 𝑆𝐶)𝑏Inst. LB LB2 cost (𝐹 ) (gap) best (gap) median (gap)UD1 10.54 111 366 (81.5) 267 (55.0) 271.5 (56.3)UD2 11.40 130 164 (12.3) 148 (6.7) 149.0 (7.1)UD3 10.30 106 214 (42.0) 161 (23.2) 164.0 (24.3)UD4 18.00 324 396 (10.5) 396 (10.5) 396.0 (10.5)UD5 11.50 3 492 (92.8) 213 (26.9) 217.0 (28.1)UD6 5.92 35 52 (21.8) 55 (25.3) 58.0 (28.6)UD7 10.95 120 634 (129.8) 218 (34.8) 223.0 (36.4)UD8 5.20 27 64 (53.9) 42 (24.6) 46.5 (31.1)UD9 11.66 136 862 (151.7) 210 (24.3) 218.0 (26.6)UD10 6.16 38 48 (12.3) 47 (11.3) 49.5 (14.2)

31

HYBRIDIZATION WITH EXACT METHODS

Problem Decomposition

PROBLEM DECOMPOSITIONBi-level problem

▶ If the optimization problem can be decomposed as:

min��,��

𝑓( 𝑥, 𝑦)𝑦 ∈ ℱ ⊆ ℤ𝑚

𝑥 ∈ 𝒢( 𝑦) ⊆ ℝ𝑛

▶ then, the 𝑦 and 𝑥 variables can be dealt with different methods▶ a metaheuristic can be more adequate for the discrete part▶ ad-hoc methods could be applied to the continuous part

▶ once the 𝑦 variables are set to 𝑦∗, the problem reduces to

min��

𝑓′( 𝑥)𝑥 ∈ 𝒢( 𝑦∗) ⊆ ℝ𝑛

where 𝑓′( 𝑥) = 𝑓( 𝑥, 𝑦∗)34

HYBRIDIZATION BY PROBLEM DECOMPOSITIONHybrid Metaheuristics Classification

HybridMetaheuristic

controlstrategy


spacedecomposition

integrative

orderof execution

parallel

interleaved

batch


(strong coupling)




heuristic methods

exacttechniques


Metaheuristics

35

THE PORTFOLIO SELECTION PROBLEMUnconstrained

▶ The classical model of investments (Markowitz 1952):▶ a set of 𝑛 assets 𝑎1, … , 𝑎𝑛▶ each asset 𝑎𝑖 has associated a return 𝑟𝑖 and a variance 𝜎𝑖▶ for each pair of assets (𝑎𝑖, 𝑎𝑗) we know the covariance 𝜎𝑖𝑗 of

the two assets▶ a total amount of money (normalized to 1)

▶ A portfolio is a vector of real values 𝑋 = {𝑥1, … , 𝑥𝑛}, suchthat:

▶ 𝑥𝑖 represents the fraction of money invested in the asset 𝑎𝑖

▶𝑛

∑𝑖=1

𝑟𝑖𝑥𝑖 is the return (or gain) to be maximized

▶𝑛

∑𝑖=1

𝑛∑𝑗=1

𝜎𝑖𝑗𝑥𝑖𝑥𝑗 is the variance → risk to be minimized

36

THE PORTFOLIO SELECTION PROBLEMConstrained

�� 𝑓(𝑋) =𝑛

∑𝑖=1

𝑛∑𝑗=1

𝜎𝑖𝑗𝑥𝑖𝑥𝑗

𝑠.𝑡.𝑛

∑𝑖=1

𝑟𝑖𝑥𝑖 ≥ 𝑅

𝑛∑𝑖=1

𝑥𝑖 = 1

0 ≤ 𝑥𝑖 ≤ 1 (𝑖 = 1, … , 𝑛)

▶ Cardinality constraint: the portfolio size is bounded1 ≤ 𝑘𝑚𝑖𝑛 ≤ ∑𝑖 𝑧𝑖 ≤ 𝑘𝑚𝑎𝑥 ≤ 𝑛

▶ Quantity constraint: the quantity of each asset is bounded𝑥𝑖 = 0 ∨ 𝜖𝑖 ≤ 𝑥𝑖 ≤ 𝛿𝑖 (0 ≤ 𝜖𝑖 ≤ 𝛿𝑖 ≤ 1)

37

PROBLEM DECOMPOSITION

▶ Problem decomposition:▶ Master solver: determine set 𝐽 = {𝑖|𝑧𝑖 = 1} of assets in the

solution▶ Slave solver: find optimal assignment with the constraints:

𝜖𝑖 ≤ 𝑥𝑖 ≤ 𝛿𝑖 for 𝑖 ∈ 𝐽𝑥𝑖 = 0 for 𝑖 ∉ 𝐽

▶ Solution procedure:▶ Master: simple Local Search on 𝑧 variables (First Descent and

Steepest Descent)▶ Slave: Quadratic Programming solver (positive definite

problem with fewer assets)

38

CONSTRAINED EFFICIENT FRONTIER

0.0005 0.0010 0.0015 0.0020 0.0025

0.0

02

0.0

03

0.0

04

0.0

05

0.0

06

0.0

07

0.0

08

0.0

09

Risk

Re

turn

UEF

kmax = 2

kmax = 4

kmax = 8

kmax = 16

kmax = 32

ORLib Instance 4

39

COMPARISONWITH OTHER METAHEURISTICS

Table: Comparison with (1) Schaerf and (2) Moral-Escudero et al.

FD+QP SD+QP TS(1) GA+QP(2)

Inst. min APL time min APL time min APL time min APL time1 0.00366 1.5s 0.00321 3.1s 0.00409 251s 0.00321 415.1s2 2.66104 9.6s 2.53139 14.1s 2.53617 531s 2.53180 552.7s3 2.00146 10.1s 1.92146 16.1s 1.92597 583s 1.92150 886.3s4 4.77157 11.2s 4.69371 18.8s 4.69816 713s 4.69507 1163.7s5 0.24176 25.3s 0.20219 45.9s 0.20258 1603s 0.20198 1465.8s

40

COMPARISONWITH OTHER METAHEURISTICS

Table: Comparison with (3) Crama and Schyns

FD+QP SD+QP SA(3)

Inst. APL time APL time APL timeS1 0.72 (0.094) 0.3s 0.35 (0.0) 1.4s 1.13 (0.13) 3.2sS2 1.79 (0.22) 0.5s 1.48 (0.0) 3.1s 3.46 (0.17) 5.4sS3 10.50 (0.51) 10.2s 8.87 (0.003) 53.3s 16.12 (0.43) 30.1s

40

Large Neighborhood Search

LARGE NEIGHBORHOOD SEARCH

▶ Local Search with a particular▶ neighborhood relation: destroy and repair a significant portion

of a solution (controlled by an intensity parameter 𝑑)▶ exploration strategy: use an exact method (e.g., Constraint

Programming) for the repair phase, optimizing the subproblem(Pisinger and Ropke 2010; Shaw 1998)procedure LargeNeighborhoodSearch(Π = ⟨𝑋, 𝐷, 𝐶, 𝑓⟩, 𝑑)

𝑖 ← 0𝑠 ← InitializeSolution(Π)while ¬StoppingCriterion(𝑠, 𝑖) do

Π′ ← Destroy(𝑠, 𝑑, Π)𝑛 ← Repair(Π′, 𝑓(𝑠))if SolutionAccepted(𝑛) then

𝑠 ← 𝑛end if𝑖 ← 𝑖 + 1


▶ the destroy step can be either random or based on the problemstructure (i.e., decomposition)

▶ reuse of the CP/ILP model42

HYBRIDIZATION BY LARGE NEIGHBORHOOD SEARCHHybrid Metaheuristics Classification

HybridMetaheuristic

controlstrategy


spacedecomposition

integrative

orderof execution

parallel

interleaved

batch


(strong coupling)




heuristic methods

exacttechniques


Metaheuristics

43

GELATO

GELATO

▶ Integrates Constraint Programming and Local Search in asingle Large Neighborhood Search environment

▶ Inherits both the flexibility of CP and the efficiency of LocalSearch

▶ Different point of views:▶ Gecode point of view: a library that adds Local Search features▶ EasyLocal++ point of view: a library that adds Constraint

Programming features▶ Global point of view: an environment to model problems and

hybrid solving strategies using high-level language and tosearch for solutions using Gecode and EasyLocal++ as low levelsolvers

▶ Reuse of the Constraint Programming model for the LargeNeighborhood Search

▶ Reuse of the EasyLocal++ Local Search metaheuristics

45

ARCHITECTURE OVERVIEW

46

RESULTSATSP and MEB problems

0 5 10 15

2000

2500

3000

3500

4000

4500

5000

ATSP − Instance 3

Time (s)

FO

bj

GECODEELCOMETGELATO

0 10 20 30 40 5050

0000

1000

000

1500

000

2000

000

2500

000

MEB − Instance 20.24

Time (s)

FO

bj

GECODEELCOMETGELATO

47

CONCLUSIONS

CONCLUSIONS

▶ An overview of my research on Hybrid Local Search-basedMetaheuristics

▶ Covers different hybridization methods▶ Other studies on hybridization with different metaheuristics

▶ Two-level ACO for Haplotype Inference under pure parsimony(Benedettini, Roli, and Di Gaspero 2008)

▶ A Hybrid ACO+CP for Balancing Bicycle Sharing Systems(Di Gaspero, Rendl, and Urli 2013)

▶ Further studies on different problems▶ A CP/LNS Approach for Multi-day Homecare Scheduling

Problems (Di Gaspero and Urli 2014)▶ Future work

▶ Working on collaborativemethods▶ Investigating better LNS decomposition approaches

49