martin grötschel institute of mathematics, technische universität berlin (tub) dfg-research...

108
Martin Grötschel Institute of Mathematics, Technische Universität Berlin (TUB) DFG-Research Center “Mathematics for key technologies” (MATHEON) Konrad-Zuse-Zentrum für Informationstechnik Berlin (ZIB) [email protected] http://www.zib.de/groetschel The Travelling Salesman Problem: A brief survey Martin Grötschel Vorausschau auf die Vorlesung Das Travelling-Salesman-Problem (ADM III) im WS 2013/14 14. Oktober 2013

Upload: rudolf-mason

Post on 17-Dec-2015

216 views

Category:

Documents


2 download

TRANSCRIPT

Page 1: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel Institute of Mathematics, Technische Universität Berlin (TUB) DFG-Research Center “Mathematics for key technologies” (MATHEON) Konrad-Zuse-Zentrum für Informationstechnik Berlin (ZIB) [email protected] http://www.zib.de/groetschel

The Travelling Salesman Problem:

A brief survey

Martin GrötschelVorausschau auf die Vorlesung

Das Travelling-Salesman-Problem (ADM III)

im WS 2013/14

14. Oktober 2013

Page 2: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

2

Contents

1. Introduction

2. The TSP and some of its history

3. The TSP and some of its variants

4. Some applications

5. Modeling issues

6. Heuristics

7. How combinatorial optimizers do it

Page 3: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

3

Contents

1. Introduction

2. The TSP and some of its history

3. The TSP and some of its variants

4. Some applications

5. Modeling issues

6. Heuristics

7. How combinatorial optimizers do it

Page 4: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

4

Given a finite set E and a subset I of the power set of E (the set of feasible solutions). Given, moreover, a value (cost, length,…) c(e) for all elements e of E. Find, among all sets in I, a set I such that its total value c(I) (= sum of the values of all elements in I) is as small (or as large) as possible.

The parameters of a combinatorial optimization problem are: (E, I, c).

An important issue: How is I given?

Combinatorial optimization

I

min (I) ( ) | I , 2E

e

c c e I where I and E finite

Page 5: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

5Special „simple“ combinatorial optimization problems

Finding a

minimum spanning tree in a graph

shortest path in a directed graph

maximum matching in a graph

minimum capacity cut separating two given nodes of a graph or digraph

cost-minimal flow through a network with capacities and costs on all edges

These problems are solvable in polynomial time.

Page 6: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

6Special „hard“ combinatorial optimization problems travelling salesman problem (the prototype problem)

location und routing

set-packing, partitioning, -covering

max-cut

linear ordering

scheduling (with a few exceptions)

node and edge colouring

These problems are NP-hard (in the sense of complexity theory).

Page 7: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

7

The travelling salesman problem

Given n „cities“ and „distances“ between them. Find a tour (roundtrip) through all cities visiting every city exactly once such that the sum of all distances travelled is as small as possible. (TSP)

The TSP is called symmetric (STSP) if, for every pair of cities i and j, the distance from i to j is the same as the one from j to i, otherwise the problem is called asymmetric (ATSP).

Page 8: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

http://www.tsp.gatech.edu/

Page 9: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

9

THE TSPbook

suggested reading for everyone interested in the TSP

Page 10: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Another recommendationBill Cook‘s new book

Martin Grötschel

10

Page 11: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

11

The travelling salesman problem

1. :

( , )

. H

( ) .

min{ ( ) | H}.

2. :

{1,..., }

n

e

n

Version

Let K V E be the complete graph or digraphwith n nodes

and let c be the length of e E Let be the set of all

hamiltonian cycles tours in K Find

c T T

Version

Find a cyclic permutation of n such that

c

( )1

.

n

i ii

is as small as possible

Two mathematical formulations of the TSP

Does that help solve the TSP?

Page 12: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

12

Contents

1. Introduction

2. The TSP and some of its history

3. The TSP and some of its variants

4. Some applications

5. Modeling issues

6. Heuristics

7. How combinatorial optimizers do it

Page 13: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

13

Usually quoted as

the forerunner of

the TSP

Usually quoted as

the origin of

the TSP

Page 14: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

about 100yearsearlier

Page 15: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

15

By a proper choice andscheduling of the tour onecan gain so much time that we have to makesome suggestions

The most important aspect is to cover as many locations as possiblewithout visiting alocation twice

Page 16: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

A TSP contest 1962: 10.000 $ Prize

Martin Grötschel

16

Page 17: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

17

Ulysses roundtrip (an even older TSP ?)

The paper „The Optimized Odyssey“ by Martin Grötschel and Manfred Padberg is downloadable from http://www.zib.de/groetschel/pubnew/paper/groetschelpadberg2001a.pdf

Page 18: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

18

Ulysses

The distance table

Page 19: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

19

Ulysses roundtrip

optimal „Ulysses tour“

Page 20: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

20

Malen nach ZahlenTSP in art ?

When was this invented?

Page 21: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

21

Survey BooksLiterature: more than 1000 entries in Zentralblatt/Math

Zbl 0562.00014 Lawler, E.L.(ed.); Lenstra, J.K.(ed.); Rinnooy Kan, A.H.G.(ed.); Shmoys, D.B.(ed.)The traveling salesman problem. A guided tour of combinatorial optimization. Wiley-Interscience Series in Discrete Mathematics. A Wiley-Interscience publication. Chichester etc.: John Wiley \& Sons. X, 465 p. (1985). MSC 2000: *00Bxx 90-06

Zbl 0996.00026 Gutin, Gregory (ed.); Punnen, Abraham P.(ed.)The traveling salesman problem and its variations. Combinatorial Optimization. 12. Dordrecht: Kluwer Academic Publishers. xviii, 830 p. (2002). MSC 2000: *00B15 90-06 90Cxx

Page 22: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

22

Contents

1. Introduction

2. The TSP and some of its history

3. The TSP and some of its variants

4. Some applications

5. Modeling issues

6. Heuristics

7. How combinatorial optimizers do it

Page 23: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

23

The Travelling Salesman Problem and Some of its

Variants The symmetric TSP The asymmetric TSP The TSP with precedences or time windows The online TSP The symmetric and asymmetric m-TSP The price collecting TSP The Chinese postman problem

(undirected, directed, mixed) Bus, truck, vehicle routing Edge/arc & node routing with capacities Combinations of these and more

Page 24: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

24http://www.densis.fee.unicamp.br/~moscato/TSPBIB_home.html

Page 25: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

25

Contents

1. Introduction

2. The TSP and some of its history

3. The TSP and some of its variants

4. Some applications

5. Modeling issues

6. Heuristics

7. How combinatorial optimizers do it

Page 26: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

26

Production of ICs and PCBs

Integrated Circuit (IC) Printed Circuit Board (PCB)

Problems: Logical Design, Physical DesignCorrectness, Simulation, Placement of

Components, Routing, Drilling,...

Page 27: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

27Correct modelling of a printed circuit board drilling problem

2103 holes to be drilled

length of a move of the drilling head:Euclidean norm,Max norm,Manhatten norm?

Page 28: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

28

Drilling 2103 holes into a PCB

Significant Improvementsvia TSP

(due to Padberg & Rinaldi)

industry solution optimal solution

Page 29: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Siemens-ProblemPCB da4

before after

Martin Grötschel, Michael Jünger, Gerhard Reinelt,Optimal Control of Plotting and Drilling Machines: A Case Study, Zeitschrift für Operations Research, 35:1 (1991) 61-84http://www.zib.de/groetschel/pubnew/paper/groetscheljuengerreinelt1991.pdf

Page 30: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Siemens-Problem PCB da1

before after

Grötschel, Jünger, Reinelt

Page 31: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

31

Page 32: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

32Leiterplatten-BohrmaschinePrinted Circuit Board Drilling Machine

Page 33: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

33

Foto einer Flachbaugruppe (Leiterplatte)

Page 34: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

34

Foto einer Flachbaugruppe (Leiterplatte) - Rückseite

Page 35: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

35

442 holes to be drilled

Page 36: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

36

Typical PCB drilling problems at Siemens

da1 da2 da3 da4

Number of holes

Number of drills

Tour length

2457

7

3518728

423

7

1049956

2203

6

1958161

2104

10

4347902

Table 4

Page 37: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

37

Fast heuristics

da1 da2 da3 da4

CPU time (min:sec)

Tour length

Improvement in %

1:58

1695042

56.87

0:05

984636

14.60

1:43

1642027

26.94

1:43

1928371

58.38

Table 5

Page 38: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

38

Optimizing the stacker cranes of a Siemens-Nixdorf warehouse

Page 39: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

39

Herlitz at Falkensee (Berlin)

Page 40: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

40

Example: Control of the stacker cranes in a Herlitz

warehouse

Page 41: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

41

Andrea Grötschel Diplomarbeit (2004)

Logistics of collectingelectronics garbage

Page 42: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

42 Location plus tour planning (m-TSP)

Page 43: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

43

The Dispatching Problem at ADAC:an online m-TSP

Dispatching Center (Pannenzentrale)

Dispatcher

Data Transm.

„Gelber Engel“

Page 44: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Online-TSP (in a metric space)

Instance:

1 2, , , nr r r ( , )i i ir t xwhere

1x

1t t0

1x

2t t0

2x

Goal:Find fastest tour serving all requests (starting and ending in 0)

Algorithm ALG is c-competitive if

ALG OPTc for all request sequences

Page 45: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

45

Implementation competitions

Page 46: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

46

Contents

1. Introduction

2. The TSP and some of its history

3. The TSP and some of its variants

4. Some applications

5. Modeling issues

6. Heuristics

7. How combinatorial optimizers do it

Page 47: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

47

LP Cutting Plane Approach

Even MODELLING is not easy!

What is the „right“ LP relaxation?

N. Ascheuer, M. Fischetti, M. Grötschel,„Solving the Asymmetric Travelling Salesman

Problem with time windows by branch-and-cut“,Mathematical Programming A (2001), see

http://www.zib.de/groetschel/pubnew/paper/ascheuerfischettigroetschel2001.pdf

Page 48: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

48

min

( ( )) 1 0

( ( )) 1 0

( ( )) | | 1 0 ,2 | |

0,1 ( , ) .

T

ij

c x

x i i V

x i i V

x A W W W V W n

x i j A

IP formulation of the asymmetric TSP

Page 49: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Time Windows

This is a typical situation in delivery problems.

Customers must be served during a certain period of time, usually a time interval is given. access to pedestrian areas

opening hours of a customer

delivery to assembly lines

just in time processesMartin Grötschel

49

Page 50: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

50

min

( ( )) 1 0

( ( )) 1 0

(1 ) , , 0

0

0,1 , .

T

i ij ij j

i i i

i

ij

c x

x i i V

x i i V

t x M t i j A j

r t d i V

t i V

x i j A

N

Model 1

Page 51: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

51

1 2 )

min

( ( )) 1 0

( ( )) 1 0

( ( )) | | 1 0 ,2 | |

( ) | | 1 2 infeasible path ( , , ,

0,1 ( , ) .

T

k

ij

c x

x i i V

x i i V

x A W W W V W n

x P P k P v v v

x i j A

Model 2

Page 52: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

52

1 0 1

min

( ( )) 1 0

( ( )) 1 0

, 0, , , , 0

0,1 ( , )

0,1,2,... ( , )

T

n n n

ij ij ij jki i ki j i j k j

i ij ij i ij

ij

ij

c x

x i i V

x i i V

y x y j V

r x y d x i j n i j i

x i j A

y i j A

Model 3

Page 53: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

min

( ( )) 1 0

( ( )) 1 0

(1 ) , , 0

0

0,1 , .

T

i ij ij j

i i i

i

ij

c x

x i i V

x i i V

t x M t i j A j

r t d i V

t i V

x i j A

N

Model 1, 2, 3

1 2 )

min

( ( )) 1 0

( ( )) 1 0

( ( )) | | 1 0 ,2 | |

( ) | | 1 2 infeasible path ( , , ,

0,1 ( , ) .

T

k

ij

c x

x i i V

x i i V

x A W W W V W n

x P P k P v v v

x i j A

1 0 1

min

( ( )) 1 0

( ( )) 1 0

, 0, , , , 0

0,1 ( , )

0,1,2,... ( , )

T

n n n

ij ij ij jki i ki j i j k j

i ij ij i ij

ij

ij

c x

x i i V

x i i V

y x y j V

r x y d x i j n i j i

x i j A

y i j A

Page 54: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

54

Cutting Planes Used for all Three Models (Separation Routines)

Subtour Elimination Constraints (SEC) 2-Matching Constraints -Inequalities "Special“ Inequalities and PCB-Inequalities Dk-Inequalities Infeasible Path Elimination Constraints (IPEC) Strengthened -Inequalities Two-Job Cuts Pool Separation SD-Inequalities + various strengthenings/liftings

, ,( , )

,

Page 55: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

55

Further Implementation Details

Preprocessing Tightening Time Windows Release and Due Date Adjustment Construction of Precedences Elimination of Arcs

Branching (only on x-variables) Enumeration Strategy (DFS, Best-FS) Pricing Frequency (every 5th iteration) Tailing Off LP-exploitation Heuristics (after a new

feasible LP solution is found),

they outperform the other heuristics

Page 56: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

56

Results

Very uneven performance

Model 1 is really bad in general

Model 2 is best on the average (winner in 16 of 22 test cases)

Model 3 is better when few time windows are active (6 times winner, last in all other cases, severe numerical problems, very difficult LPs)

How could you have guessed?

Page 57: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

57

Unevenness of Computational Results

problem #nodes gap #cutting planes

#LPs time

rbg041a

43 9.16% > 1 mio 109,402 > 5 h

rbg067a

69 0% 176 2 6 sec

Largest problem solved to optimality: 127 nodesLargest problem not solved optimally: 43 nodes

Page 58: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

58

Contents

1. Introduction

2. The TSP and some of its history

3. The TSP and some of its variants

4. Some applications

5. Modeling issues

6. Heuristics

7. How combinatorial optimizers do it

Page 59: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

59

Need for Heuristics Many real-world instances of hard combinatorial

optimization problems are (still) too large for exact algorithms.

Or the time limit stipulated by the customer for the solution is too small.

Therefore, we need heuristics!

Exact algorithms usually also employ heuristics.

What is urgently needed is a decision guide:

Which heuristic will most likely work well on what problem ?

Page 60: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

60

Primal and Dual Heuristics

Primal Heuristic: Finds a (hopefully) good feasible solution.

Dual Heuristic: Finds a bound on the optimum solution value (e.g., by finding a feasible solution of the LP-dual of an LP-relaxation of a combinatorial optimization problem).

Minimization:

dual heuristic value ≤ optimum value ≤ primal heuristic value

quality guaranteein practice and theory

Page 61: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

61

Heuristics: A Survey Greedy Algorithms

Exchange & Insertion Algorithms

Neighborhood/Local Search

Variable Neighborhood Search, Iterated Local Search

Random sampling

Simulated Annealing

Taboo search

Great Deluge Algorithms

Simulated Tunneling

Neural Networks

Scatter Search

Greedy Randomized Adaptive Search Procedures

Page 62: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

62

Heuristics: A Survey Genetic, Evolutionary, and similar Methods

DNA-Technology

Ant and Swarm Systems

(Multi-) Agents

Population Heuristics

Memetic Algorithms (Meme are the “missing links” gens and mind)

Fuzzy Genetics-Based Machine Learning

Fast and Frugal Method (Psychology)

Method of Devine Intuition (Psychologist Thorndike)

…..

Page 63: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

The typical heuristics junk

Hyper-heuristics in Co-operative SearchThe interest in parallel co-operative approaches has risen considerably due to,

not only the availability of co-operative environments at low cost, but also their success to provide novel ways to combine different (meta-)heuristics. Current research has shown that the parallel execution and co-operation of several (meta-)heuristics could improve the quality of the solutions that each of them would be able to find by itself working on a standalone basis. Moreover, parallel and distributed approaches can be used to provide more powerful and robust problem solving environments in a variety of problem domains. Hyper-heuristics, on the other hand, represent a set of search methodologies which are applicable to different problem domains. They aim to raise the level of generality, for example by choosing and/or generating new methodologies on demand during the search process. The goal of this study is to explore the cooperative search mechanisms within a hyper-heuristic framework. This exciting research area lies at the interface between operational research and computer science and involves understanding of (distributed) decision making mechanisms and learning, design, implementation and analysis of automated search methodologies. The application domains will be cross disciplinary.

Martin Grötschel

63

Page 64: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

64

Heuristics: A SurveyCurrently best heuristic with respect to worst-case guarantee:

Christofides heuristic

compute shortest spanning tree

compute minimum perfect 1-matching of graph induced by the odd nodes of the minimum spanning tree

the union of these edge sets is a connected Eulerian graph

turn this graph into a tour by making short-cuts.

For distance functions satisfying the triangle inequality, the resulting tour is at most 50% above the optimum value

Page 65: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

65

Understanding Heuristics, Approximation Algorithms worst case analysisworst case analysis

There is no polynomial time approx. algorithm for STSP/ATSP. Christofides algorithm for the STSP with triangle inequality

average case analysisaverage case analysis Karp‘s analysis of the patching algorithm for the ATSP

probabilistic problem analysisprobabilistic problem analysis for Euclidean STSP in unit square, TSP constant 1.714..

polynomial time approximation schemes (PAS)polynomial time approximation schemes (PAS) Arora‘s polynomial-time approximation schemes for

Euclidean STSPs fully-polynomial time approximation schemes (FPAS)fully-polynomial time approximation schemes (FPAS)

not for TSP/ATSP but, e.g., for knapsack (Ibarra&Kim)

These concepts – unfortunately – often do not really help to guide practice.

experimental evaluationexperimental evaluation Lin-Kernighan for STSP (DIMACS challenges))

n

Page 66: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

66

Contents

1. Introduction

2. The TSP and some of its history

3. The TSP and some of its variants

4. Some applications

5. Modeling issues

6. Heuristics

7. How combinatorial optimizers do it

Page 67: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

67

Polyhedral Theory (of the TSP)

STSP-, ATSP-,TSP-with-side-constraints-

Polytope:= Convex hull of all incidence

vectors of feasible tours

To be investigated:

Dimension

Equation system defining the affine hull

Facets

Separation algorithms

Page 68: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

68

The symmetric travelling salesman polytope

|: { } ( 1 , 0)

{ | ( ( )) 2

( ( )) | | 1 \ 1 ,3 | | 3

0 1 }

min

( ( )) 2

( ( )) | | 1 \ 1 ,3 | | 3

0,1

n T E TT n ij

E

ij

T

ij

T tour in KQ conv if ij T else

x x i i V

x E W W W V W n

x ij E

c x

x i i V

x E W W W V W n

x ij E

Z

R

The LP relaxation is solvable in polynomial time

Page 69: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

69

Relation between IP and LP-relaxation

Open Problem:

If costs satisfy the triangle inequality, then

IP-OPT <= 4/3 LP-SEC

IP-OPT <= 3/2 LP-SEC (Wolsey)

Page 70: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

70

General cutting plane theory:Gomory Mixed-Integer Cut Given and

Rounding: Where define

Then

Disjunction:

Combining

, ,jy x ¢

, 0ij jy a x d d f f ,ij ij ja a f

: :ij j j ij j jt y a x f f a x f f ¢

: 1 :j j j j j jf x f f f x f f d t

:

1 : 1

j j j

j j j

t d f x f f f

t d f x f f f

: 1 1 : 1j j j j j jf f x f f f f x f f

Page 71: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

clique trees

A clique tree is a connected graph C=(V,E), composed of cliques satisfying the following properties

Martin Grötschel

71

Page 72: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

72

Polyhedral Theory of the TSPComb inequality

2-matchingconstraint

toothhandle

Page 73: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

73

Clique Tree Inequalities

Page 74: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

74 Clique Tree Inequalities

http://www.zib.de/groetschel/pubnew/paper/groetschelpulleyblank1986.pdf

1 1 1

1 1 1 1

( ( )) ( ( )) | | 2

1( ( )) ( ( )) | | (| | )

2

h h

i i

h h

i

i j i

i

t

j i i

j

t

j ji

t

jT

H

H

H hT t

tT t

x

x H

x

E x E

Hi, i=1,…,h are the handles

Tj, j=1,…,t are the teeth

tj is the number of handles that tooth Tj intersects

Page 75: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

75

Valid Inequalities for STSP Trivial inequalities Degree constraints Subtour elimination constraints 2-matching constraints, comb inequalities Clique tree inequalities (comb) Bipartition inequalities (clique tree) Path inequalities (comb) Star inequalities (path) Binested Inequalities (star, clique tree) Ladder inequalities (2 handles, even # of

teeth) Domino inequalities Hypohamiltonian, hypotraceable inequalities etc.

Page 76: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

76

A very special case

Petersen graph, G = (V, F),

the smallest hypohamiltonian graph

10( ) 9

, 11

T

nT

x F defines a facet of Q

but not a facet of Q n

M. Grötschel & Y. Wakabayashi

Page 77: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Hypotraceable graphs and the STSP

On the right is the smallestknown hypotraceable graph(Thomassen graph, 34 nodes).

Such graphs have no hamiltonian path, but when any node is deleted, theremaining graph has ahamiltonian path.

How do such graphs induceinequalities valid for thesymmetric travelling salesmanpolytope?

For further information see:http://www.zib.de/groetschel/pubnew/paper/groetschel1980b.pdfMartin

Grötschel

77

Page 78: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

“Wild facets of the asymmetric travelling salesman polytope”

Hypohamiltonian and hypotraceable directed graphs also exist and induce facets of the polytopes associated with the asymmetric TSP.

Information “hypohamiltonian” and “hypotraceable” inequalities can be found inhttp://www.zib.de/groetschel/pubnew/paper/groetschelwakabayashi1981a.pdfhttp://www.zib.de/groetschel/pubnew/paper/groetschelwakabayashi1981b.pdf

Martin Grötschel

78

Page 79: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

79Valid and facet defining

inequalities for STSP: Survey articles

M. Grötschel, M. W. Padberg (1985 a, b)

M. Jünger, G. Reinelt, G. Rinaldi (1995)

D. Naddef (2002)

The TSP book (ABCC, 2006)

Page 80: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

80

Counting Tours and Facets

n # tours # different facets # facet classes

3 1 0 0

4 3 3 1

5 12 20 2

6 60 100 4

7 360 3,437 6

8 2520 194,187 24

9 20,160 42,104,442 192

10 181,440 >= 52,043,900,866 >=15,379

Page 81: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

81

Separation Algorithms Given a system of valid inequalities

(possibly of exponential size).

Is there a polynomial time algorithm (or a good heuristic) that, given a point,

checks whether the point satisfies all inequalities of the system, and

if not, finds an inequality violated by the given point?

Page 82: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

82

Separation

K

Grötschel, Lovász, Schrijver (GLS):“Separation and optimizationare polynomial time equivalent.”

Page 83: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

83

Separation Algorithms There has been great success in finding exact

polynomial time separation algorithms, e.g., for subtour-elimination constraints

for 2-matching constraints (Padberg&Rao, 1982)

or fast heuristic separation algorithms, e.g., for comb constraints

for clique tree inequalities

and these algorithms are practically efficient

Page 84: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

84

Polyhedral Combinatorics

This line of research has resulted in powerful cutting plane algorithms for combinatorial optimization problems.

They are used in practice to solve exactly or approximately (including branch & bound) large-scale real-world instances.

Page 85: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

85

Deutschland 15,112

D. Applegate, R.Bixby, V. Chvatal, W. Cook

15,112

cities

114,178,716

variables

2001

Page 86: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

86

How do we solve a TSP like this?

Upper bound:

Heuristic search Chained Lin-Kernighan

Lower bound: Linear programming

Divide-and-conquer

Polyhedral combinatorics

Parallel computation

Algorithms & data structures

The LOWER BOUND is the mathematically andalgorithmically hard part of the work

Page 87: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

87

Work on LP relaxations of the symmetric travelling salesman polytope

|: { }

min

( ( )) 2

( ( )) | | 1 \ 1 ,3 | | 3

0 1

0,1

n T ET n

T

ij

ij

T tour in KQ conv

c x

x i i V

x E W W W V W n

x ij E

x ij E

Z

Integer Programming Approach

Page 88: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

88

cutting plane technique for integer and mixed-integer programming

Feasibleintegersolutions

LP-based relaxation

Convex hull

Objectivefunction

Cuttingplanes

Page 89: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

89

Clique-tree cut for pcb442 from B. Cook

Page 90: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

90

LP-based Branch & Bound

Root

Integer

v =0 v =1

x =

0 x =1

y =0

y =1

z =0 z = 1

Lower Bound

Integer

Upper Bound

Infeas

z =

0z = 1

GAP

GAP

Remark: GAP = 0 Proof of optimality

Solve LP relaxation: v=0.5 (fractional)

Page 91: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

91

A BranchingTree

Applegate

Bixby

Chvátal

Cook

Page 92: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

92

Managing the LPs of the TSP

CORE LP

astr

onom

ical

|V|(|V|-1)/2C

uts:

Sep

arat

ion

~ 3|V| variables~1.5|V| constraints

Column generation: Pricing.

Page 93: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

93

A Pictorial History of Some TSP World Records

Page 94: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

94

Some TSP World Records

year authors # cities # variables

1954 DFJ 42/49 820/1,146

1977 G 120 7,140

1987 PR 532 141,246

1988 GH 666 221,445

1991 PR 2,392 2,859,636

1992 ABCC 3,038 4,613,203

1994 ABCC 7,397 27,354,106

1998 ABCC 13,509 91,239,786

2001 ABCC 15,112 114,178,716

2004 ABCC 24,978 311,937,753

number of cities2000x

increase

4,000,000times

problem sizeincrease

in 52years

2005 W. Cook, D. Epsinoza, M. Goycoolea 33,810 571,541,145

2006pla 85,900

solved3,646,412,050

variables

Page 95: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

95

The current champions

ABCC stands for

D. Applegate, B. Bixby, W. Cook, V. Chvátal

almost 15 years of code development

presentation at ICM’98 in Berlin, see proceedings

have made their code CONCORDE available in the Internet

Page 96: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

96

USA 49

49 cities1,146 variables

1954

G. Dantzig, D.R. Fulkerson, S. Johnson

Page 97: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

97West-Deutschland und

Berlin

120 Städte7140 Variable

1975/1977/1980

M. Grötschel

Page 98: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

98

A tour around the world666 cities

221,445 variables

1987/1991

M. Grötschel, O. Holland, seehttp://www.zib.de/groetschel/pubnew/paper/groetschelholland1991.pdf

Page 99: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

99

USA cities with population >500

13,509 cities

91,239,786Variables

1998

D. Applegate, R.Bixby, V. Chvátal, W. Cook

Page 100: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

100

usa13509: The branching tree

0.01% initial gap

Page 101: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

101

Summary: usa13509

9539 nodes branching tree

48 workstations (Digital Alphas, Intel Pentium IIs, Pentium Pros, Sun UntraSparcs)

Total CPU time: 4 cpu years

Page 102: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

102

Overlay of3 OptimalGermanytours

fromABCC 2001

http://www.math.princeton.edu/tsp/d15sol/dhistory.html

Page 103: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

103

Optimal Tour of Sweden

311,937,753

variables

ABCCplusKeld

HelsgaunRoskilde

Univ. Denmark.

Page 104: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel

104

World Tour, current statushttp://www.tsp.gatech.edu/world/

We give links to several images of the World TSP tourof length 7,516,353,779 found by Keld Helsgaun in December 2003. A lower bound provided by the Concorde TSP code shows that this tour is at most 0.076% longer than an optimal tour through the 1,904,711 cities.

Page 105: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Vorlesungsplan Kapitel 1. Das Travelling-Salesman- und verwandte Probleme:

ein Überblick und Anwendungen

1. Vorlesung: ppt-Überblick über das TSP, alte Folien und Cook-Book, Archäologie, Dantzig, Fulkerson und Johnson

Kapitel 2. Hamiltonsche und hypohamiltonsche Graphen und Digraphen

2. Vorlesung: Hamiltonsche Graphen aus Bondy und Murty

3. Vorlesung: Hypohamiltonsche und Hypobegehbare Graphen (Thomassen-Paper und Paper mit Yoshiko)

Kapitel 3. Die „natürlichen“ IP-Formulierungen des TSP und des ATSP, Travelling Salesman-PolytopeSubtour-Formulierungen, STSP und ATSP-Polytop

Kapitel 4. Kombinatorische Verwandte des TSPDas 1-Baum-, 2-Matching-, Zuordnungs-, 1-Arboreszenz-Problem

Martin Grötschel

105

Page 106: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Vorlesungsplan Kapitel 5. Gütegarantien für Heuristiken,

Eröffnungsheuristiken für das TSP (NN, Insert, Christofides,...)

Kapitel 6. Verbesserungsheuristiken und ein polynomiales Approximationsschema (Exchange, LK, Helsgaun, Simulated Annealing, evolutionäre Algorithmen,...)

Kapitel 7. Ein Branch&Bound-Verfahren für das ATSPAssignment-B&B

Kapitel 8. 1-Bäume, Lagrange-Relaxierung und untere Schranken durch ein Subgradientenverfahren (Held&Karp)

Kapitel 9. Alternative IP-Modelle

Kapitel 10. Das symmetrische TSP-Polytop

Kapitel 11. Schnittebenenerkennung/Separationsalgorithmen

Martin Grötschel

106

Page 107: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Vorlesungsplan Kapitel 12. Zur praktischen Lösung großer TSPs

Kapitel 13. TSPs mit Nebenbedingungen (Reihenfolgebedingungen, Zeitfenster, Multi-Salesmen,...) 

Unterwegs einbauen: Malen nach Zahlen, TSP-Portraits (Gesichter), Knights-Problem im Schach, Routenplanung,...

Martin Grötschel

107

Page 108: Martin Grötschel  Institute of Mathematics, Technische Universität Berlin (TUB)  DFG-Research Center “Mathematics for key technologies” (M ATHEON )

Martin Grötschel Institute of Mathematics, Technische Universität Berlin (TUB) DFG-Research Center “Mathematics for key technologies” (MATHEON) Konrad-Zuse-Zentrum für Informationstechnik Berlin (ZIB) [email protected] http://www.zib.de/groetschel

The Travelling Salesman Problem

a brief survey

Martin GrötschelVorausschau auf die Vorlesung

Das Travelling-Salesman-Problem (ADM III)

14. Oktober 2013

The END