1/16/2017

1

Evolutionary and Heuristic Optimisation (ITNPBD8) Lecture 1: Introduction

Gabriela Ochoa http://www.cs.stir.ac.uk/~goc/

Computing Science and Mathematics,

School of Natural Sciences

University of Stirling, Stirling, Scotland

Content

• Welcome and introductions

• Outline of the module

• Evaluation

• Resources

2

1/16/2017

2

Outline of the module

1. Optimisation problems – Optimisation & search

– Classic mathematical models

– Example problems: Knapsack, Travelling Salesperson, Timetabling

2. Optimisation methods – Heuristics and metaheuristcis

– Single point algorithms

– Population-based algorithms

3. Advanced topics – Fitness landscape analysis

– Multi-objective optimisation

3

Outline of the Module

4

1/16/2017

3

Evaluation

• 10%: Checkpoints in the Lab – Python programming exercises, programming

algorithms from scratch, but part of code will be given – Based on a simple problem: knapsack

• 40%: Assignment – Select a paper applying metaheuristics to a real-world problem. List will be provided. – 15%: Slides in Powerpoint with fixed template – 15% : Summary 1 page in Word – 10%: Oral presentation

• 50%: Exam

5

Resources: Article & Online Book

• Online Book

– Essentials of Metaheuristics, Sean Luke, Department of Computer Science George Mason University

• Survey Article

– Metaheuristics in combinatorial optimization: Overview and conceptual comparison, C Blum, A Roli ACM Computing Surveys (CSUR) 35 (3), 268-308

6

1/16/2017

4

Resources: Books • Burke , E. K; Kendall, G., (Eds.) (2005, 2015) Search

Methodologies: Introductory Tutorials in Optimization and Decision Support Techniques, Springer.

• A.E. Eiben and J.E. Smith (2003,2015), Introduction to Evolutionary Computing, Springer,

• Hoos, H; Stutzle, T. (2004) Stochastic Local Search Foundations and Applications, Elsevier.

• Z. Michalewicz, D.B. Fogel (2000) How to Solve It: Modern Heuristics, Springer.

• Talbi, E-G (2009), Metaheuristics: From Design to Implementation, Wiley.

7

Borg Vogons

Biotop

Art

Life Sciences Social Sciences

Mathematics Physics

Software Engineering

Neural Nets Evolutionary Computing Fuzzy Systems

Computational Intelligence etc

Computer Science etc

Exact Sciences etc

Science Politics Sports etc

Society Stones & Seas etc

Earth etc

Universe

You are here

Positioning of Evolutionary and Heuristic Optimisation

A.E. Eiben and J.E. Smith (2003,2015), Introduction to Evolutionary Computing, Springer

1/16/2017

5

OPTIMISATION PROBLEMS What is Optimisation? What is an Optimisation Problem?

9

Optimisation problems • Wide variety of applications across industry,

commerce, science and government • Optimisation occurs in the minimisation of time,

cost and risk, or the maximisation of profit, quality, and efficiency

• Examples – Finding shortest round trips in graphs (TSP) – Finding models of propositional formulae (SAT) – Science: Determining the 3D-structure of proteins – Operational research, Industry: Planning, scheduling, cutting &

packing, logistics, transportation, communications, timetabling, resource allocation, genome sequencing

– Software engineering: test case minimisation and prioritisation, requirements analysis, code design and repair, etc.

10

1/16/2017

6

Optimisation problems are everywhere!

Logistics, transportation, supply change management

Manufacturing, production lines Timetabling

Cutting & packing Computer networks and Telecommunications

Software - SBSE

11

Optimisation problems General constrained optimisation problem:

Optimisation through search

Iteratively generate and evaluate

candidate solutions.

• Systematic search

• (Stochastic) local search

Search Space: set of candidate solutions. All possible combinations of the decision variables.

Min/Max f( )

x

Subject to:

0)( xgi

0)( xhj

i = 1,…,p

j = 1,…,n

12

1/16/2017

7

Search in Computing Science

1. Search for stored data • Finding information stored in disc or

memory. • Examples: Sequential search, Binary

search

2. Search for web documents • Finding information on the world wide

web • Results are presented as a list of results

3. Search for paths or routes • Finding a set of actions that will bring us

from an initial stat to a goal stat • Relevant to AI • Examples: depth first search, breath first

search, branch and bound, A*, Monte Carlo tree search.

4. Search for solutions • Find a solution in a large space of

candidate solutions • Relevant to AI, Optimisation, OR • Examples: evolutionary algorithms, Tabu

search, simulated annealing, ant colony optimisation, etc.

At least 4 meanings of the word search in CS

13

Search and optimisation in practice

Real-world problem

Model

Solution

Formulation

Algorithm

Problem Model • Problem representation • Constraints • An objective function

Solution to the Model • Feasible candidate solution • Lead to the optimal (or good enough) value of the objective function

Optimisation/search Algorithm • Exact methods • Approximate (heuristic) methods

Many challenging applications in science and industry can be formulated as optimisation problems!

14

1/16/2017

8

Optimisation problems: two categories

Continuous

• Continuous variables

• Looking for a set (vector) of real numbers [45.78, 8.91, 3.36]

• Objective function has a mathematical expression

• Special cases studied in mathematics and OR: Convex, Linear

Combinatorial

• Discrete variables

• Looking for an object from a finite set – Binary digits [1011101010]

– Integer [1, 53, 4, 67, 39]

– Permutation [3,5,1,2,4]

– Graph

• Generally have quite different flavours and methods for solving them • Have become divergent

15

Classic mathematical models

Linear Programs (LP)

• Linear optimisation

• A single objective

• The objective and constraints are linear

• Decision variables, allowed to have any values

• Easy to solve numerically (simplex method)

Importance

• Many applications

• See Wolfram page

Integer Programs (IP)

• LP in which some or all variables are constrained to take on integer values

• Harder to solve. Software packages: Excel, LINGO/LINDO and MPL/CPLEX,

Importance

• problems in which variables required to be integer

• many decisions are essentially discrete (yes/no, go/no-go)

16

1/16/2017

9

Integer program: canonical form maximise c1x1+c2x2+…+cnxn (objective function)

subject to

a11x1+a12x2+…+a1nxn b1 (functional constraints)

a21x1+a22x2+…+a2nxn b2

….

am1x1+am2x2+…+amnxn bm x1, x2 , …, xn Z+ (set constraints)

In vector form: maximise cx (objective function) subject to Ax b (functional constraints) x Zn

+ (set constraints)

17

The knapsack problem • Given a knapsack of capacity W, and a number n of items, each

with a weight and value. The objective is to maximise the total value of the items in the knapsack

Maximise Subject to

18

Examples of Applications • Resource allocation • Cutting and packing problems • Requirements engineering

1/16/2017

10

The knapsack problem • Formulation. Mathematical model

maximise

4x1+2x2+x3+10x4 +2x5

subject to

12x1+2x2+x3+4x4+x5 15

x1,x2,x3,x4,x5 {0, 1}

Xi = 1 If we select item i 0 Otherwise

• Can be formulated as an Integer Programming problem, and solved efficiently using Dynamic Programming

• Binary representation [11010], using heuristic methods

• Search space size = 2n

• n = 100, 2100 ≈ 1030

19

Real-world example of the Knapsack problem

• Consider a cargo company, that has an airplane and need to carry packages.

• Customers state the weight of the cargo item they would like delivered, and the amount they are prepared to pay.

• The airline is constrained by the total amount of weight the plane is allowed to carry.

• The company must choose a subset of the packages (bids) to carry in order to make the maximum possible profit, given the weight limit that they must respect. 20

1/16/2017

11

Travelling salesman problem (TSP)

• A salesman must visit number of cities minimising the total cost of traveling

• Most prominent classical example of an NP-complete combinatorial optimisation problem

• BBC 4 Documentary (min 28:00 – 33:45 TSP): The Secret Rules of Modern Living: Algorithms

• Configurations: permutation (ordering) of cities. – s1= (A B C D), f(s1)= 20+30+12+35= 97

– s2= (A B D C), f(s2)= 20+34+12+42=108

– s3= (A C B D), f(s3)= 42+30+34+35= 141

• Local change (move): swap two cities – s1 = (A B C D)

– s1’ = (A D C B)

Gabriela Ochoa, goc@cs.stir.ac.uk 22

Check: Permutation Generator

1/16/2017

12

ACADEMIC TIMETABLING PROBLEMS

Gabriela Ochoa, goc@cs.stir.ac.uk

Academic timetabling problems

• assigns teachers to a set of specific subjects or topics. Faculty timetabling Faculty timetabling

• assigns subjects to a fixed and specific group of students.

Class-teacher timetabling

Class-teacher timetabling

• ensures that every teacher-subject pair has an available classroom.

Classroom-assignment Classroom-assignment

• assigns one-time events like final exams or special lectures to individual students.

Examination timetabling Examination timetabling

• assigns subjects to individual students Course timetabling Course timetabling

Gabriela Ochoa, goc@cs.stir.ac.uk

1/16/2017

13

The course timetabling problem

• Weekly scheduling of lectures for several university courses • Given number of rooms and time periods • Conflicts between courses are set according to the curricula

published by the University • Components

– Days, Timeslots, and Periods. Periods = the product of the days times the day timeslots.

– Courses and Teachers. Fixed number of lectures on distinct periods (spread). Attended by a number students, taught by a teacher.

– Rooms. With a capacity – Curricula. Group of courses with common students

Gabriela Ochoa, goc@cs.stir.ac.uk

Course timetabling test instances, ITC- 2007

Hard constraints

• Students cannot attend more than one course at a time

• Rooms should be big enough to hold the students

• One course per room, per timeslot

Soft constraints

Avoid as much as possible to:

• Assign students to a course in the last timeslot of the day

• Assign students to 3 or more timeslots in a row

• Assign students to only one event in a particular day

Gabriela Ochoa, goc@cs.stir.ac.uk

• Real-world data from the University of Udine, Italy (24) • For all instances there exists at least one feasible solution (no hard

constraint violations), but it is not known which is the optimal value for the soft constraints.

1/16/2017

14

Course timetabling problem

• E = {e1, e2,…, en} Events (courses

of subjects) Events (courses

of subjects)

• T = {t1, t2 …, ts} Time periods Time periods

• P = {p1, p2 …, ps} Places

(classrooms) Places

(classrooms)

• A = {a1, a2 …, as} Students Students

• quadruple (e,t,p,S) S subset A Assignment Assignment

• complete set of n assignments, that satisfies the constraints

Timetabling solution

Timetabling solution

Gabriela Ochoa, goc@cs.stir.ac.uk

Representation: set of integers representing indexes

Objective function:

Assigns subjects to individual students

Summary of optimisation problems

Real-world problem

Model

Solution

Formulation

Algorithm

Problem Model • Problem representation • Constraints • A fitness function

Solution to the Model • Feasible candidate solution • Lead to the optimal (or good enough) value of the objective function

Optimisation/search Algorithm • Exact methods • Approximate (heuristic) methods

Many challenging applications in science and industry can be formulated as optimisation problems!

28