Slide 1

Introduction to multi-objective optimizationWe often have more than one objectiveThis means that design points are no longer arranged in strict hierarchyThere are points that are clearly poorer than others because all objectives are worseIn optimization jargon we call these points dominatedPoints that are not dominated are called non-dominated or Pareto optimal

In many engineering problems we have more than one objective. Typical objectives in structural optimization may be weight, cost, failure load or probability of failure.

Once we have more than one objectives we cannot arrange points in strict hierarchy. For example, I worked on a problem of designing composite laminates, where we could combine two materials, glass epoxy and graphite epoxy. The former was cheaper and heavier than the latter.

While some designs cannot be ranked compared to other designs, some can. If we have the two objectives of weight and cost, and one design is both lighter and cheaper than the other, it is clearly better. We say that the poorer design is dominated.

Points in design space that are not dominated are called non-dominated, or Pareto optimal. The set of all Pareto-optimal points is called the Pareto front.

The Italian economist Vilfredo Pareto (1848-1923) was one of the leaders of the Lausanne School. Although only mildly influential during his lifetime, his "tastes-and-obstacles" approach to general equilibrium theory were resurrected during the great "Paretian Revival" of the 1930s and have guided much of economics since (from Wikipedia)

Chapter 11 of Belegundu and Chandrupatla,s Optimization Concepts and Applicatons in Engineering, is placed in the resource section of the course for additional material.

1Definition of Pareto optimalityFor a problem with m objective functions, a design variable vector x* is Pareto optimal if and only if there is no vector x in the feasible space with the characteristics

This slide provides the formal definition of Pareto optimality. Given an optimization problem with m objectives, a feasible x* is Pareto-optimal or non dominated, if we cannot find another feasible x that dominates it. That is, we cannot find an x that has a lower value for one objective and is at least as good for all the others.2Problem dominationWe have two objectives that are to be minimized. The following are the pairs of objectives at 10 design points. Identify the dominated ones.

(67,71) (48,72) (29,88) (-106, 294) (32,13), (-120,163), (103,-30) (-78,114) (-80,143) (75,-171) Solution in notes page

(67,71) (48,72) dominated by (32,13)

(-106, 294) dominated by (-120,163)

(103,-30) dominated by (75,-171)3Work choice exampleItem (task)Pay ($)Time (min)Fun index1133225231224321Minimize time and maximize fun so that you make $100. Will need between 33.3 to 100 items. Time can vary from 66.7 minutes to 300. Fun can vary between 33.3 and 300.

Choosing what line of work you want to engage in is often a matter of maximizing your income, maximizing your free time, and maximizing your job satisfaction. Here is a problem that captures these considerations. However, to reduce the number of objectives to two, we assume that you need to make $100 a day from a mix of four different tasks defined in the table.4Comparing tasksItem (task)Pay ($)/hourTotal time (hours)Total fun12053002244.171003303.331004901.1133.3task 2 is dominated by task 3. Tasks 1,3,4 are Pareto optimal

One gets more insight into the selection process by looking at the amount of time required to make $100 from one task. This is shown in the table, showing that as we progress we reduce time and fun at the same time. Except from the second task, which is clearly inferior (or dominated) to task 3 which produces the same fun in less time.5Multi-objective Formulation

ItemPay Time Fun 1133225231224321

We now pose the problem as an optimization problem, defining the design variable xi as the number of times we repeat task i. The constraint on the required $100 is written in a standard and normalized form.

The common sense constraints derive from the extreme case of using only one task to satisfy the dollar requirement. Thus, if we use only Task 1, which pays $1, we will need to repeat it 100 times. If we also do some other tasks, x1 will be smaller than 100, hence the common sense constraint on x1.

6Solution by enumerationIn the range of three variables calculate time and fun for all combinations that produce exactly $100.Pareto front is upper boundary and it is almost a straight line.Matlab code given in notes page.

function [ fun,time,pay,npoint,x] = multobjfun()%Fun maximization and time minimization in task selection% Go over all selections of tasks 1,2,3,4 from multi-optimization% lecture that yield a pay of $100.npoint=0;for i1=0:100for i3=0:100 for i4=0:34 payc=i1+i3+3*i4; if payc==100 npoint=npoint+1; pay(npoint)=i1+i3+3*i4; fun(npoint)=3*i1+2*i3+i4; time(npoint)=3*i1+2*i3+2*i4; x(1,npoint)=i1; x(3,npoint)=i3; x(4,npoint)=i4; end endendend end

7Pay-fun ProblemFormulate the problem of maximizing fun and pay for five hours (300 minutes) including only non-dominated variables. SolutionObtain the Pareto front analytically or by enumeration and writes its equation (fun as a function of pay). Solution.8More efficient solution methodsMethods that try to avoid generating the Pareto frontGenerate utopia pointDefine optimum based on some measure of distance from utopia pointGenerating entire Pareto frontWeighted sum of objectives with variable coefficientsOptimize one objective for a range of constraints on the othersNiching methods with population based algorithms

In most problem we cannot solve the problem by enumeration, or by covering the design space by a dense grid. For a more efficient approach, there are methods that avoid generating the Pareto front, and instead try for a predetermined compromise.

One example is to start by generating a utopia point, which is obtained by minimizing each function individually and then taking the best values of all objectives. Then a single objective can be generated based on some measure of the distance from the utopia point, as will be shown in the example on the next slide.

For generating the Pareto front three approaches are popular. The first takes a weighted average of the objectives for a range of weights. For example, 0.1 times the first objective plus 0.9 times the second, then 0.2 times the first plus 0.8 times the second, and so on. This method may fail to produce the entire front if it is not convex.

A second method, which I recommend is to optimize one objective for a series of constraints on the other objective, which we will look into for the task example.Finally, multi-objective genetic algorithms will be studied later on in the course.9The utopia point is (66.7,300).One approach is to use it to form a compromise objective as shown above.It gives time=168 minutes, fun=160 (x3=76,x4=8)

For our problem, the best time is 66.7 minutes, and the highest fun index is 300, so the utopia point is (66.7, 300). If we have the entire front, we may decide to select the nearest point to the utopia point, but that would be difficult to generate without knowing the entire front.

Instead, we may use the utopia values to normalize a combined objective, as shown in the equation on the right. Searching the vectors of time and fun obtained by enumeration, we find that the compromise objective is minimized for time=168, and fun index=160, corresponding to x1=0, x3=76, x4=8.10Series of constraints

The next slides provides a Matlab segment for solving this optimization problem using the function fmincon, but without the requirement of integer variables.How will you change the formulation so that a single Matlab run will give us results for any required earning level R? Solution in the notes page.The recommended approach for generating the Pareto front is to minimize one objective with a range of constraints on the other objectives. Here we selected to minimize the time for a series of constraints on the fun. We know that the best time is obtained by using only x4, and then the fun is 30, while the highest fun is objtained when we use only x1, and then it is equal to 300.

So in the next slide we will show a Matlab segment that solves this problem for a series of constraints on the fun ranging from 30 to 300 in increments of 5.

If the required earning (pay) is R instead of 100, that is x1+x3+3x4R

we can normalize the design variables as yi=100xi/R, then we can use the same problem formulation as before, and just multiply the solution by R/100.11Matlab segmentx0 = [10 10 10];for fun_idx = 30:5:300 A = [-1 -1 -3; -3 -2 -1]; b = [-100;-fun_idx]; lb = zeros(3,1); options = optimset('Display','off'); [x,fval,exitflag,output,lambda] = fmincon('myfun',x0,A,b,[],[],lb,[],[],options); pareto_sol(fun_idx,:) = x; pareto_fun(fun_idx,1) = fval; pareto_fun(fun_idx,2) = 3*x(1) + 2*x(2) + x(3);End

function f = myfun(x)f = 3*x(1) + 2*x(2) + 2*x(3);

We will study the algorithm behind fmincon later in the semester. For the time being you may want to look at the documentation for fmincon, in order to clarify the Matlabe statements above.12Pareto-front ProblemGenerate the Pareto front for the pay and fun maximization using a series of constraints, and also find a compromise point on it using the utopia point. SolutionWhat is responsible for the slope discontinuity in the Pareto front on Slide 10? Solution in notes pageBefore the discontinuity point, the Pareto optimal correspond to x1=0, while after the discontinuity point, the Pareto optimal correspond to x4=0. This is because in the first part of Pareto front, time minimization is more dominant, x3 and x4 are better choices since their time/pay ratio is lower than for x1 . And in the second part of Pareto front, fun maximization becomes more dominant, x1 and x3 are better choices since their fun/pay ratio is higher than x4.13