Slide 1

Graphical optimizationSome problems are cheap to simulate or test.Even if they are not, we may fit a surrogate that is cheap to evaluate.Relying on optimization software to find the optimum is foolhardy.It is better to thoroughly explore manually.In two dimensions, graphical optimization is a good way to go.For problems with more variables, it may still be good to identify two important variables and proceed graphically.In higher dimensions, two dimensional cuts can help understand the relationship between local optima.

1ExamplePlot and estimate minimum ofIn the rangeCan do contour plot or mesh plot

x=linspace(-2,0,40); y=linspace(0,3,40);[X,Y]=meshgrid(x,y); Z=2+X-Y+2*X.^2+2*X.*Y+Y.^2;cs=contour(X,Y,Z);

cs=surfc(X,Y,Z);xlabel('x_1'); ylabel('x_2'); zlabel('f(x_1,x_2)');2Then zoomZooming by reducing the range does not change the estimate of the position (-1,1.5), but the value is now below 0.8.

x=linspace(-1.5,-0.5,40); y=linspace(1,2,40);[X,Y]=meshgrid(x,y);Z=2+X-Y+2*X.^2+2*X.*Y+Y.^2

Once we know the approximate position of the minimum we can zoom on a narrower range using the Matlab sequence:

x=linspace(-1.5,-0.5,40);y=linspace(1,2,40);[X,Y]=meshgrid(x,y);Z=2+X-Y+2*X.^2+2*X.*Y+Y.^2

The figures do not change our estimate of the position of the minimum (-1,1.5), but update our estimate of the value of the minimum (below 0.8).3In higher dimensionsOften different solutions of optimization problems are obtained from multiple starting points or even from same starting point.It is tempting to ascribe this to local optima.Often it is the result of algorithmic or software failure.Generate line connecting two solutions

4ExampleConsider Two candidate local optima (0,0,0) and (0.94,0.94,0.94).Line connecting first and second point indicates that they may be local optima.

alpha=linspace(0,1,101);x1=[0 0 0]; x2=[0.94 0.94 0.94];x=x2(1)*alpha+x1(1)*(1-alpha); y=x2(2)*alpha+x1(2)*(1-alpha);z=x2(3)*alpha+x1(3)*(1-alpha);f=abs(sin(2*pi.*x).*sin(2*pi.*y).*sin(2*pi.*z))+x.*sqrt(y).*z.^2;plot(alpha,f);

5Two-dimensional cut

6ExampleOptimize g=@(x) (sin(2*pi*x(1))*sin(2*pi*x(2))*sin(2*pi*x(3)))+x(1)*sqrt(x(2))*x(3)^2; X=fminsearch(@(x) g(x), [0,0,0])

7

Unconstraiend optimization in Matlab [X,gval,exitflag]=fminsearch(@(x) g(x), [0,0,0])X =0.2492 0.2496 -0.2485gval =-0.9923exitflag =1[X,gval,exitflag]=fminsearch(@(x) g(x), [0.5,0.5,0.5])X =0.2487 0.7498 0.2473gval =-0.9867exitflag =1 1 Maximum coordinate difference between current best point and other points in simplex is less than or equal to TolX, and corresponding difference in function values is less than or equal to TolFun. 0 Maximum number of function evaluations or iterations reached. -1 Algorithm terminated by the output function.

Can set convergence criteria in optimset.

8Alternative using derivatives[X,gval,exitflag]=fminunc(@(x) g(x), [0.24,0.24,-0.24])Warning: Gradient must be provided for trust-region algorithm; using line-search algorithm instead. > In fminunc at 383 Local minimum found.Optimization completed because the size of the gradient is less than the default value of the function tolerance.

X =0.2492 0.2496 -0.2484gval = -0.9923exitflag =1

[X,gval,exitflag]=fminunc(@(x) g(x), [0,0,0])Warning: Gradient must be provided for trust-region algorithm; using line-search algorithm instead. > In fminunc at 383Initial point is a local minimum.Optimization completed because the size of the gradient at the initial point is less than the default value of the function tolerance.

X = 0 0 0gval = 0exitflag = 1

10 dimensional constrained exampleAerodynamic design of supersonic transport.Objective function, take-off gross weight is a cheap and simple function of design variables.Design variables define geometry of wing and fuselage.Constraints include range, take-off and landing constraints, and maneuvrability constraints.The constraints are all uni-modal functions, but in combination, they create a complex, non-convex feasible domain.

To complement the previous simple example we also look at an example where multiple optima are not due to waviness of the objective functions but due to the fact that the feasible domain (the domain where all constraints are satisfied) is not convex (that is two points in the domain can be connected by a line that passes outside the domain). The example is taken from: Knill, D.L., Giunta, A.A., Baker, C.A., Grossman, B., Mason, W.H., Haftka, R.T., and Watson, L.T., Response Surface Models Combining Linear and Euler Aerodynamics for Supersonic Transport Design, Journal of Aircraft, 36(1), pp. 75-86, 1999

The objective function is the take-off weight of a supersonic transport, the design variables define the shape of the wing and fuselage, and there is a large number of constraints such as range, take-off clearance constraints, and maneuverability constraints.

All the constraints are approximated by quadratic polynomials, so they are not wavy, and the objective function is also unimodal (not wavy). However the constraints still create different local optima at vertices of the feasible domain. 10Simple constraints create non-convex design space.Can thathappen withlinear constraints?

Two local optima and a third point

11

Problems