solving optimization problems with matlab · perform exact computations using familiar matlab...
TRANSCRIPT
![Page 1: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/1.jpg)
1© 2018 The MathWorks, Inc.
Solving Optimization Problems with MATLAB
![Page 2: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/2.jpg)
2
Introduction
Least-squares minimization
Nonlinear optimization
Mixed-integer programming
Global optimization
Topics
![Page 3: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/3.jpg)
3
Optimization Problems
Minimize Risk
Maximize Profits
Maximize Fuel Efficiency
![Page 4: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/4.jpg)
4
Design Process
Initial
Design
Variables
System
Modify
Design
Variables
Optimal
DesignObjectives
met?
No
Yes
![Page 5: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/5.jpg)
5
Manually (trial-and-error or iteratively)
Why use Optimization?
Initial
Guess
![Page 6: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/6.jpg)
6
Automatically (using optimization techniques)
Initial
Guess
Why use Optimization?
![Page 7: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/7.jpg)
7
Why use Optimization?
Finding better (optimal) designs and decisions
Faster design and decision evaluations
Automate routine decisions
Useful for trade-off analysis
Non-intuitive designs may be found
Antenna Design Using Genetic Algorithmhttp://ic.arc.nasa.gov/projects/esg/research/antenna.htm
![Page 8: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/8.jpg)
8
Introduction
Least-squares minimization
Nonlinear optimization
Mixed-integer programming
Global optimization
Topics
![Page 9: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/9.jpg)
9
Curve Fitting Demo
Given some data:
Fit a curve of the form:
teccty 21
)(
t = [0 .3 .8 1.1 1.6 2.3];
y = [.82 .72 .63 .60 .55 .50];
![Page 10: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/10.jpg)
10
How to solve?
Ecc
cey
eccty
t
t
2
1
21
1
)(
As a linear system of equations:
2
2min yEc
Ecy
c
Can’t solve this exactly
(6 eqns, 2 unknowns)
An optimization problem!
2
1
3.2
6.1
1.1
8.0
3.0
0
1
1
1
1
1
1
50.0
55.0
60.0
63.0
72.0
82.0
c
c
e
e
e
e
e
e
![Page 11: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/11.jpg)
11
Introduction
Least-squares minimization
Nonlinear optimization
Mixed-integer programming
Global optimization
Topics
![Page 12: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/12.jpg)
12
Nonlinear Optimization
min𝑥
log 1 + 𝑥1 −4
3
2
+ 3 𝑥1 + 𝑥2 − 𝑥13 2
![Page 13: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/13.jpg)
13
Nonlinear Optimization - Modeling Gantry Crane
Determine acceleration profile that
minimizes payload swing
s25s4
s20s1
s20s1
:sConstraint
2
1
21
•
•
•
•
f
p
p
ppf
t
t
t
ttt
![Page 14: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/14.jpg)
14
Symbolic Math Toolbox
Perform exact computations using familiar
MATLAB syntax in MATLAB
– Integration
– Differentiation
– Equation solving
– Transformations
– Simplification
– Unit conversion
– Variable precision arithmetic
Results in typeset math in Live Editor
Integrates with MATLAB, Simulink, Simscape
![Page 15: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/15.jpg)
15
Introduction
Least-squares minimization
Nonlinear optimization
Mixed-integer programming
Global optimization
Topics
![Page 16: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/16.jpg)
16
Mixed-Integer Programming
Many things exist in discrete amounts:
– Shares of stock
– Number of cars a factory produces
– Number of cows on a farm
Often have binary decisions:
– On/off
– Buy/don’t buy
Mixed-integer linear programming:
– Solve optimization problem while enforcing that certain variables need to be integer
![Page 17: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/17.jpg)
17
Continuous and integer variables
𝑥1 ∈ 0, 100 𝑥2 ∈ 1,2,3,4,5
Linear objective and constraints
min𝑥
−𝑥1 − 2𝑥2
ቊ𝑥1 + 4𝑥2 ≤ 20𝑥1 + 𝑥2 = 10
such that
Mixed-Integer Linear Programming
![Page 18: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/18.jpg)
18
Optimize Gift Card Spending
Problem:
Given gift cards to different stores and a shopping list of desired purchases, decide how to
spend the gift cards to use as much of the gift card money as possible.
Constraints:
You cannot overspend the gift card.
You can purchase one of any item, and must purchase one of a specific item.
![Page 19: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/19.jpg)
19
Traveling Salesman Problem
Problem
How to find the shortest path through a series of points?
Solution
Calculate distances between all combinations of points
Solve an optimization problem where variables correspond to trips between two points
1
11
0
1
1
0
0
00
![Page 20: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/20.jpg)
20
Introduction
Least-squares minimization
Nonlinear optimization
Mixed-integer programming
Global optimization
Topics
![Page 21: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/21.jpg)
21
Example Global Optimization Problems
Why does fmincon have a hard time finding the
function minimum?
0 5 10
-10
-5
0
5
10
x
Starting at 10
0 5 10
-10
-5
0
5
10
x
Starting at 8
0 5 10
-10
-5
0
5
10
x
Starting at 6
x s
in(x
) +
x c
os(2
x)
0 5 10
-10
-5
0
5
10
x
Starting at 3
0 5 10
-10
-5
0
5
10
x
Starting at 1
0 5 10
-10
-5
0
5
10
x
Starting at 0
x s
in(x
) +
x c
os(2
x)
![Page 22: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/22.jpg)
22
Initial
Guess
Example Global Optimization Problems
Why didn’t fminunc find the maximum efficiency?
![Page 23: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/23.jpg)
23
Example Global Optimization Problems
Why didn’t nonlinear regression find a good fit?
0 20 40 60 80 100 1200
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
t
c
c=b1e-b
4t+b
2e-b
5t+b
3e-b
6t
![Page 24: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/24.jpg)
24
Global Optimization
Goal:
Want to find the lowest/largest value of
the nonlinear function that has many local
minima/maxima
Problem:
Traditional solvers often return one of the
local minima (not the global)
Solution:
A solver that locates globally optimal
solutions
Global Minimum at [0 0]
Rastrigin’s Function
![Page 25: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/25.jpg)
25
Global Optimization Solvers Covered Today
Multi Start and Global Search
Pattern Search
Genetic Algorithm
Surrogate Optimization
Particle Swarm
Simulated Annealing
![Page 26: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/26.jpg)
26
MultiStart Demo – Nonlinear Regression
lsqcurvefit solution MultiStart solution
![Page 27: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/27.jpg)
27
MULTISTART
![Page 28: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/28.jpg)
28
What is MultiStart?
Run a local solver from each set
of start points
Option to filter starting points
based on feasibility
Supports parallel computing
![Page 29: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/29.jpg)
29
MultiStart Demo – Peaks Function
![Page 30: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/30.jpg)
30
GLOBAL SEARCH
![Page 31: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/31.jpg)
31
What is GlobalSearch?
Multistart heuristic algorithm
Calls fmincon from multiple
start points to try and find a
global minimum
Filters/removes non-promising
start points
![Page 32: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/32.jpg)
32
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview Schematic Problem
Peaks function
Three minima
Green, z = -0.065
Red, z= -3.05
Blue, z = -6.55
x
y
![Page 33: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/33.jpg)
33
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 0Run from specified x0
x
y
![Page 34: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/34.jpg)
34
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 1
3
6
0
00
4
0
-2
x
y
![Page 35: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/35.jpg)
35
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 1
3
6
0
00
4
0
-2
x
y
![Page 36: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/36.jpg)
36
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 1
x
y
![Page 37: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/37.jpg)
37
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 2
x
y
![Page 38: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/38.jpg)
38
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 2
x
y
![Page 39: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/39.jpg)
39
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 2
x
y
![Page 40: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/40.jpg)
40
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 2
x
y
![Page 41: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/41.jpg)
41
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 2
6
Current penalty
threshold value : 4
x
y
![Page 42: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/42.jpg)
42
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 2
x
y
![Page 43: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/43.jpg)
43
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 2
Current penalty
threshold value : 4
-3
x
y
![Page 44: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/44.jpg)
44
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 2Expand basin of attraction if minimum already found
Current penalty threshold value : 2
-0.1
x
y Basins can overlap
![Page 45: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/45.jpg)
45
GlobalSearch Demo – Peaks Function
![Page 46: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/46.jpg)
46
PATTERN SEARCH
(DIRECT SEARCH)
![Page 47: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/47.jpg)
47
What is Pattern Search?
An approach that uses a pattern
of search directions around the
existing points
Expands/contracts around the
current point when a solution is
not found
Does not rely on gradients: works
on smooth and nonsmooth
problems
![Page 48: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/48.jpg)
48
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Pattern Search Overview – Iteration 1Run from specified x0
x
y
3
![Page 49: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/49.jpg)
49
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Pattern Search Overview – Iteration 1Apply pattern vector, poll new points for improvement
x
y
3
Mesh size = 1
Pattern vectors = [1,0], [0,1], [-1,0], [0,-1]
0_*_ xvectorpatternsizemeshPnew
0]0,1[*1 x1.6
0.4
4.6
2.8
First poll successful
Complete Poll (not default)
![Page 50: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/50.jpg)
50
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Pattern Search Overview – Iteration 2
x
y
3
Mesh size = 2
Pattern vectors = [1,0], [0,1], [-1,0], [0,-1]
1.6
0.4
4.6
2.8
-4
0.3-2.8
Complete Poll
![Page 51: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/51.jpg)
51
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Pattern Search Overview – Iteration 3
x
y
3
Mesh size = 4
Pattern vectors = [1,0], [0,1], [-1,0], [0,-1]
1.6
0.4
4.6
2.8
-4
0.3-2.8
![Page 52: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/52.jpg)
52
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Pattern Search Overview – Iteration 4
x
y
3
Mesh size = 4*0.5 = 2
Pattern vectors = [1,0], [0,1], [-1,0], [0,-1]
1.6
0.4
4.6
2.8
-4
0.3-2.8
![Page 53: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/53.jpg)
53
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Pattern Search Overview – Iteration NContinue expansion/contraction until convergence…
x
y
31.6
0.4
4.6
2.8
-4
0.3-2.8
-6.5
![Page 54: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/54.jpg)
54
Pattern Search – Peaks Function
![Page 55: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/55.jpg)
55
Pattern Search Climbs Mount Washington
![Page 56: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/56.jpg)
56
GENETIC ALGORITHM
![Page 57: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/57.jpg)
57
What is a Genetic Algorithm?
Uses concepts from evolutionary
biology
Start with an initial generation of
candidate solutions that are tested
against the objective function
Subsequent generations evolve
from the 1st through selection,
crossover and mutation
![Page 58: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/58.jpg)
58
How Evolution Works – Binary Case
Selection
– Retain the best performing bit strings from one generation to the next. Favor these for
reproduction
– parent1 = [ 1 0 1 0 0 1 1 0 0 0 ]
– parent2 = [ 1 0 0 1 0 0 1 0 1 0 ]
Crossover
– parent1 = [ 1 0 1 0 0 1 1 0 0 0 ]
– parent2 = [ 1 0 0 1 0 0 1 0 1 0 ]
– child = [ 1 0 0 0 0 1 1 0 1 0 ]
Mutation
– parent = [ 1 0 1 0 0 1 1 0 0 0 ]
– child = [ 0 1 0 1 0 1 0 0 0 1 ]
![Page 59: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/59.jpg)
59
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Genetic Algorithm – Iteration 1Evaluate initial population
x
y
![Page 60: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/60.jpg)
60
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Genetic Algorithm – Iteration 1Select a few good solutions for reproduction
x
y
![Page 61: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/61.jpg)
61
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Genetic Algorithm – Iteration 2Generate new population and evaluate
x
y
![Page 62: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/62.jpg)
62
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Genetic Algorithm – Iteration 2
x
y
![Page 63: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/63.jpg)
63
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Genetic Algorithm – Iteration 3
x
y
![Page 64: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/64.jpg)
64
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Genetic Algorithm – Iteration 3
x
y
![Page 65: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/65.jpg)
65
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Genetic Algorithm – Iteration NContinue process until stopping criteria are met
x
y
Solution found
![Page 66: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/66.jpg)
66
Genetic Algorithm – Peaks Function
![Page 67: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/67.jpg)
67
Genetic Algorithm – Integer Constraints
Mixed Integer Optimization
s.t. some x are integers
Examples
Only certain sizes of components
available
Can only purchase whole shares of stock
)(min xfx
![Page 68: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/68.jpg)
68
Application: Circuit Component Selection
6 components to size
Only certain sizes available
Objective:
– Match Voltage vs. Temperature
curve
Thermistor Circuit
300 Ω
330 Ω
360 Ω
…
180k Ω
200k Ω
220k Ω
Thermistors:
Resistance varies
nonlinearly with
temperature
?
𝑅𝑇𝐻 =𝑅𝑇𝐻,𝑁𝑜𝑚
𝑒𝛽𝑇−𝑇𝑁𝑜𝑚𝑇∗𝑇𝑁𝑜𝑚
![Page 69: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/69.jpg)
69
SURROGATE OPTIMIZATION
![Page 70: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/70.jpg)
70
What is Surrogate Optimization?
An approach that creates and
optimizes a surrogate of the
function
Searches randomly to explore
and adaptively to refine
Does not rely on gradients: works
on smooth and nonsmooth
problems
![Page 71: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/71.jpg)
71
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Surrogate Optimization OverviewConstruction phase – evaluate at random points to construct surrogate
x
y
Random points
Incumbent
![Page 72: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/72.jpg)
72
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Surrogate Optimization OverviewSearch phase – minimize merit function of surrogate and point spread
x
y
Random points
Adaptive points
Incumbent
![Page 73: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/73.jpg)
73
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Surrogate Optimization OverviewReset – Repeat construction and search phases
x
y
Random points
Adaptive points
Incumbent
![Page 74: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/74.jpg)
74
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Surrogate Optimization OverviewContinue process until stopping criteria are met
x
y
Random points
Adaptive points
Solution found
Incumbent
![Page 75: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/75.jpg)
75
Surrogateopt Demo – Peaks Function
![Page 76: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/76.jpg)
76
PARTICLE SWARM
![Page 77: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/77.jpg)
77
What is Particle Swarm Optimization?
A collection of particles move
throughout the region
Particles have velocity and are
affected by the other particles in
the swarm
Does not rely on gradients: works
on smooth and nonsmooth
problems
![Page 78: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/78.jpg)
78
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Particle Swarm Overview – Iteration 1Initialize particle locations and velocities, evaluate all locations
x
y
![Page 79: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/79.jpg)
79
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Particle Swarm Overview – Iteration NUpdate velocities for each particle
x
yPrevious
Velocity
Best Location
for this Particle
Best Location for
Neighbor Particles
![Page 80: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/80.jpg)
80
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Particle Swarm Overview – Iteration NUpdate velocities for each particle
x
y
New
Velocity
![Page 81: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/81.jpg)
81
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Particle Swarm Overview – Iteration NMove particles based on new velocities
x
y
![Page 82: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/82.jpg)
82
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Particle Swarm Overview – Iteration NContinue swarming until convergence
x
y
![Page 83: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/83.jpg)
83
Particle Swarm – Peaks Function
![Page 84: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/84.jpg)
84
SIMULATED ANNEALING
![Page 85: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/85.jpg)
85
What is Simulated Annealing?
A probabilistic metaheuristic
approach based upon the physical
process of annealing in
metallurgy.
Controlled cooling of a metal
allows atoms to realign from a
random higher energy state to an
ordered crystalline (globally) lower
energy state
![Page 86: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/86.jpg)
86
Simulated Annealing Overview – Iteration 1Run from specified x0
x
y
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
0.9
![Page 87: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/87.jpg)
87
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration 1
3
x
y 0.9
Possible New Points:
Standard Normal N(0,1) * Temperature
Temperature = 1
![Page 88: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/88.jpg)
88
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration 1
3
x
y 0.9
Temperature = 1
11.01
1/)9.03(
Taccepte
P
![Page 89: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/89.jpg)
89
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration 1
3
x
y 0.9
Temperature = 1
0.3
![Page 90: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/90.jpg)
90
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration 1
3
x
y 0.9
Temperature = 1
0.3
![Page 91: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/91.jpg)
91
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration 2
3
x
y 0.9
Temperature = 1
0.3
![Page 92: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/92.jpg)
92
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration 2
3
x
y 0.9
Temperature = 0.75
0.3
-1.2
![Page 93: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/93.jpg)
93
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration N-1
3
x
y 0.9
Temperature = 0.1
0.3
-1.2
-3
![Page 94: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/94.jpg)
94
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration NReannealing
3
x
y 0.9
Temperature = 1
0.3
-1.2
-3
-2
![Page 95: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/95.jpg)
95
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration NReannealing
3
x
y 0.9
Temperature = 1
0.3
-1.2
-3
-2
27.01
1/))3(2(
Taccepte
P
![Page 96: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/96.jpg)
96
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration NReannealing
3
x
y 0.9
Temperature = 1
0.3
-1.2
-3
-2
![Page 97: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/97.jpg)
97
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration N+1
3
x
y 0.9
Temperature = 0.75
0.3
-1.2
-3
-2
-3
![Page 98: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/98.jpg)
98
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration N+1
3
x
y 0.9
Temperature = 0.75
0.3
-1.2
-3
-2
-3
![Page 99: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/99.jpg)
99
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration …
3
x
y 0.9
Temperature = 0.75
0.3
-1.2
-3
-2
-3
-6.5
![Page 100: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/100.jpg)
100
Simulated Annealing – Peaks Function
![Page 101: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/101.jpg)
101
Global Optimization Toolbox Solvers
GlobalSearch, MultiStart
– Well suited for smooth objective and constraints
– Relies on gradient calculations
– Return the location of local and global minima
ga, simulannealbnd, particleswarm
– Many function evaluations to sample the search space
– Work on both smooth and nonsmooth problems
patternsearch, surrogateopt
– Fewer function evaluations than ga, simulannealbnd, particlewarm
– Work on both smooth and nonsmooth problems
![Page 102: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/102.jpg)
102
Optimization Toolbox Solvers
fmincon, fminbnd, fminunc, fgoalattain, fminimax
– Nonlinear constraints and objectives
– Gradient-based methods for smooth objectives and constraints
quadprog, linprog
– Linear constraints and quadratic or linear objective, respectively
intlinprog
– Linear constraints and objective and integer variables
lsqlin, lsqnonneg
– Constrained linear least squares
lsqnonlin, lsqcurvefit
– Nonlinear least squares
fsolve
– Nonlinear equations
![Page 103: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/103.jpg)
103
Speeding-up with Parallel Computing
Global Optimization Toolbox– patternsearch, surrogateopt, ga,
gamultiobj, particleswarm: Points
evaluated in parallel at each iteration
– MultiStart: Start points evaluated in parallel
Optimization Toolbox
– fmincon, fminunc, fminimax,
fgoalattain, fsolve, lsqcurvefit,
lsqnonlin: Parallel evaluation of objective function
for finite differences
Parallel Computing can also be used in the
Objective Function– parfor
![Page 104: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/104.jpg)
104
Speed up parallel applications
Take advantage of GPUs
Prototype code for your cluster
Parallel Computing Toolbox for the Desktop
Simulink, Blocksets,
and Other Toolboxes
Local
Desktop Computer
MATLAB
……
![Page 105: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/105.jpg)
105
Scale Up to Clusters and Clouds
Cluster
Scheduler
Computer Cluster
…
…
…
…
…
…
… … …
Simulink, Blocksets,
and Other Toolboxes
Local
Desktop Computer
MATLAB
……
![Page 106: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/106.jpg)
106
Learn More about Optimization with MATLAB
Recorded webinar: Optimization in
MATLAB: An Introduction to Quadratic
Programming
Optimization Toolbox Web demo:
Finding an Optimal Path using MATLAB
and Optimization Toolbox
MATLAB Digest: Improving
Optimization Performance with Parallel
Computing
MATLAB Digest: Using Symbolic
Gradients for Optimization
Recorded webinar: Linear and Mixed
Integer Linear Programming in MATLAB
Recorded webinar: Optimization in
MATLAB for Financial Applications
1 2 3 4 50
500
1000
1500
2000
2500
3000
Bonds
# P
urc
ha
se
d
Cash Flow Matching Example
![Page 107: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/107.jpg)
107
Key Takeaways
Solve a wide variety of optimization problems in MATLAB
– Linear and Nonlinear
– Continuous and mixed-integer
– Smooth and Nonsmooth
Find better solutions to multiple minima and non-smooth problems using global
optimization
Use symbolic math for setting up problems and automatically calculating gradients
Using parallel computing to speed up optimization problems
![Page 108: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/108.jpg)
108
MATLAB Answers
Blogs
Cody
File Exchange
and more…
Answers Blogs
ThingSpeak
Every month, over 2 million MATLAB & Simulink users visit MATLAB Central to get questions answered,
download code and improve programming skills.
MATLAB Central Community
MATLAB Answers: Q&A forum; most questions get
answered in only 60 minutes
File Exchange: Download code from a huge repository of
free code including tens of thousands of open source
community files
Cody: Sharpen programming skills while having fun
Blogs: Get the inside view from Engineers who build
and support MATLAB & Simulink
ThingSpeak: Explore IoT Data
And more for you to explore…
Learn ConnectContribute
![Page 109: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/109.jpg)
109
Training: Optimization Techniques in MATLAB
After this 1-day course you will be able to:
Write objective function files and
pass extra parameters
Add different types of constraints
Select an appropriate solver and algorithm
Interpret the output from the solver and
diagnose the progress of an optimization
![Page 110: Solving Optimization Problems with MATLAB · Perform exact computations using familiar MATLAB syntax in MATLAB – Integration – Differentiation – Equation solving ... Genetic](https://reader030.vdocuments.site/reader030/viewer/2022040120/5e915c64e4275d44ae3012c4/html5/thumbnails/110.jpg)
110© 2018 The MathWorks, Inc.