optimization in matlab - mathworks · learn more about optimization with matlab matlab digest:...
TRANSCRIPT
-
1© 2014 The MathWorks, Inc.
Optimization in MATLAB
Seth DeLand
-
2
Topics
Intro
Using gradient-based solvers
Optimization in Comp. Finance toolboxes
Global optimization
Speeding up your optimizations
-
3
Optimization workflow
Initial
Design
Variables
Model
Modify
Design
Variables
Optimal
DesignObjectives
met?
No
Yes
-
4
Optimization Problems
Minimize Risk
Maximize Profits
Maximize Fuel Efficiency
-
5
Demo: Solving an optimization problem
Simple objective function of 2 variables:
𝑓 𝑥 = log 1 + 3 𝑥2 − 𝑥13− 𝑥1
2
+ 𝑥1 −4
3
2
Bound constraints:
−2.5 ≤ 𝑥1 ≤ 2.5
−2.5 ≤ 𝑥2 ≤ 2.5
-2
-1
0
1
2
-2
-1
0
1
2
0
2
4
6
-
6
Portfolio Optimization – Quadratic Constraints
Maximize returns, constraint is risk
minimize − 𝑓𝑇𝑥 𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜1
2𝑥𝑇𝐶𝑥 ≤ 𝐶𝑚𝑎𝑥
Use fmincon interior-point algorithm
– Specify analytic gradients of objective function and constraints
𝑔 = −𝑓 (gradient of objective function)
𝑔𝑐 = 𝐶𝑥 (gradient of constraint)
– Hessian (H) of Lagrangian L = −𝑓𝑇𝑥 + λ(
1
2𝑥𝑇𝐶𝑥 − 𝐶𝑚𝑎𝑥)
H = ∇2 L = λC
This is a second order conic programming problem
-
7
Demo: Cash-flow matching
Idea: Buy bonds to cover pension
fund obligations
Variables: How many of each
bond to buy?
Constraints: Payments from
bonds must be greater than or
equal to pension fund obligations
Objective: Minimize the size of
the investment you make 1 2 3 4 5 6 7 80
20
40
60
80
100
120
Time Period
Ca
sh
Flo
w (
$)
1 2 3 4 5 6 7 80
5
10
15x 10
5
Time Period
Pa
ym
en
ts (
$)
-
8
𝑛1⋮𝑛5
Extending the problem
−𝑐𝑎𝑠ℎ𝐹𝑙𝑜𝑤𝑠
𝑛1⋮𝑛5
≤ −𝑜𝑏𝑙𝑖𝑔𝑎𝑡𝑖𝑜𝑛𝑠
𝑛1⋮𝑛5𝑦1⋮𝑦5
1
0⋱
0
1
−1000
0⋱
0
−1000
𝑛1⋮𝑛5𝑦1⋮𝑦5
=0⋮0
min𝑥
𝑓𝑇𝑥 𝐴𝑥 ≤ 𝑏
𝐴𝑒𝑞𝑥 = 𝑏𝑒𝑞𝑙𝑏 ≤ 𝑥 ≤ 𝑢𝑏
s.t.
𝑥(intcon) must be integers𝑥:
𝑛1 = 1000𝑦1⋮
𝑛5 = 1000𝑦5
−𝑐𝑎𝑠ℎ𝐹𝑙𝑜𝑤𝑠 𝑧𝑒𝑟𝑜𝑠(8,5)
𝑛1⋮
𝑛5𝑦1⋮
𝑦5
≤ −𝑜𝑏𝑙𝑖𝑔𝑎𝑡𝑖𝑜𝑛𝑠
-
9
Optimization Toolbox solvers
Linear and Mixed-Integer
• LINPROG
• INTLINPROG
Quadratic
• QUADPROG
Nonlinear
• FMINCON
• FMINUNC
• FMINBND
• FMINSEARCH
• FSEMINF
Least Squares
• LSQLIN
• LSQNONNEG
• LSQCURVEFIT
• LSQNONLIN
Nonlinear Equation Solving
• FSOLVE
• FZERO
Multiobjective
• FGOALATTAIN
• FMINIMAX
-
10
Key optimization problems addressed by
financial toolboxes
• Mean-variance portfolio optimization
• Conditional Value at Risk (CVaR) portfolio optimization
Financial Toolbox
• Parameter estimation of conditional mean/variance models
Econometrics Toolbox
• Hedging portfolios
• Fitting interest rate curves
• Bootstrapping
Financial Instruments
Toolbox
-
11
-2
0
2
-3-2
-10
12
3
-6
-4
-2
0
2
4
6
8
x
Peaks
y
Global optimization algorithms
MultiStart
GlobalSearch
Patternsearch
Genetic Algorithm
Simulated Annealing
Local minimaGlobal
minima
-
12
What is MultiStart?
Run a local solver from
each set of start points
Option to filter starting
points based on feasibility
Supports parallel
computing
-
13
MultiStart demo – nonlinear regression
lsqcurvefit solution MultiStart solution
-
14
What is Pattern Search?
An approach that uses a
pattern of search directions
around the existing points
Expands/contracts around
the current point when a
solution is not found
Does not rely on gradients:
works on smooth and
nonsmooth problems
-
15
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Pattern Search overview – Iteration 1Run from specified x0
x
y
3
-
16
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Pattern Search overview – Iteration 1Apply pattern vector, poll new points for improvement
x
y
3
Mesh size = 1
Pattern vectors = [1,0], [0,1], [-1,0], [0,-1]
0_*_ xvectorpatternsizemeshPnew
0]0,1[*1 x
1.6
0.4
4.6
2.8
First poll successful
Complete Poll (not default)
-
17
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Pattern Search overview – Iteration 2
x
y
3
Mesh size = 2
Pattern vectors = [1,0], [0,1], [-1,0], [0,-1]
1.6
0.4
4.6
2.8
-4
0.3-2.8
Complete Poll
-
18
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Pattern Search overview – Iteration 3
x
y
3
Mesh size = 4
Pattern vectors = [1,0], [0,1], [-1,0], [0,-1]
1.6
0.4
4.6
2.8
-4
0.3-2.8
-
19
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Pattern Search overview – Iteration 4
x
y
3
Mesh size = 4*0.5 = 2
Pattern vectors = [1,0], [0,1], [-1,0], [0,-1]
1.6
0.4
4.6
2.8
-4
0.3-2.8
-
20
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Pattern Search overview – Iteration NContinue expansion/contraction until convergence…
x
y
31.6
0.4
4.6
2.8
-4
0.3-2.8
-6.5
-
21
Patternsearch demo – stochastic function
Stochastic objective function
-
22
What is a Genetic Algorithm?
Uses concepts from
evolutionary biology
Start with an initial generation
of candidate solutions that are
tested against the objective
function
Subsequent generations
evolve from the 1st through
selection, crossover and
mutation
-
23
How evolution works – binary case
Selection
– Retain the best performing bit strings from one generation to the next.
Favor these for reproduction
– parent1 = [ 1 0 1 0 0 1 1 0 0 0 ]
– parent2 = [ 1 0 0 1 0 0 1 0 1 0 ]
Crossover
– parent1 = [ 1 0 1 0 0 1 1 0 0 0 ]
– parent2 = [ 1 0 0 1 0 0 1 0 1 0 ]
– child = [ 1 0 0 0 0 1 1 0 1 0 ]
Mutation
– parent = [ 1 0 1 0 0 1 1 0 0 0 ]
– child = [ 0 1 0 1 0 1 0 0 0 1 ]
-
24
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Genetic Algorithm – Iteration 1Evaluate initial population
x
y
-
25
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Genetic Algorithm – Iteration 1Select a few good solutions for reproduction
x
y
-
26
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Genetic Algorithm – Iteration 2Generate new population and evaluate
x
y
-
27
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Genetic Algorithm – Iteration 2
x
y
-
28
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Genetic Algorithm – Iteration 3
x
y
-
29
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Genetic Algorithm – Iteration 3
x
y
-
30
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Genetic Algorithm – Iteration NContinue process until stopping criteria are met
x
y
Solution found
-
31
Genetic Algorithm demo - Multiobjective
Use gamultiobj to find Pareto front
Two competing objectives:
Second objective has sinusoidal
component – results in
discontinuous front
-
32
Global Optimization
Solvers designed to explore the solution space and find
global solutions
Problems can be stochastic, or nonsmooth
Solvers in Global Optimization Toolbox
– MultiStart
– GlobalSearch
– simulannealbnd
– patternsearch
– ga (single- and multi-objective)
Parallel Computing with ga, patternsearch, MultiStart
-
33
Speeding up with Parallel Computing Toolbox
Global Optimization Toolbox:– ga: Members of population evaluated in parallel at each generation
– patternsearch: Pattern evaluated in parallel (CompletePoll == on)
– MultiStart: Start points evaluated in parallel
Optimization Toolbox:– fmincon: parallel evaluation of objective function for finite differences
– fminimax, fgoalattain: same as fmincon
In the objective function– parfor
-
34
Key takeaways
Solvers for a wide variety of problems
– New Mixed-Integer Linear Programming Solver
Supply gradient and Hessian if possible
Global solvers for multiple minimum
and nonsmooth problems
Speed up with parallel computing
-
35
Learn more about optimization with MATLAB
MATLAB Digest: Using Symbolic
Gradients for Optimization
Recorded webinar: Tips and
Tricks – Getting Started Using
Optimization with MATLAB
Recorded webinar: Global
Optimization with MATLAB
Products
MATLAB Digest: Improving
Optimization Performance with
Parallel Computing