introduction to the calculus of variations
TRANSCRIPT
7/27/2019 Introduction to the Calculus of variations
http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 1/31
CALCULUS OF VARIATIONS IN DISCRETE SPACE
FOR CONSTRAINED NONLINEAR DYNAMIC
OPTIMIZATION
Yixin Chen and Benjamin W. Wah
Department of Electrical and Computer Engineering
and the Coordinated Science LaboratoryUniversity of Illinois at Urbana-Champaign
Urbana, IL 61801, USA
ICTAI 2002, USA
November 4, 2002
7/27/2019 Introduction to the Calculus of variations
http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 2/31
Calculus of Variations in Discrete Space for Constrained Nonlinear Dynamic Optimization
Outline
• Introduction
– Discrete-space dynamic optimization problems
– Existing approaches
• Calculus of variations in discrete time and discrete state space
– Necessary and sufficient conditions for constrained local minima
– Variational search algorithm with node dominance
• Implementations in ASPEN
– Some sample results
• Conclusions
Yixin Chen and Benjamin W. Wah 1
7/27/2019 Introduction to the Calculus of variations
http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 3/31
INTRODUCTION
7/27/2019 Introduction to the Calculus of variations
http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 4/31
Calculus of Variations in Discrete Space for Constrained Nonlinear Dynamic Optimization Introduction
Dynamic Optimization Problems
Optimization problems with time varying dynamic variables
Control theory and
classical theory of calculus of variations
StateTime
DiscreteContinuous
Continuous
Continuous−state
DiscreteDiscrete−state
Continuous−state
Discrete−time
Continuous−time
Continuous−time
Discrete−time
Discrete−state
Yixin Chen and Benjamin W. Wah 3
7/27/2019 Introduction to the Calculus of variations
http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 5/31
Calculus of Variations in Discrete Space for Constrained Nonlinear Dynamic Optimization Introduction
Discrete-Time Discrete-State Constrained Dynamic Problems
General Constraints I(Y) = 0
Constraints
G(0) G(1) G(N)
Functional
Objective
F(0) F(1) F(N)
Lagrange
y(0) y(1) y(N) y(N+1)
E(N+1)E(N)E(1)Constraints
E(0)
Local
minimize J [{y( j)}] =N
j=0
F ( j, y( j + 1), y( j)) (1)
s.t. G( j, y( j + 1), y( j)) = 0, j = 0, 1, · · · , N (2)
E ( j, y(y)) = 0, (3)I (Y ) = 0 (4)
where y( j) is defined in discrete space Y ,F
,G
andI
arenot
necessarily continuous or differentiableYixin Chen and Benjamin W. Wah 4
7/27/2019 Introduction to the Calculus of variations
http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 6/31
Calculus of Variations in Discrete Space for Constrained Nonlinear Dynamic Optimization Introduction
Unconstrained Problems or Problems with Lagrange Constraints
• Path Dominance: Principle of Optimality
– Principle of Optimality in dynamic programming applied on feasible state c
J 2 ≤ J 1 =⇒ P 2 → P 1
s
P 2
P 1
J 2J 1 c
Stage t If c lies on the optimalpath between s and d and
• Polynomial worst-case complexity: O
N |Y|2
Yixin Chen and Benjamin W. Wah 5
7/27/2019 Introduction to the Calculus of variations
http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 7/31
Calculus of Variations in Discrete Space for Constrained Nonlinear Dynamic Optimization Introduction
Problems with General Constraints
• Path dominance not applicable
– A dominating path may become infeasible due to general constraints
– Exponential worst-case complexity: O
|Y|N
, assuming NP hard
• Node dominance applicable
node
relationdominance
node dominance relations
c2
P 1c1
Termination of c1 by
– Worst-case complexity without path dominance: O
N |Y||s|N
∗ Assuming |s| nodes are not pruned in each stage
∗ Substantially less than O(|Y|N ) when |s| |Y|
Yixin Chen and Benjamin W. Wah 6
7/27/2019 Introduction to the Calculus of variations
http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 8/31
Calculus of Variations in Discrete Space for Constrained Nonlinear Dynamic Optimization Introduction
Benefits of Using Node Dominance
Dominated nodes at each stage
| s | N + 2
b u n d l e s
s
Utilizing node dominance
Not utilizing node dominance Dominating nodes at each stage
Stage 0 Stage 1 Stage N + 1
Y Y Y
ss
Complexity of each iteration is (N + 2)|Y|
| Y | N + 2
b u n d l e s
Yixin Chen and Benjamin W. Wah 7
7/27/2019 Introduction to the Calculus of variations
http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 9/31
Calculus of Variations in Discrete Space for Constrained Nonlinear Dynamic Optimization Introduction
Worst-Case Complexities in Path and Node Dominance
Constraint Without General Constraints With General ConstraintsType (Conventional DP) (Without Path Dominance)
Dominance With Path With Path and Without Node With NodeUsed Dominance Node Dominance Dominance Dominance
Complexity O
N |Y|2
O
N |Y| + N |s|2
O
|Y|N
O
N |Y||s|N
Node dominance works well when |s| |Y|
Yixin Chen and Benjamin W. Wah 8
7/27/2019 Introduction to the Calculus of variations
http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 10/31
CALCULUS OF VARIATIONS IN DISCRETE SPACE
7/27/2019 Introduction to the Calculus of variations
http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 11/31
Calculus of Variations in Discrete Space for Constrained Nonlinear Dynamic Optimization Calculus of Variations in Discrete Space
Constrained Local-Minimum Bundle in Discrete Space
• State vector s in discrete state space Y has user-defined discrete neighborhood
• Discrete neighborhood of bundle Y = {y( j)} is the union of discrete
neighborhoods of all stages, each defined on neighborhoods of states in each stage
Bundle Y
• Bundle Y is a constrained local minimum in discrete space (CLM dn) if
– Y is feasible
– No feasible bundle in N b(Y ) has better functional value than J [Y ]
Yixin Chen and Benjamin W. Wah 10
7/27/2019 Introduction to the Calculus of variations
http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 12/31
Calculus of Variations in Discrete Space for Constrained Nonlinear Dynamic Optimization Calculus of Variations in Discrete Space
Solving Overall Problem using Discrete-Space Lagrangian Theory
• First-order necessary and sufficient conditions based on the theory of Lagrangemultipliers in discrete space
Saddle Point Conditions(SP dn)
First-order conditions ⇐⇒
CLM dn ⇐⇒
O(|Y |N )
Overall Search Space
Yixin Chen and Benjamin W. Wah 11
7/27/2019 Introduction to the Calculus of variations
http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 13/31
Calculus of Variations in Discrete Space for Constrained Nonlinear Dynamic Optimization Calculus of Variations in Discrete Space
Intuitive Meaning Behind Saddle Points
dynamicallyvarying
(h(x) = 0)
(∆xL(x, λ) = 0)
minx[f (x) + λT H (h(x))]↓
λ↑λ↑
Penalties λ are constraints are satisfied
Gradient descents in x space
Equilibrium point where
to reduce objective function
space to increase penalties
on violated constraints
Gradient ascents in λ
and constraint violations
and objective is minimum
Yixin Chen and Benjamin W. Wah 12
7/27/2019 Introduction to the Calculus of variations
http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 14/31
Calculus of Variations in Discrete Space for Constrained Nonlinear Dynamic Optimization Calculus of Variations in Discrete Space
Discrete-Time Discrete-State Euler-Lagrange Equation
• Decompose first-order condition into Discrete-Space Euler-Lagrange Equations
(ELE dn) for each stage
• Decompose SP dn in into N + 2 distributed saddle-point conditions DSP dn
• Distributed necessary and sufficient conditions for CLM dn
¨
©CLM dn ≡ ELE dn ≡ DSP dn
– Only necessary when there are general constraints
• Differences from continuous variational calculus theory
– Do not require differentiability or continuity of functions
– Continuous ELE conditions are only necessary but not sufficient, even in the
absence of general constraints
Yixin Chen and Benjamin W. Wah 13
7/27/2019 Introduction to the Calculus of variations
http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 15/31
Calculus of Variations in Discrete Space for Constrained Nonlinear Dynamic Optimization Calculus of Variations in Discrete Space
Solving Distributed Subproblems Using Node Dominance
General Constraints
| s | N + 2
b u n d l e s
s
Y
Stage N+1
| Y | N + 2
b u n d l e s
s s
Y Y
Stage 0 Stage 1
Yixin Chen and Benjamin W. Wah 14
7/27/2019 Introduction to the Calculus of variations
http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 16/31
Calculus of Variations in Discrete Space for Constrained Nonlinear Dynamic Optimization Calculus of Variations in Discrete Space
Heuristic Search Procedure For Finding DSP dn
stopY
N
Y
N
Stopping condition
met?
Generate new candidate(s)
in the µ subspace µ loop
Search in
µ subspace?
Start in stage 0 in stage 1
Find SP dn Find SP dn Find SP dn
in stage N + 1
Local Descent in x subspace
Local Ascent in λ subspace
Constrained Simulated Annealing(CSA):
Yixin Chen and Benjamin W. Wah 15
7/27/2019 Introduction to the Calculus of variations
http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 17/31
DEMONSTRATIONS ON ASPEN
7/27/2019 Introduction to the Calculus of variations
http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 18/31
Calculus of Variations in Discrete Space for Constrained Nonlinear Dynamic Optimization Demonstrations on ASPEN
ASPEN Planner
• Automated Scheduling and Planning Environment at Jet Propulsion Laboratory
• ASPEN models have discrete time horizons
– Each time point is a stage
– Adjacent time points can be collapsed into a single stage
– Current implementation: maximum 100 stages• ASPEN repair/optimization actions provide promising descent directions in
state-variable subspace
– Repair actions: resolve conflicts
– Optimization actions: optimize preferences
• ASPEN does not have the UNDO mechanism
Yixin Chen and Benjamin W. Wah 17
7/27/2019 Introduction to the Calculus of variations
http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 19/31
Calculus of Variations in Discrete Space for Constrained Nonlinear Dynamic Optimization Demonstrations on ASPEN
Distributed Lagrangian Formulation
• Assign each conflict ci a unique Lagrange Multiplier λi
• Augmented distributed Lagrangian function of stage t:
Γdn(t) = −ws · Score +
ci∈C (t)
λi ∗ H (ci) +
ci∈C (t)
H (ci)2
2
– ws: weight of score
– Score: preference score of schedule
– C (t): set of conflicts whose time duration intersects with stage t
– H (ci): non-negative value assigned to a conflict reflecting its degree of violation
∗ H (ci) = 1 in current implementation
Yixin Chen and Benjamin W. Wah 18
7/27/2019 Introduction to the Calculus of variations
http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 20/31
Calculus of Variations in Discrete Space for Constrained Nonlinear Dynamic Optimization Demonstrations on ASPEN
Distributed Heuristic Search for Finding SP dn in Stage t
• Descent of Γdn(t) in state subspace
– Choose probabilistically from repair actions and optimization actions
– Select random feasible action at each choice point– Apply selected action to current schedule in a child process
– Evaluate Γdn(t) of the new schedule
– Accept new schedule according to Metropolis probability controlled by ageometrically decreasing temperature
– Repeat action in parent process if accepted; otherwise, discard result of child
process (overcome lack of UNDO but limited to 3685 forks)
• Ascent of Γdn(t) in Lagrange-multiplier subspace
λi ←− λi + αiH (ci)
Yixin Chen and Benjamin W. Wah 19
7/27/2019 Introduction to the Calculus of variations
http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 21/31
SOME SAMPLE RESULTS
7/27/2019 Introduction to the Calculus of variations
http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 22/31
Calculus of Variations in Discrete Space for Constrained Nonlinear Dynamic Optimization Some Sample Results
Benchmark CX1-PREF
• Citizen Explorer-I satellite design and operation planning benchmark
–Multiple competing preferences to be optimized
– Problem generator to generate different problem instances
perl probgen.pl < random seed > < number of orbits >
• ASPEN search setting:
a) Find feasible schedule using repair
b) Optimize score using optimize (default 200 iterations)
c) repeat (a) and (b)
Yixin Chen and Benjamin W. Wah 21
7/27/2019 Introduction to the Calculus of variations
http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 23/31
Calculus of Variations in Discrete Space for Constrained Nonlinear Dynamic Optimization Some Sample Results
Best Feasible Solution on an 8-Orbit Problem
0.7488
0.749
0.74920.7494
0.7496
0.7498
0.75
0.7502
0 10000 20000
B e s t F e a s i b l e S c o r e
Iteration
ASPENASPEN+DCV+CSA
CSA
Yixin Chen and Benjamin W. Wah 22
7/27/2019 Introduction to the Calculus of variations
http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 24/31
Calculus of Variations in Discrete Space for Constrained Nonlinear Dynamic Optimization Some Sample Results
Search Progress on an 8-Orbit Problem
• Conflicts vs. Iteration:
0
100
200
300400
500
600
0 10000 20000
N u m b e r o f C o
n f l i c t s
Iteration
ASPEN solving CX1-PREF Benchmark with 8 Orbits
ASPEN
0
100
200
300400
500
600
0 10000 20000
N u m b e r o f C o
n f l i c t s
Iteration
ASPEN+DCV+CSA solving CX1-PREF Benchmark with 8 Orbits
ASPEN+DCV+CSA
• Score vs. Iteration:
0.30.350.4
0.450.5
0.550.6
0.650.7
0.750.8
0 10000 20000
O b j e c
t i v e S c o r e
Iteration
ASPEN solving CX1-PREF Benchmark with 8 Orbits
ASPEN
0.30.350.4
0.450.5
0.550.6
0.650.7
0.750.8
0 10000 20000
O b j e c
t i v e S c o r e
Iteration
ASPEN+DCV+CSA solving CX1-PREF Benchmark with 8 Orbits
ASPEN+DCV+CSA
Yixin Chen and Benjamin W. Wah 23
7/27/2019 Introduction to the Calculus of variations
http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 25/31
Calculus of Variations in Discrete Space for Constrained Nonlinear Dynamic Optimization Some Sample Results
Best Feasible Solution on a 16-Orbit Problem
0.45
0.5
0.55
0.6
0.65
0.7
0.75
0 10000 20000 30000 40000 50000
B e s t F e a s i b l e S c o r e
Iteration
ASPENASPEN+DCV+CSA
CSA
Yixin Chen and Benjamin W. Wah 24
7/27/2019 Introduction to the Calculus of variations
http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 26/31
7/27/2019 Introduction to the Calculus of variations
http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 27/31
Calculus of Variations in Discrete Space for Constrained Nonlinear Dynamic Optimization Some Sample Results
Best Feasible Solution on OPTIMIZE Benchmark
0.8
0.81
0.82
0.83
0.84
0.85
0.86
0 10000 20000 30000 40000 50000
B e s t F e a s i b l e S c o r e
Iteration
ASPENASPEN+DCV+CSA
CSA
Yixin Chen and Benjamin W. Wah 26
7/27/2019 Introduction to the Calculus of variations
http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 28/31
Calculus of Variations in Discrete Space for Constrained Nonlinear Dynamic Optimization Some Sample Results
Search Progress on OPTIMIZE Benchmark
• Conflicts vs. Iteration:
0
2
4
6
8
10
12
0 10000 20000 30000 40000 50000
N u m b e r o f C o
n f l i c t s
Iteration
ASPEN solving OPTIMIZE Benchmark
ASPEN
0
2
4
6
8
10
12
0 10000 20000 30000 40000 50000
N u m b e r o f C o
n f l i c t s
Iteration
ASPEN+DCV+CSA solving OPTIMIZE Benchmark
ASPEN+DCV+CSA
• Score vs. Iteration:
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
0 10000 20000 30000 40000 50000
O b j e c t i v e S c o r e
Iteration
ASPEN solving OPTIMIZE Benchmark
ASPEN0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
0 10000 20000 30000 40000 50000
O b j e c t i v e S c o r e
Iteration
ASPEN+DCV+CSA solving OPTIMIZE Benchmark
ASPEN+DCV+CSA
Yixin Chen and Benjamin W. Wah 27
7/27/2019 Introduction to the Calculus of variations
http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 29/31
Calculus of Variations in Discrete Space for Constrained Nonlinear Dynamic Optimization Some Sample Results
Best Feasible Solution on PREF Benchmark
0.48
0.5
0.52
0.54
0.56
0.58
0.6
0 10000 20000 30000 40000 50000
B e s t F e a s i b l e S c o r e
Iteration
ASPEN+DCV+CSAASPEN
CSA
Yixin Chen and Benjamin W. Wah 28
C l l f V D S f C d N l D O S S l R l
7/27/2019 Introduction to the Calculus of variations
http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 30/31
Calculus of Variations in Discrete Space for Constrained Nonlinear Dynamic Optimization Some Sample Results
Search Progress on PREF Benchmark
• Conflicts vs. Iteration:
0
5
10
15
20
25
30
0 10000 20000 30000 40000 50000
N u m b e r o f C o
n f l i c t s
Iteration
ASPEN solving PREF Benchmark
ASPEN
0
5
10
15
20
25
30
0 10000 20000 30000 40000 50000
N u m b e r o f C o
n f l i c t s
Iteration
ASPEN+DCV+CSA solving PREF Benchmark
ASPEN+DCV+CSA
• Score vs. Iteration:
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0.55
0.6
0 10000 20000 30000 40000 50000
O b j e c t i v e S c o r e
Iteration
ASPEN solving PREF Benchmark
ASPEN
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0.55
0.6
0 10000 20000 30000 40000 50000
O b j e c t i v e S c o r e
Iteration
ASPEN+DCV+CSA solving PREF Benchmark
ASPEN+DCV+CSA
Yixin Chen and Benjamin W. Wah 29
C l l f V i ti i Di t S f C t i d N li D i O ti i ti C l i
7/27/2019 Introduction to the Calculus of variations
http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 31/31
Calculus of Variations in Discrete Space for Constrained Nonlinear Dynamic Optimization Conclusions
Conclusions
• Extension of calculus of variations in continuous space to discrete space
• Partitioning general necessary conditions into distributed necessary conditions
• Relying on theory of Lagrange multipliers for discrete constrained optimization
• Significant reduction in the base of the exponential complexity
• Significant improvement in search times with equal or better quality in planning
and scheduling problems as compared to those of ASPEN
Yixin Chen and Benjamin W. Wah 30