1/29

The L-Shaped method: A tutorial

Jesus A. Rodriguez

Nov. 9th 2017

CIRRELT/GERAD/MORSC Student Seminar

2/29

Outline

Introduction

Benders decomposition and L-Shaped method

Industrial example

3/29

Two-stage stochastic programs

Example 1: The Generator Maintenance Scheduling Problem (GMSP)

Stage 2: !Inflows scenarios and !hydropower operation!

Stage 1: !Maintenance

schedule!

Scenario 2!

Time periods!

Gene

rato

r out

ages!

Time periods!

Time periods!

Time periods!

Inflo

ws!

Inflo

ws!

Inflo

ws!

4/29

Two-stage stochastic programs (cont.)

Example 2: Stochastic Vehicle Routing (SVRP)

d1=?!d2=?!

d3=?!

d4=?!d5=?!

K = 12!

d1=?!d2=?!

d3=?!

d4=?!d5=?!

d1=6!d2=?!

d3=5!

d4=3!d5=?!

d1=3!d2=?!

d3=4!

d4=3!d5=4!

Scenario 1 !

Stage 1!Given a set of customers !with uncertain demands,

plan the route!

Scenario 2!

Stage 2!Visit the customers,

observe the demand and take recourse action !

5/29

Extended formulationIn general,

Stage 1: Non-anticipative decisionsStage 2: ∀ scenario ω ∈ Ω, uncertain parameters ⇒ recourse decisions.

If the problem is linear with W =| Ω | scenarios, its extensive form is

minxω

f ᵀy +∑ω∈Ω

pωcᵀxω

s.t. Bωy +Axω ≥ ξω, ∀ω ∈ Ω,

xω ≥ 0, ∀ω ∈ Ω,y ∈ S.

We wan to exploit its dual block angular structureBᵀ

1 Aᵀ1

Bᵀ2 Aᵀ

2...

. . .

BᵀW Aᵀ

W

5/29

Extended formulationIn general,

Stage 1: Non-anticipative decisionsStage 2: ∀ scenario ω ∈ Ω, uncertain parameters ⇒ recourse decisions.

If the problem is linear with W =| Ω | scenarios, its extensive form is

minxω

f ᵀy +∑ω∈Ω

pωcᵀxω

s.t. Bωy +Axω ≥ ξω, ∀ω ∈ Ω,

xω ≥ 0, ∀ω ∈ Ω,y ∈ S.

We wan to exploit its dual block angular structureBᵀ

1 Aᵀ1

Bᵀ2 Aᵀ

2...

. . .

BᵀW Aᵀ

W

5/29

Extended formulationIn general,

Stage 1: Non-anticipative decisionsStage 2: ∀ scenario ω ∈ Ω, uncertain parameters ⇒ recourse decisions.

If the problem is linear with W =| Ω | scenarios, its extensive form is

minxω

f ᵀy +∑ω∈Ω

pωcᵀxω

s.t. Bωy +Axω ≥ ξω, ∀ω ∈ Ω,

xω ≥ 0, ∀ω ∈ Ω,y ∈ S.

We wan to exploit its dual block angular structureBᵀ

1 Aᵀ1

Bᵀ2 Aᵀ

2...

. . .

BᵀW Aᵀ

W

6/29

Compact representation

This problem can be compactly represented as

miny ∈S

f ᵀy + Q(y),

where Q(y) is the expected value function

Q(y) = EωQω(y),

and for each scenario ω,

Qω(y) = minxω

cᵀxω

s.t. Bωy + Axω ≥ ξωxω ≥ 0.

6/29

Compact representation

This problem can be compactly represented as

miny ∈S

f ᵀy + Q(y),

where Q(y) is the expected value function

Q(y) = EωQω(y),

and for each scenario ω,

Qω(y) = minxω

cᵀxω

s.t. Bωy + Axω ≥ ξωxω ≥ 0.

7/29

Benders decomposition: basic idea

Given

minx,y

cᵀx + f ᵀy

s.t. Ax + By ≥ b,

x ≥ 0,

y ∈ S ,

(P)

where S is a non-convex set.We fix y = y , to obtain the subproblems:

Q(y) = minx

cᵀx

s.t. Ax ≥ b − By

x ≥ 0,

(SP)

And we use the subproblem solution to approximate Q(y).

7/29

Benders decomposition: basic idea

Given

minx,y

cᵀx + f ᵀy

s.t. Ax + By ≥ b,

x ≥ 0,

y ∈ S ,

(P)

where S is a non-convex set.We fix y = y , to obtain the subproblems:

Q(y) = minx

cᵀx

s.t. Ax ≥ b − By

x ≥ 0,

(SP)

And we use the subproblem solution to approximate Q(y).

8/29

Graphical representation for 1 variable y

y

z(y)

y

z(y)

y

z(y)

y

z(y)

9/29

Graphical representation for 1 variable y

y

z(y)

y

z(y)

y

z(y)

y

z(y)

10/29

Graphical representation for 1 variable yy

z(y)

y

z(y)

y

z(y)

y

z(y)

11/29

Graphical representation for 1 variable yy

z(y)

y

z(y)

y

z(y)

y

z(y)

12/29

Graphical representation for 1 variable y

y

z(y)

y

z(y)

13/29

Graphical representation for 1 variable y

y

z(y)

y

z(y)

y

z(y)

Optimalsolution

14/29

Graphical representation for 1 variable yy

z(y)

y

z(y)

y

z(y)

Optimalsolution

15/29

Benders decomposition: feasibility

Q(y) = minx

cᵀx

s.t. Ax ≥ b − By

x ≥ 0,

To guarantee the feasibility of SP,

y ∈ R =y ∈ S | ∃ x ≥ 0; Ax ≥ b − By

,

=y ∈ S | [b − By ]ᵀu ≤ 0 | u ≥ 0;Aᵀu ≤ 0

by Farkas Lemma.

Since the polyhedral cone C = u ≥ 0;Aᵀu ≤ 0 has a finite number ofgenerators

R =y ∈ S | [b − By ]ᵀuri ≤ 0,∀i ∈ R

(1)

15/29

Benders decomposition: feasibility

Q(y) = minx

cᵀx

s.t. Ax ≥ b − By

x ≥ 0,

To guarantee the feasibility of SP,

y ∈ R =y ∈ S | ∃ x ≥ 0; Ax ≥ b − By

,

=y ∈ S | [b − By ]ᵀu ≤ 0 | u ≥ 0;Aᵀu ≤ 0

by Farkas Lemma.Since the polyhedral cone C = u ≥ 0;Aᵀu ≤ 0 has a finite number ofgenerators

R =y ∈ S | [b − By ]ᵀuri ≤ 0,∀i ∈ R

(1)

15/29

Benders decomposition: feasibility

Q(y) = minx

cᵀx

s.t. Ax ≥ b − By

x ≥ 0,

To guarantee the feasibility of SP,

y ∈ R =y ∈ S | ∃ x ≥ 0; Ax ≥ b − By

,

=y ∈ S | [b − By ]ᵀu ≤ 0 | u ≥ 0;Aᵀu ≤ 0

by Farkas Lemma.Since the polyhedral cone C = u ≥ 0;Aᵀu ≤ 0 has a finite number ofgenerators

R =y ∈ S | [b − By ]ᵀuri ≤ 0,∀i ∈ R

(1)

16/29

Benders decomposition: derivation

P = minx,y

f (y) + cᵀx | Ax ≥ b − By ; x ≥ 0; y ∈ S

= miny∈R

f (y) + min

xcᵀx | Ax ≥ b − By ; x ≥ 0

= min

y∈R

f (y) + max

u[b − By ]ᵀu | Aᵀu ≤ c ; u ≥ 0

= min

y∈R

f (y) + max

p∈P[b − By ]ᵀup

= min

y∈R

f (y) + wSP | wSP ≥ [b − By ]ᵀup ∀ p ∈ P

16/29

Benders decomposition: derivation

P = minx,y

f (y) + cᵀx | Ax ≥ b − By ; x ≥ 0; y ∈ S

= min

y∈R

f (y) + min

xcᵀx | Ax ≥ b − By ; x ≥ 0

= miny∈R

f (y) + max

u[b − By ]ᵀu | Aᵀu ≤ c ; u ≥ 0

= min

y∈R

f (y) + max

p∈P[b − By ]ᵀup

= min

y∈R

f (y) + wSP | wSP ≥ [b − By ]ᵀup ∀ p ∈ P

16/29

Benders decomposition: derivation

P = minx,y

f (y) + cᵀx | Ax ≥ b − By ; x ≥ 0; y ∈ S

= min

y∈R

f (y) + min

xcᵀx | Ax ≥ b − By ; x ≥ 0

= min

y∈R

f (y) + max

u[b − By ]ᵀu | Aᵀu ≤ c ; u ≥ 0

= miny∈R

f (y) + max

p∈P[b − By ]ᵀup

= min

y∈R

f (y) + wSP | wSP ≥ [b − By ]ᵀup ∀ p ∈ P

16/29

Benders decomposition: derivation

P = minx,y

f (y) + cᵀx | Ax ≥ b − By ; x ≥ 0; y ∈ S

= min

y∈R

f (y) + min

xcᵀx | Ax ≥ b − By ; x ≥ 0

= min

y∈R

f (y) + max

u[b − By ]ᵀu | Aᵀu ≤ c ; u ≥ 0

= min

y∈R

f (y) + max

p∈P[b − By ]ᵀup

= miny∈R

f (y) + wSP | wSP ≥ [b − By ]ᵀup ∀ p ∈ P

16/29

Benders decomposition: derivation

P = minx,y

f (y) + cᵀx | Ax ≥ b − By ; x ≥ 0; y ∈ S

= min

y∈R

f (y) + min

xcᵀx | Ax ≥ b − By ; x ≥ 0

= min

y∈R

f (y) + max

u[b − By ]ᵀu | Aᵀu ≤ c ; u ≥ 0

= min

y∈R

f (y) + max

p∈P[b − By ]ᵀup

= min

y∈R

f (y) + wSP | wSP ≥ [b − By ]ᵀup ∀ p ∈ P

17/29

Benders decomposition

The Relaxed Master Problem (RMP) results from relaxing dualconstraints (optimality cuts and feasiblity cuts) in MP. The most violatedconstraints can be iteratively computed and included from the dualinformation of the subproblem SP,

Q(y) = minx

cᵀx

s.t. Ax ≥ b − By

x ≥ 0,

17/29

Benders decomposition

The Relaxed Master Problem (RMP) results from relaxing dualconstraints (optimality cuts and feasiblity cuts) in MP. The most violatedconstraints can be iteratively computed and included from the dualinformation of the subproblem SP,

Q(y) = minx

cᵀx

s.t. Ax ≥ b − By

x ≥ 0,

18/29

Example: Generator Maintenance Scheduling

Relaxed!Master Problem (RMP)!

!Maintenance scheduling!

!+ optimality cuts (up)!+ feasibility cuts (ur)!

!Combinatorial optimization

problem!

Subproblems (SP)!!

Hydro power operation!!

- Fixed maintenance schedule (y)!

!

Stochastic linear program decomposed by scenario

Candidate maintenance schedules (y*)

Dual solutions (up)!Extreme directions (ur)!

Coordinatedhydropoweroperation&maintenanceschedulingviadecomposition(HOMS)

19/29

Benders algorithm

Solution algorithm

1. Solve the relaxed MPI If unfeasible, STOP. P is also unfeasible.I Otherwise, read solution y k and optimal value zk (LB).

2. Solve SPω(yk) ∀ ω ∈ Ω.

I If unfeasible, read extreme dual ray urω(y

k).I If optimal, read dual solution up

ω(yk) and solution value zω(y

k).

3. Use zω(yk) to compute UB

4. If stopping criteria are met (UB − LB ≤ ε; time; iterations), STOP.

5. Otherwise, compute cuts and add cuts to MP. Go to step 1.

19/29

Benders algorithm

Solution algorithm

1. Solve the relaxed MP

I If unfeasible, STOP. P is also unfeasible.I Otherwise, read solution y k and optimal value zk (LB).

2. Solve SPω(yk) ∀ ω ∈ Ω.

I If unfeasible, read extreme dual ray urω(y

k).I If optimal, read dual solution up

ω(yk) and solution value zω(y

k).

3. Use zω(yk) to compute UB

4. If stopping criteria are met (UB − LB ≤ ε; time; iterations), STOP.

5. Otherwise, compute cuts and add cuts to MP. Go to step 1.

19/29

Benders algorithm

Solution algorithm

1. Solve the relaxed MPI If unfeasible, STOP. P is also unfeasible.

I Otherwise, read solution y k and optimal value zk (LB).

2. Solve SPω(yk) ∀ ω ∈ Ω.

I If unfeasible, read extreme dual ray urω(y

k).I If optimal, read dual solution up

ω(yk) and solution value zω(y

k).

3. Use zω(yk) to compute UB

4. If stopping criteria are met (UB − LB ≤ ε; time; iterations), STOP.

5. Otherwise, compute cuts and add cuts to MP. Go to step 1.

19/29

Benders algorithm

Solution algorithm

1. Solve the relaxed MPI If unfeasible, STOP. P is also unfeasible.I Otherwise, read solution y k and optimal value zk (LB).

2. Solve SPω(yk) ∀ ω ∈ Ω.

I If unfeasible, read extreme dual ray urω(y

k).I If optimal, read dual solution up

ω(yk) and solution value zω(y

k).

3. Use zω(yk) to compute UB

4. If stopping criteria are met (UB − LB ≤ ε; time; iterations), STOP.

5. Otherwise, compute cuts and add cuts to MP. Go to step 1.

19/29

Benders algorithm

Solution algorithm

1. Solve the relaxed MPI If unfeasible, STOP. P is also unfeasible.I Otherwise, read solution y k and optimal value zk (LB).

2. Solve SPω(yk) ∀ ω ∈ Ω.

I If unfeasible, read extreme dual ray urω(y

k).I If optimal, read dual solution up

ω(yk) and solution value zω(y

k).

3. Use zω(yk) to compute UB

4. If stopping criteria are met (UB − LB ≤ ε; time; iterations), STOP.

5. Otherwise, compute cuts and add cuts to MP. Go to step 1.

19/29

Benders algorithm

Solution algorithm

1. Solve the relaxed MPI If unfeasible, STOP. P is also unfeasible.I Otherwise, read solution y k and optimal value zk (LB).

2. Solve SPω(yk) ∀ ω ∈ Ω.

I If unfeasible, read extreme dual ray urω(y

k).

I If optimal, read dual solution upω(y

k) and solution value zω(yk).

3. Use zω(yk) to compute UB

4. If stopping criteria are met (UB − LB ≤ ε; time; iterations), STOP.

5. Otherwise, compute cuts and add cuts to MP. Go to step 1.

19/29

Benders algorithm

Solution algorithm

1. Solve the relaxed MPI If unfeasible, STOP. P is also unfeasible.I Otherwise, read solution y k and optimal value zk (LB).

2. Solve SPω(yk) ∀ ω ∈ Ω.

I If unfeasible, read extreme dual ray urω(y

k).I If optimal, read dual solution up

ω(yk) and solution value zω(y

k).

3. Use zω(yk) to compute UB

4. If stopping criteria are met (UB − LB ≤ ε; time; iterations), STOP.

5. Otherwise, compute cuts and add cuts to MP. Go to step 1.

20/29

Toy problem exampleImplementation in C++ with Xpress BCL.

Code available at: http://osg.polymtl.ca

21/29

Parallel implementation

Server! Compute nodes!Read problem parameters!

Create master problem!Subproblem parameters!

Allocate scenarios to nodes!

Create subproblems!Compute lower bound!

Solve master problem!Master problem solution!

Solve subproblems!Subproblems objective values!

Check stopping criteria! Read subproblem dual solutions!Dual solutions!

Compute and add cuts!

Fix master problem solutions into subproblems!

22/29

Branch and cut approach

When the Master Problem is difficult to solve, the classical approach canperform poorly. A branch and cut approach can be used to solve a singletree.Whenever an integer solution is found in a node of the tree, thesubproblems are solved to check optimality and feasibility. Cuts areadded if necessary, if necessary.

23/29

Integer L-Shaped

If the subproblems are integer, dual variables are unavailable. Cuts canbe computed from bounds and objective values of the subproblems(Laporte and Louveaux, 1993)

wSP ≥ (Q(y)− LBsp)

( ∑(m,t)∈A

(ymt − 1)−∑

(m,t)/∈A

(ymt)

)− Q(y) (2)

24/29

Application example: Generator MaintenanceScheduling Problem

Howtouseadecomposi/onapproachtoefficentlyproducemaintenanceschedulesthatminimizethemaintenanceandopera/oncostsinhydropowersytems?

Adecomposi+onapproachforthehydropoweropera+onandmaintenanceschedulingproblem

JesusRodriguez,MiguelAnjos,CharlesAudetandPascalCôté

Hydropowerprovides97%oftherenewableelectricityinCanadaand46%intheUSA.

Maintenanceoutages

•  ReducesservicedisrupJons•  Preventscostlybreakdowns

Maintenanceofgenerators

/me

UncertainInflows

NonlinearpowerfuncJonsTemporalandspaJal

interdependencies

Equation

Pt = f (Qt , ht ,t)

Figure: Factors that impact the operation of hydropower systems

25/29

HGMSP: Problem definition

Maintenance scheduling problem!!

Minimize Maintenance cost!!

Subject to:!!- Maintenance constraints!

-  Completion of maintenance tasks!-  Time windows!-  Maximum number of outages!-  Maintenance capacity!

Required maintenance

tasks !Maintenance

Plan!

Generator Outages!

Operation costs?!

Hydropower maintenance scheduling!problem!

Minimize Maintenance and Operation cost!!

Subject to:!!-  Maintenance constraints!

-  Completion of maintenance tasks!-  Time windows!-  Maximum number of outages!-  Maintenance capacity!

-  Hydropower operation constraints!-  Mass balance!-  Power balance !-  Power production function!-  Bounds on reservoir levels and

discharges.!

Maintenance & Operation Plan!

Required maintenance tasks !!

Hydropower operation parameters!

26/29

xω : hydropower operation variables of scenario ω,

y : binary variables representing the beginning of maintenanceactivities,

z : binary variables representing the number of active generators,

minxω,y

∑ω∈Ω

ϕωcᵀxω + f ᵀy

s.t.A1xω + C1z ≥ b1ω, ∀ω ∈ ΩA2xω ≥ b2ω, ∀ω ∈ Ω

B3y + C3z = b3

B4y ≥ b4

C5z ≥ b5

xω ≥ 0, ∀ω ∈ Ωy ∈ 0, 1

z ∈ 0, 1

27/29

fixing z splits the problem into operation subproblems for each scenarioω ∈ Ω,

Qw (z) = minxω

cᵀxω

s.t.A1xω ≥ b1ω − C1z ⊥ λω1A2xω ≥ b2ω ⊥ λω2

xω ≥ 0

and a master problem,

minwSP ,y ,z

wSP+ f ᵀy

s.t.B3y + C3z = b3

B4y ≥ b4

C5z ≥ b5

y ∈ 0, 1z ∈ 0, 1

wSP− f ᵀy +∑ω∈Ω

ϕωλωᵀ1p [C1z ] ≥

∑ω∈Ω

ϕω(bᵀ1ωλω1p + bᵀ2ωλ

ω2p)

∀p ∈ P

28/29

Conclusions

I A careful analysis of the problem structure is necessary for a goodchoice of the solution strategy.

I Explore alternative formulations. The stronger the formulation, thebetter the performance of the decomposition method.

I The intuitive partitioning of the problem is not necessarily the mosteffective one.

I Straightforward implementation of decomposition methods can bedisappointing. Customization and acceleration strategies arenecessary.

I If subproblems are linear, you can benefit from open source solvers.

29/29

Slides and code of the example will be available on the website ofOptimization for Smart Grids (OSG)

http://osg.polymtl.ca

Thank you!

jesus.rodriguez@gerad.ca