robust combinatorial optimization with variable uncertainty · robust combinatorial optimization...

Robust combinatorial optimization with variableuncertainty

Michael Poss

Heudiasyc UMR CNRS 7253, Universite de Technologie de Compiegne

17th Aussois Combinatorial Optimization Workshop

M. Poss (Heudiasyc) Variable uncertainty Aussois 1 / 31

Outline

1 Robust optimization

2 Variable budgeted uncertainty

3 Cost uncertainty

Outline

3 Cost uncertainty

Combinatorial optimization under uncertainty

minn∑

s.t.n∑

aijxj ≤ bi , i = 1, . . . ,m

x ∈ {0, 1}n

Suppose that the parameters (a, b, c) are uncertain:

They vary over time

They must be predicted from historical data

They cannot be measured with enough accuracy

Let’s do something clever (and useful)!

How much do we know?

Stochastic programming︷︸︸︷A lot ⇔

Robust programming︷︸︸︷A little

Robust pr. Uncertain parameters are merely assumed to belong to anuncertainty set U ⇒ one wishes to optimize some worst-caseobjective over the uncertainty set

Stochastic pr. Uncertain parameters are precisely described by probabilitydistributions ⇒ one wishes to optimize some expectation,variance, Value-at-risk, . . .

Intermediary models exist: distributionally robust optimization, ambiguouschance-constrained

How much do we know?

Stochastic programming︷︸︸︷A lot ⇔

Robust programming︷︸︸︷A little

Robust pr. Uncertain parameters are merely assumed to belong to anuncertainty set U ⇒ one wishes to optimize some worst-caseobjective over the uncertainty set

Stochastic pr. Uncertain parameters are precisely described by probabilitydistributions ⇒ one wishes to optimize some expectation,variance, Value-at-risk, . . .

Intermediary models exist: distributionally robust optimization, ambiguouschance-constrained

When do we take decisions?

Now All decisions must be taken before the uncertainty is knownwith precision ⇒ probability constraints, (static) robustoptimization

Delayed Some decisions may be delayed until the uncertainty isrevealed ⇒ multi-stage stochastic programming, adjustablerobust optimization

Robust combinatorial optimization

minn∑

s.t.n∑

aijxj ≤ bi , i = 1, . . . ,m, ∀ai ∈ Ui

x ∈ {0, 1}n,

The linear relaxation of this problem is tractable if Ui is defined by conicconstraints:

Ui = {ai ∈ Rn : u ai − v ∈ K}.

In particular, polyhedrons and polytopes are nice (K = Rn+).

Feasibility set

∑aixi ≤ b, ∀a ∈ U ⇔

∑j cjαj ≤ b∑j ujiαj ≥ xj

αj ≥ 0

The feasibility set of the constraint is a polyhedron (thus, convex) !

A (very) popular polyhedral uncertainty set is (Bertsimas and Sim, 2004):

UΓ :={

a ∈ Rn : ai = ai + δi ai ,−1 ≤ δi ≤ 1,∑|δi | ≤ Γ

Main reasons for popularity:

Nice computational properties for MIP and combinatorial problems.

Intuitive interpretation.

Probabilistic interpretation.

Feasibility set

∑aixi ≤ b, ∀a ∈ U ⇔

∑j cjαj ≤ b∑j ujiαj ≥ xj

αj ≥ 0

The feasibility set of the constraint is a polyhedron (thus, convex) !A (very) popular polyhedral uncertainty set is (Bertsimas and Sim, 2004):

UΓ :={

a ∈ Rn : ai = ai + δi ai ,−1 ≤ δi ≤ 1,∑|δi | ≤ Γ

Main reasons for popularity:

Nice computational properties for MIP and combinatorial problems.

Intuitive interpretation.

Probabilistic interpretation.

Consider a vehicle routing problem with uncertain travel times t and timelimit T . For simplicity, we suppose t ∈ UΓ for Γ = 5.

Consider two robust-feasible routes x1 and x2 : ‖x1‖1 = 3 and ‖x2‖1 = 10.

Because x1 and x2 are robust-feasible:∑i :x1

t i ≤ T , ∀a ∈ UΓ, and∑i :x2

t i ≤ T , ∀a ∈ UΓ

which becomes∑i :x1

t i ≤ T , and∑i :x2

t i ≤ T , ∀a ∈ UΓ

For any probability distribution for t:

∑i :x1

t i > T

If x2 is not robust-feasible for Γ = 10, there exists probability distributions:

∑i :x2

t i > T

t i ≤ T , ∀a ∈ UΓ, and∑i :x2

t i ≤ T , ∀a ∈ UΓ

∑i :x1

t i > T

∑i :x2

t i > T

t i ≤ T , ∀a ∈ UΓ, and∑i :x2

t i ≤ T , ∀a ∈ UΓ

∑i :x1

t i > T

∑i :x2

t i > T

t i ≤ T , ∀a ∈ UΓ, and∑i :x2

t i ≤ T , ∀a ∈ UΓ

∑i :x1

t i > T

∑i :x2

t i > T

t i ≤ T , ∀a ∈ UΓ, and∑i :x2

t i ≤ T , ∀a ∈ UΓ

∑i :x1

t i > T

∑i :x2

t i > T

t i ≤ T , ∀a ∈ UΓ, and∑i :x2

t i ≤ T , ∀a ∈ UΓ

∑i :x1

t i > T

∑i :x2

t i > T

Outline

3 Cost uncertainty

Robust optimization and probabilistic constraint

Let ai be random variables and ε > 0. The chance constraint

aixi > b)≤ ε (1)

leads to very difficult optimization problems in general.

In some situations, we know that (1) can be approximated by∑aixi ≤ b ∀a ∈ U (2)

for a properly chosen U.These approximations are conservative: any x feasible for (2) is feasible for(1).We must balance conservatism and protection cost ⇒ devise goodprotection sets U.

Let ai be random variables and ε > 0. The chance constraint

aixi > b)≤ ε (1)

leads to very difficult optimization problems in general.In some situations, we know that (1) can be approximated by∑

aixi ≤ b ∀a ∈ U (2)

for a properly chosen U.These approximations are conservative: any x feasible for (2) is feasible for(1).We must balance conservatism and protection cost ⇒ devise goodprotection sets U.

What about UΓ ?

Let ai be random variables independently and symmetrically distributed in[ai − ai , ai + ai ].Bertsimas and Sim (2004) prove that if a vector x satisfies the robustconstraint ∑

aixi ≤ b ∀a ∈ UΓ,

then it satisfies also the probabilistic constraint

aixi > b)≤ exp

(−Γ2

What about UΓ ?Let ai be random variables independently and symmetrically distributed in[ai − ai , ai + ai ].Bertsimas and Sim (2004) prove that if a vector x satisfies the robustconstraint ∑

aixi ≤ b ∀a ∈ UΓ,

then it satisfies also the probabilistic constraint

aixi > b)≤ exp

(−Γ2

Something is wrong ...

aixi > b)≤ exp

(−Γ2

we see that choosing Γ = (−2 ln(ε))1/2 n1/2 yields

aixi > b)≤ ε.

For many problems, ‖x‖1 < n1/2 for optimal (or feasible) vectors x(network design, assignment, ...)⇒ Γ > n1/2 already for ε = 0.5⇒ for these problems, protecting with probability 0.5 yields protectionwith probability 0!⇒ overprotection !

Something is wrong ...

aixi > b)≤ exp

(−Γ2

we see that choosing Γ = (−2 ln(ε))1/2 n1/2 yields

aixi > b)≤ ε.

For many problems, ‖x‖1 < n1/2 for optimal (or feasible) vectors x(network design, assignment, ...)⇒ Γ > n1/2 already for ε = 0.5⇒ for these problems, protecting with probability 0.5 yields protectionwith probability 0!⇒ overprotection !

Multifunctions

It is easy to see that the bound from Bertsimas and Sim can be adapted to

aixi > b)≤ exp

(− Γ2

2‖x‖1

Γ can be reduced when x is small

Let’s use multifunctions !

Defineαε(x) = (−2 ln(ε)‖x‖1)1/2 .

Consider x∗ be given. If∑aix∗i ≤ b ∀a ∈ Uαε(x∗) ,

aix∗i > b

)≤ exp

(−αε(x∗)2

2‖x∗‖1

)= ε.

Multifunctions

aixi > b)≤ exp

(− Γ2

2‖x‖1

Defineαε(x) = (−2 ln(ε)‖x‖1)1/2 .

aix∗i > b

)≤ exp

(−αε(x∗)2

2‖x∗‖1

)= ε.

Multifunctions

aixi > b)≤ exp

(− Γ2

2‖x‖1

Defineαε(x) = (−2 ln(ε)‖x‖1)1/2 .

aix∗i > b

)≤ exp

(−αε(x∗)2

2‖x∗‖1

)= ε.

New robust model

Let γ : {0, 1}n → R+ be a non-negative function.

Uγ(x) :={

a ∈ Rn : ai = ai + δi ai ,−1 ≤ δi ≤ 1,∑|δi | ≤ γ(x)

We have shown that the new model∑aixi ≤ b ∀a ∈ Uαε(x),

should be considered instead of the classical model∑aixi ≤ b ∀a ∈ UΓ.

New robust model

Let γ : {0, 1}n → R+ be a non-negative function.

Uγ(x) :={

a ∈ Rn : ai = ai + δi ai ,−1 ≤ δi ≤ 1,∑|δi | ≤ γ(x)

We have shown that the new model∑aixi ≤ b ∀a ∈ Uαε(x),

should be considered instead of the classical model∑aixi ≤ b ∀a ∈ UΓ.

Better bound

The previous bound is bad. Bertsimas and Sim propose a better bound:

aix∗i > b

)≤ B(n, Γ) =

(1− µ)

n∑l=bνc+1

where ν = (Γ + n)/2, µ = ν − bνc.

We can make this bound dependent on x by considering B(‖x‖1, Γ).

βε(x) is the solution of the equation

B(‖x∗‖1, Γ)− ε = 0 (3)

in variable Γ.

We solve (3) numerically.

Tractability. Example: Knapsack problem

maxn∑

s.t.n∑

aixi ≤ b, a ∈ Uγ

x ∈ {0, 1}n,which can be rewritten as

maxn∑

s.t.n∑

aixi + max0 ≤ δi ≤ 1∑δi ≤ γ(x)

n∑i=1

δi aixi ≤ b,

x ∈ {0, 1}n,M. Poss (Heudiasyc) Variable uncertainty Aussois 17 / 31

Knapsack problem

Using the dualization approach:

maxn∑

s.t.n∑

aixi + zγ(x) +n∑

pi ≤ b,

z + pi ≥ aixi , i = 1, . . . , n

z , p ≥ 0,

x ∈ {0, 1}n.

Non-convex reformulation.

x binary may help.

Dualization

Theorem

Consider robust constraint

aT x ≤ b, ∀a ∈ Uγ(x),x ∈ {0, 1}n, (4)

and suppose that γ = γ0 +∑γixi is an affine function of x, non-negative

for x ∈ {0, 1}n. Then, (4) is equivalent to

n∑i=1

aixi + γ0z +n∑

γiwi +n∑

pi ≤ b

z + pi ≥ aixi , i = 1, . . . , n,wi − z ≥ −maxj(aj)(1− xi ), i = 1, . . . , n,

p,w , z ≥ 0, x ∈ {0, 1}n.

Non-affine functions γ

0 100 200 300 400 500 600 700 800 900 1000

β0.01

0 100 200 300 400 500 600 700 800 900 1000

β0.01

min(γ1, γ2)

Computational results

Objective

1 Is there a benefit in using Uβ instead of UΓ ?

2 Computational “complexity” of solving the robust counterparts.

ModelsWe compare the following at ε = 0.01:

UΓ The classical robust model with budget uncertainty.

Uγ1Our new model with variable budget uncertainty: γ1

overapproximates β.

Uγ1γ2Our new model with variable budget uncertainty:min(γ1, γ2) over-approximates β.

Computational results

Objective

1 Is there a benefit in using Uβ instead of UΓ ?

2 Computational “complexity” of solving the robust counterparts.

ModelsWe compare the following at ε = 0.01:

UΓ The classical robust model with budget uncertainty.

Uγ1Our new model with variable budget uncertainty: γ1

overapproximates β.

Uγ1γ2Our new model with variable budget uncertainty:min(γ1, γ2) over-approximates β.

The price of robustness at ε = 0.01

Instances from Bertsimas and Sim (2004)

100 200 300 400 500 600 700 800 900 1000

Number of items n

Uγ1γ2

+ ++ + + +

••

• • • • • •

•UΓ

��

Computational complexity

model Uγ1 Uγ1γ2 Uγ1γ2γ3

time model/time UΓ 1.7 3.4 6.1gap model/gap UΓ 0.87 0.98 1.1

Fixing M to maxj(aj) affects the LP relaxation.

If M = 1000, gap Uγ1/gap UΓ → 3.9 !

Outline

3 Cost uncertainty

Cost uncertainty

Suppose that only cost coefficient are uncertain

min maxc∈U

n∑i=1

s.t.n∑

aijxj ≤ bi , i = 1, . . . ,m

x ∈ {0, 1}n,

which can be rewritten

COΓ ≡ minx∈X

maxc∈UΓ

cT x .

The previous probabilistic approximation leads to a relation between COΓ

andminx∈X

VaRε cT x .

Value-at-risk

Definition: VaRε(cT x) = inf{t|P(cT x ≤ t) ≥ 1− ε}.We see easily that

COΓ provides an upper bound of the optimization of VaR

The upper bound is very bad for small cardinality vectors

Model COγ overcomes this flaw

COγ ≡ minx∈X

maxc∈Uγ

cT x .

Value-at-risk

COγ ≡ minx∈X

maxc∈Uγ

cT x .

Value-at-risk

COγ ≡ minx∈X

maxc∈Uγ

cT x .

Shortest path problem

0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.1

Value of ε

NE1AL1

MN1IA1

Complexity of the resulting problems

Theorem

When γ is an affine function, COγ can be solved by solving n + 1 problemsCO and taking the cheapest optimal solution.

Theorem

When γ is a non-decreasing function of ‖x‖1, COγ can be solved bysolving n cardinality constrained problems COΓ and taking the cheapestoptimal solution.

Complexity of the resulting problems

Theorem

When γ is an affine function, COγ can be solved by solving n + 1 problemsCO and taking the cheapest optimal solution.

Theorem

When γ is a non-decreasing function of ‖x‖1, COγ can be solved bysolving n cardinality constrained problems COΓ and taking the cheapestoptimal solution.

Dynamic Programming

We use the notation Γ′ = min(n, maxk=0,...,n

γ(k)).

Theorem

Consider a combinatorial optimization problem that can be solved in O(τ)by using dynamic programming. If γ(k) ∈ Z for each k = 0, . . . , n, thenCOγ can be solved in O(nΓ′τ). Otherwise, COγ can be solved inO(n2Γ′τ).

Theorem

Consider a combinatorial optimization problem that can be solved in O(τ)by using dynamic programming. Then, COΓ can be solved in O(Γτ).

If Γ ∼ n1/2, we get O(n1/2τ), improving over the O(nτ) from Bertsimasand Sim.

Dynamic Programming

We use the notation Γ′ = min(n, maxk=0,...,n

γ(k)).

Theorem

Consider a combinatorial optimization problem that can be solved in O(τ)by using dynamic programming. If γ(k) ∈ Z for each k = 0, . . . , n, thenCOγ can be solved in O(nΓ′τ). Otherwise, COγ can be solved inO(n2Γ′τ).

Theorem

Consider a combinatorial optimization problem that can be solved in O(τ)by using dynamic programming. Then, COΓ can be solved in O(Γτ).

If Γ ∼ n1/2, we get O(n1/2τ), improving over the O(nτ) from Bertsimasand Sim.

Concluding remarks

We introduce a new class of uncertainty models.

They correct the flaw of Bertsimas and Sim model.

The tractability of the new model is often comparable (or equal) tothe traditional model.

Remark: The model can be extended to non-combinatorial problems buttractability becomes an issue.

Thank’s for your attention

robust combinatorial optimization with variable uncertainty · robust combinatorial optimization...

Documents

robust decision-making with model uncertainty in aerospace

adaptive robust control under model uncertainty · adaptive...

managing uncertainty using robust optimization

uncertainty: sensitivity analysis & robust optimization ·...

network planning under demand uncertainty with robust...

robust facility location under demand location...

uncertainty handling for environment- robust …1...

robust winners and winner determination policies under...

robust airline scheduling under block time...

robust workforce planning: dealing with uncertainty

uncertainty assessment and robust model predictive control...

parametric uncertainty reduction in robust multivariable...

a survey on metaheuristics for stochastic combinatorial...

robust and efficient uncertainty quantification and ... ·...

quantitative robust uncertainty principles and optimally...

se 5102: uncertainty analysis, robust design, and

robust optimization under multi-band uncertainty part i...

robust visual localization with dynamic uncertainty...

approach for uncertainty propagation and robust design...

adaptive robust hedging under model uncertainty