approximating pareto curves using semidefinite...

Pareto Curves Parametric POP Outer Approximations of f (S) Perspectives

Approximating Pareto Curves using SemidefiniteRelaxations

Victor MAGRON

Postdoc LAAS-CNRS(Joint work with Didier Henrion and Jean-Bernard Lasserre)

Applications of Real Algebraic Geometry2014 March 1

Victor MAGRON (Arag 2014) Approximating Pareto Curves using SDP 1 / 37


Multiobjective Polynomial Optimization

Optimization Problems with several criteria in engineering,economics, applied mathematics.Design of a beam of length l, heigth x1 and width x2:

1 light construction: minimize the volume lx1x22 cheap construction: minimize the sectional area π/4(x2

1 + x22)

3 under stress and nonnegativity constraints

x1

x2




Let f1, f2 ∈ Rd[x] two conflicting criteria

Let S := {x ∈ Rn : g1(x) > 0, . . . , gm(x) > 0} a semialgebraic set

(P){

minx∈S

(f1(x) f2(x))>}

Assumption

The image space R2 is partially ordered in a natural way (R2+ is the

ordering cone).




DefinitionLet the previous assumption be satisfied. A point x ∈ S is called aweakly Edgeworth-Pareto (EP) optimal point of Problem P, when there isno x ∈ S such that fj(x) < fj(x), j = 1, 2.

f1(x) := x1 ,

f2(x) := x2 ,

S := {x ∈ R2 : 0 6 x1 6 1, 0 6 x2 6 1} . y1

y2

10

f (S)1



Some Examples: f (S) + R2+ is convex

g1 := −x21 + x2 ,

g2 := −x1 − 2x2 + 3 ,

S := {x ∈ R2 : g1 > 0, g2 > 0} .

−1.5 −1 −0.5 0 0.5 10

0.5

1

1.5

2

2.5

x1

x2

f1 := −x1 ,

f2 := x1 + x22 .

−1 −0.5 0 0.5 1 1.5−1

0

1

2

3

4

y1

y2



Some Examples: f (S) + R2+ is not convex

g1 := −(x1 − 2)3/2− x2 + 2.5 ,

g2 := −x1 − x2 + 8(−x1 + x2 + 0.65)2 + 3.85 ,

S := {x ∈ R2 : g1 > 0, g2 > 0} .

0 0.2 0.4 0.6 0.80

0.2

0.4

0.6

0.8

1

x1

x2

f1 := (x1 + x2 − 7.5)2/4 + (−x1 + x2 + 3)2 ,

f2 := (x1 − 1)2/4 + (x2 − 4)2/4 .

0 10 20 30 40 50 600

1

2

3

4

5

6

y1

y2



Scalarization Techniques

Common workaround by reducing P to a scalar POP :

(Ppλ)

{minx∈S

f λ(x) := (λ|f1(x)− µ1|p + (1− λ)|f2(x)− µ2|p)1p

},

with the weight λ ∈ [0, 1] and the goals µ1, µ2 ∈ R.

Possible choice: µj < minx∈S

fj(x), j = 1, 2.




When p = 1, weighted sum formulation P1λ:

f λ(x) := λf1(x) + (1− λ)f2(x)

Theorem ([Borwein 77], [Arrow-Barankin-Blackwell 53])

Assume that f (S) +R2+ is convex. A point x ∈ S is a weakly EP optimal

point of Problem P⇐⇒ ∃λ such that x is a solution of Problem P1λ.

−1 −0.5 0 0.5 1 1.5−1

0

1

2

3

4

y1

y2

−1 −0.5 0 0.5 1−0.5

0

0.5

1

1.5

2

y1

y2




When p = 1, weighted sum formulation P1λ:

f λ(x) := λf1(x) + (1− λ)f2(x)

Theorem ([Borwein 77], [Arrow-Barankin-Blackwell 53])

Assume that f (S) +R2+ is convex. A point x ∈ S is a weakly EP optimal

point of Problem P⇐⇒ ∃λ such that x is a solution of Problem P1λ.

0 10 20 30 40 50 600

1

2

3

4

5

6

y1

y2

6 8 10 12 14 16 18 200

0.5

1

1.5

2

2.5

y1

y2




When p = ∞, weigthed Chebyshev approximation P∞λ

f λ(x) := max{λ(f1(x)− µ1), (1− λ)(f2(x)− µ2)}

Theorem ([Jahn 87], [Bowman 76], [Steuer-Choo 83])

Suppose that ∀x ∈ S, µj < fj(x), j = 1, 2. A point x ∈ S is a weaklyEP optimal point of Problem P⇐⇒ ∃λ such that x is a solution of Prob-lem P∞

λ .

0 10 20 30 40 50 600

1

2

3

4

5

6

y1

y2

6 8 10 12 14 16 18 200

0.5

1

1.5

2

2.5

y1

y2



Questions

Is it mandatory to use discretization schemes?

Can we approximate the Pareto curve in a relatively strong sense?



Contributions

Yes!We provide two numerical schemes that avoid computing finitelymany points.

1 The first approximates the Pareto curve (f ∗1 (λ), f ∗2 (λ)), λ ∈ [0, 1],with polynomials that minimize the L2-norm∫ 1

0(f ∗j (λ)− h(λ))2dλ, j = 1, 2.

2 The second provides certified underestimators of the Pareto curveby computing a hierarchy of outer approximations of f (S).



Outline

1 Parametric POP

2 Outer Approximations of f (S)

3 Perspectives



Preliminaries

Parametric POP (P1λ) : f ∗(λ) := min

x∈Sf λ(x)

Let S := [0, 1]× S

LetM(S) the set of probability measures supported on S

(P)

ρ := min

µ∈M(S)

∫S

f λ(x)dµ(λ, x)

s.t.∫

Sλkdµ(λ, x) = 1/(1 + k), k ∈N .



Preliminaries

Lemma (Corollary of [1, Theorem 2.2])

Problem (P) has an optimal solution µ∗ ∈ M(S). Then,

ρ =∫

Sf λ(x)dµ∗ =

∫ 1

0f ∗(λ)dλ .

Moreover, suppose that (P) has a unique global minimizer x∗(λ) ∈ Sand let f ∗j (λ) := fj(x∗(λ)), j = 1, 2. Then,

ρ =∫ 1

0[λf ∗1 (λ) + (1− λ)f ∗2 (λ)]dλ .

1J.B. Lasserre. A “joint + marginal” approach to parametric polynomialoptimization (2010)



A hierarchy of semidefinite relaxations

Let g ∈ R[λ, x] with g(λ, x) := ∑k,α

gkαλkxα.

Consider the real sequence z = (zkα), (k, α) ∈Nn+1d

The entries (i, α), (j, β) ∈Nn+1d of the localizing matrix Md(g z)

are:Md(g z)((i, α), (j, β)) := ∑

r,γgrγz(i+j+r)(α+β+γ)

Consider the linear functional Lz(g) := ∑k,α

gkαzkα



A hierarchy of semidefinite relaxations

Let vl := deg gl, l = 1, . . . , m.

Let kmax := max{d, v1, . . . , vm}.Consider the semidefinite relaxations of (P):

(Ps)

min

zLz(f λ)

s.t. Ms−vl(gl z) < 0, l = 0, . . . , m ,Lz(λ

k) = 1/(1 + k), k = 0, . . . , 2s− kmax .



Polynomial underestimators of f ∗(λ)

The dual SDP of (Ps) reads:

(Ds)

minp,(σl)

∑k

pk/(1 + k)

s.t. f λ(x)− p(λ) =m

∑l=0

σl(λ, x)gl(x)

p ∈ R2s−kmax [λ], σl ∈ Σ[λ, x], l = 0, . . . , m ,deg(σlgl) 6 2s− kmax, l = 0, . . . , m .

The hierarchy (Ds) provides a sequence (ps) of polynomialunderestimators of f ∗(λ).

lims→∞

∫ 1

0(f ∗(λ)− ps(λ))dλ = 0




On the convex example:

−1 −0.5 0 0.5 1 1.5−1

0

1

2

3

4

y1

y2

0 0.2 0.4 0.6 0.8 1−1.2

−1

−0.8

−0.6

−0.4

−0.2

0

0.2

λ

p2p4f∗(λ)




On the non-convex example:

0 10 20 30 40 50 600

1

2

3

4

5

6

y1

y2

0 0.2 0.4 0.6 0.8 1−6

−4

−2

0

2

4

6

8

λ

p

2

p4

f*



An inverse problem from generalized moments

Lemma (Corollary of [1, Theorem 3.3])

Assume that for a.a. λ ∈ [0, 1], Problem (P) has a unique global opti-mizer x∗(λ) and let zs be an optimal solution of (Ps). Then,

lims→∞

zskα =

∫ 1

0λk(x∗(λ))αdλ, k ∈N .

In particular,

mkj := lim

s→∞ ∑α

fjαzskα =

∫ 1

0λkf ∗j (λ)dλ, j = 1, 2, k ∈N .

1J.B. Lasserre. A “joint + marginal” approach to parametric polynomialoptimization (2010)




Let 2s′ := 2s− kmax. One can compute:

Approximation of the vector mj := (mkj )

Approximations of f ∗j (λ), j = 1, 2, by solving:

minh∈R2s′ [λ]

{∫ 1

0(f ∗j (λ)− h(λ))2dλ

}, j = 1, 2 .




Theorem

The Problem minh∈R2s′ [λ]

{∫ 1

0(f ∗j (λ)− h(λ))2dλ

}has an optimal solution

psj ∈ R2s′ [λ], whose vector of coefficients is psj = H−1s mj, j = 1, 2,

where Hs ∈ S2s′+1 is the Hankel matrix, whose entries are defined by:

Hs(a, b) := 1/(1 + a + b), a, b = 0, . . . , 2s′, .




Proof.

∫ 1

0(f ∗j (λ)− h(λ))2dλ =

∫ 1

0f ∗j (λ)

2dλ︸︷︷︸A

−2∫ 1

0f ∗j (λ)h(λ)dλ︸︷︷︸

B

+∫ 1

0h(λ)2dλ︸︷︷︸

C

,

B = h′mj, C = h′Hsh ,

thus the problem can be reformulated as:

minh{h′Hsh− 2h′mj}, j = 1, 2 .



With the weighted sum approach

On the convex example:

−1 −0.5 0 0.5 1 1.5−1

0

1

2

3

4

y1

y2

−1 −0.5 0 0.5 1 1.5 2−0.5

0

0.5

1

1.5

2

2.5

3

y1

y2

(p

4, 1, p

4, 2)

(p6, 1

, p6, 2

)

(p8, 1

, p8, 2

)

(f1* , f

2*)



With the weighted sum approach


0 10 20 30 40 50 600

1

2

3

4

5

6

y1

y2

6 8 10 12 14 16 180

0.5

1

1.5

2

2.5

y1

y2

(p

4, 1, p

4, 2)

(p6, 1

, p6, 2

)

(p8, 1

, p8, 2

)

(f1* , f

2*)



With the weigthed Chebyshev approximation


0 10 20 30 40 50 600

1

2

3

4

5

6

y1

y2

6 8 10 12 14 16 180

0.5

1

1.5

2

2.5

y1

y2

(p

3, 1, p

3, 2)

(p4, 1

, p4, 2

)

(p5, 1

, p5, 2

)

(f1* , f

2*)



Approximation of sets defined with “∃”

Let B ⊂ R2 be the unit ball and assume that f (S) ⊂ B.

Another point of view:

f (S) = {y ∈ B : ∃x ∈ S s.t. h(x, y) 6 0} ,

withh(x, y) := (y1 − f1(x))2 + (y2 − f2(x))2 .

Approximate f (S) as closely as desired by a sequence of sets ofthe form :

Θk := {y ∈ B : Jk(y) 6 0} ,

for some polynomials Jk ∈ R2k[y]2d.



Approximation of sets defined with “∃”

Let g0 := 1 and Qk(S) be the k-truncated quadratic modulegenerated by g0, . . . , gm:

Qk(S) ={ m

∑l=0

σl(x, y)gl(x), with σl ∈ Σk−vl [x, y]}

Define H(y) := minx∈S

h(x, y)

Hierarchy of Semidefinite programs:

ρk := minJ∈R2k[y],σl

{∫B(H− J)dy : h− J ∈ Qk(S)

}.

Yet another SOS program with an optimal solution Jk ∈ R2k[y]!



A hierarchy of outer approximations of f (S)

From the definition of Jk, the sublevel sets

Θk := {y ∈ B : Jk(y) 6 0} ⊃ f (S), k > kmax ,

provide a sequence of certified outer approximations of f (S).

It comes from the following:

∀(x, y) ∈ S× B, J(y) 6 h(x, y)⇐⇒ ∀y, J(y) 6 H(y) .



Strong convergence property

Theorem [Lasserre]1 The sequence of underestimators (Jk)k>kmax converges to H w.r.t

the L1(B)-norm:

limk→∞

∫B|H− Jk|dy = 0 .

2 When the contour set {y ∈ B : H(y) = 0} has a null Lebesguemeasure, then

limk→∞

V(Θk\f (S)) = 0 .



Back to the non-convex example

0 10 20 30 40 50 600

1

2

3

4

5

6

y1

y2

f (S)

y1

y2

8 10 12 14 16 18 20−1

0

1

2

3

4

Θ2




0 10 20 30 40 50 600

1

2

3

4

5

6

y1

y2

f (S)

y1

y2

8 10 12 14 16 18 20−1

0

1

2

3

4

Θ3




0 10 20 30 40 50 600

1

2

3

4

5

6

y1

y2

f (S)

y1

y2

8 10 12 14 16 18 20−1

0

1

2

3

4

Θ4




0 10 20 30 40 50 600

1

2

3

4

5

6

y1

y2

f (S)

y1

y2

8 10 12 14 16 18 20−1

0

1

2

3

4

Θ5




0 10 20 30 40 50 600

1

2

3

4

5

6

y1

y2

f (S)

y1

y2

8 10 12 14 16 18 20−1

0

1

2

3

4

Θ6



Branch and Bound: Zoom on the left

0 10 20 30 40 50 600

1

2

3

4

5

6

y1

y2

f (S)

y1

y2

7.4 7.6 7.8 8 8.2 8.4

0

1

2

3

4

Θ2




0 10 20 30 40 50 600

1

2

3

4

5

6

y1

y2

f (S)

y1

y2

7.4 7.6 7.8 8 8.2 8.4

0

1

2

3

4

Θ3




0 10 20 30 40 50 600

1

2

3

4

5

6

y1

y2

f (S)

y1

y2

7.4 7.6 7.8 8 8.2 8.4

0

1

2

3

4

Θ4




0 10 20 30 40 50 600

1

2

3

4

5

6

y1

y2

f (S)

y1

y2

7.4 7.6 7.8 8 8.2 8.4

0

1

2

3

4

Θ5




0 10 20 30 40 50 600

1

2

3

4

5

6

y1

y2

f (S)

y1

y2

7.4 7.6 7.8 8 8.2 8.4

0

1

2

3

4

Θ6



Transcendental conflicting criteria

Now, consider the following Problem:

(P){

minx∈S

(f1(x) f2(x))> .}

with transcendental criteria f1, f2.

Generalization of the single criterion problem minx∈S

f (x)

Hard to combine SOS hierarchies with Taylor/Chebyshevapproximations




Definition: Semiconvex functionLet γ > 0. A function φ : Rn → R is said to be γ-semiconvex if thefunction x 7→ φ(x) +

γ

2‖x‖2

2 is convex.

Proposition (by Legendre-Fenchel duality)

The set of functions f : Rn → R which can be written as the max-plus linear combination f = sup

w∈B(a(w) + w) for some function a : B →

R ∪ {−∞} is precisely the set of lower semicontinuous γ-semiconvexfunctions.




a

y

par−a1

arctan

m Ma1




a

y

par−a1

par−a2

arctan

m Ma1a2




a

y

par−a1

par−a2

par−a3

arctan

m Ma1a2 a3



Sublevel sets of semialgebraic underestimators

The sublevel sets

Θk := {y ∈ B : Jk(y) 6 0} ⊃ f (S), k > kmax ,

provide a sequence of certified outer approximations of f (S).

To avoid Branch and bound iterations (“Zooms”), one couldunderestimate H with a rational function

J := F/1 + σ ,

with F ∈ R2k[y], σ ∈ Σk0 [y].



Thanks for your attention!

Kiitos Alexander, Cordian!!!


approximating pareto curves using semidefinite...

Documents