convex optimization problems prof. daniel p. palomar
TRANSCRIPT
![Page 1: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/1.jpg)
Convex Optimization Problems
Prof. Daniel P. PalomarThe Hong Kong University of Science and Technology (HKUST)
MAFS5310 - Portfolio Optimization with RMSc in Financial Mathematics
Fall 2020-21, HKUST, Hong Kong
![Page 2: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/2.jpg)
Outline
1 Optimization Problems
2 Convex Optimization
3 Quasi-Convex Optimization
4 Classes of Convex Problems: LP, QP, SOCP, SDP
5 Multicriterion Optimization (Pareto Optimality)
6 Solvers
![Page 3: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/3.jpg)
Outline
1 Optimization Problems
2 Convex Optimization
3 Quasi-Convex Optimization
4 Classes of Convex Problems: LP, QP, SOCP, SDP
5 Multicriterion Optimization (Pareto Optimality)
6 Solvers
![Page 4: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/4.jpg)
Optimization Problem in Standard Form
General optimization problem in standard form:
minimizex
f0 (x)
subject to fi (x) ≤ 0 i = 1, . . . ,mhi (x) = 0 i = 1, . . . , p
wherex = (x1, . . . , xn) is the optimization variablef0 : Rn −→ R is the objective functionfi : Rn −→ R, i = 1, . . . ,m are inequality constraintfunctionshi : Rn −→ R, i = 1, . . . , p are equality constraintfunctions.
Goal: find an optimal solution x? that minimizes f0 while satisfying allthe constraints.
D. Palomar Convex Problems 4 / 48
![Page 5: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/5.jpg)
Optimization Problem in Standard Form
Feasibility:a point x ∈ dom f0 is feasible if it satisfies all the constraints andinfeasible otherwisea problem is feasible if it has at least one feasible point and infeasibleotherwise.
Optimal value:
p? = inf {f0 (x) | fi (x) ≤ 0, i = 1, . . . ,m, hi (x) = 0, i = 1, . . . , p}
p? =∞ if problem infeasible (no x satisfies the constraints)p? = −∞ if problem unbounded below.
Optimal solution: x? such that f (x?) = p? (and x? feasible).
D. Palomar Convex Problems 5 / 48
![Page 6: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/6.jpg)
Global and Local Optimality
A feasible x is optimal if f0 (x) = p?; Xopt is the set of optimal points.A feasible x is locally optimal if is optimal within a ball
Examples:f0 (x) = 1/x , dom f0 = R++: p? = 0, no optimal pointf0 (x) = x3 − 3x : p? = −∞, local optimum at x = 1.
D. Palomar Convex Problems 6 / 48
![Page 7: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/7.jpg)
Implicit Constraints
The standard form optimization problem has an explicit constraint:
x ∈ D =m⋂i=0
dom fi ∩p⋂
i=1
dom hi
D is the domain of the problemThe constraints fi (x) ≤ 0, hi (x) = 0 are the explicit constraintsA problem is unconstrained if it has no explicit constraintsExample:
minimizex
log(b − aT x
)is an unconstrained problem with implicit constraint b > aT x .
D. Palomar Convex Problems 7 / 48
![Page 8: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/8.jpg)
Feasibility Problem
Sometimes, we don’t really want to minimize any objective, just tofind a feasible point:
findx
x
subject to fi (x) ≤ 0 i = 1, . . . ,mhi (x) = 0 i = 1, . . . , p
This feasibility problem can be considered as a special case of ageneral problem:
minimizex
0
subject to fi (x) ≤ 0 i = 1, . . . ,mhi (x) = 0 i = 1, . . . , p
where p? = 0 if constraints are feasible and p? =∞ otherwise.
D. Palomar Convex Problems 8 / 48
![Page 9: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/9.jpg)
Outline
1 Optimization Problems
2 Convex Optimization
3 Quasi-Convex Optimization
4 Classes of Convex Problems: LP, QP, SOCP, SDP
5 Multicriterion Optimization (Pareto Optimality)
6 Solvers
![Page 10: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/10.jpg)
Convex Optimization Problem
Convex optimization problem in standard form:
minimizex
f0 (x)
subject to fi (x) ≤ 0 i = 1, . . . ,mAx = b
where f0, f1, . . . , fm are convex and equality constraints are affine.Local and global optima: any locally optimal point of a convexproblem is globally optimal.Most problems are not convex when formulated.Reformulating a problem in convex form is an art, there is nosystematic way.
D. Palomar Convex Problems 10 / 48
![Page 11: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/11.jpg)
Example
The following problem is nonconvex (why not?):
minimizex
x21 + x2
2
subject to x1/(1 + x2
2)≤ 0
(x1 + x2)2 = 0
The objective is convex.The equality constraint function is not affine; however, we can rewriteit as x1 = −x2 which is then a linear equality constraint.The inequality constraint function is not convex; however, we canrewrite it as x1 ≤ 0 which again is linear.We can rewrite it as
minimizex
x21 + x2
2
subject to x1 ≤ 0x1 = −x2
D. Palomar Convex Problems 11 / 48
![Page 12: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/12.jpg)
Equivalent Reformulations
Eliminating/introducing equality constraints:
minimizex
f0 (x)
subject to fi (x) ≤ 0 i = 1, . . . ,mAx = b
is equivalent to
minimizez
f0 (Fz + x0)
subject to fi (Fz + x0) ≤ 0 i = 1, . . . ,m
where F and x0 are such that Ax = b ⇐⇒ x = Fz + x0 for some z .
D. Palomar Convex Problems 12 / 48
![Page 13: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/13.jpg)
Equivalent Reformulations
Introducing slack variables for linear inequalities:
minimizex
f0 (x)
subject to aTi x ≤ bi i = 1, . . . ,m
is equivalent to
minimizex ,s
f0 (x)
subject to aTi x + si = bi i = 1, . . . ,msi ≥ 0
D. Palomar Convex Problems 13 / 48
![Page 14: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/14.jpg)
Equivalent Reformulations
Epigraph form: a standard form convex problem is equivalent to
minimizex ,t
t
subject to f0 (x)− t ≤ 0fi (x) ≤ 0 i = 1, . . . ,mAx = b
Minimizing over some variables:
minimizex ,y
f0 (x , y)
subject to fi (x) ≤ 0 i = 1, . . . ,m
is equivalent to
minimizex
f̃0 (x)
subject to fi (x) ≤ 0 i = 1, . . . ,m
where f̃0 (x) = infy f0 (x , y).D. Palomar Convex Problems 14 / 48
![Page 15: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/15.jpg)
Outline
1 Optimization Problems
2 Convex Optimization
3 Quasi-Convex Optimization
4 Classes of Convex Problems: LP, QP, SOCP, SDP
5 Multicriterion Optimization (Pareto Optimality)
6 Solvers
![Page 16: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/16.jpg)
Quasiconvex Optimization
Quasi-convex optimization problem in standard form:
minimizex
f0 (x)
subject to fi (x) ≤ 0 i = 1, . . . ,mAx = b
where f0 : Rn −→ R is quasiconvex and f1, . . . , fm are convex.Observe that it can have locally optimal points that are not (globally)optimal:
D. Palomar Convex Problems 16 / 48
![Page 17: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/17.jpg)
Quasiconvex Optimization
Convex representation of sublevel sets of a quasiconvex function f0:there exists a family of convex functions φt (x) for fixed t such that
f0 (x) ≤ t ⇐⇒ φt (x) ≤ 0.
Example:
f0 (x) =p (x)
q (x)
with p convex, q concave, and p (x) ≥ 0, q (x) > 0 on dom f0. Wecan choose:
φt (x) = p (x)− tq (x)
for t ≥ 0, φt (x) is convex in xp (x) /q (x) ≤ t if and only if φt (x) ≤ 0.
D. Palomar Convex Problems 17 / 48
![Page 18: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/18.jpg)
Quasiconvex Optimization
Solving a quasiconvex problem via convex feasibility problems: theidea is to solve the epigraph form of the problem with a sandwichtechnique in t:
for fixed t the epigraph form of the original problem reduces to afeasibility convex problem
φt (x) ≤ 0, fi (x) ≤ 0 ∀i , Ax ≤ b
if t is too small, the feasibility problem will be infeasibleif t is too large, the feasibility problem will be feasiblestart with upper and lower bounds on t (termed u and l , resp.) anduse a sandwitch technique (bisection method): at each iteration uset = (l + u) /2 and update the bounds according to thefeasibility/infeasibility of the problem.
D. Palomar Convex Problems 18 / 48
![Page 19: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/19.jpg)
Outline
1 Optimization Problems
2 Convex Optimization
3 Quasi-Convex Optimization
4 Classes of Convex Problems: LP, QP, SOCP, SDP
5 Multicriterion Optimization (Pareto Optimality)
6 Solvers
![Page 20: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/20.jpg)
Linear Programming (LP)
LP:minimize
xcT x + d
subject to Gx ≤ hAx = b
Convex problem: affine objective and constraint functions.Feasible set is a polyhedron:
D. Palomar Convex Problems 20 / 48
![Page 21: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/21.jpg)
`1- and `∞- Norm Problems as LPs
`∞-norm minimization:
minimizex
‖x‖∞subject to Gx ≤ h
Ax = b
is equivalent to the LP
minimizet,x
t
subject to −t1 ≤ x ≤ t1Gx ≤ hAx = b.
D. Palomar Convex Problems 21 / 48
![Page 22: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/22.jpg)
`1- and `∞- Norm Problems as LPs
`1-norm minimization:
minimizex
‖x‖1subject to Gx ≤ h
Ax = b
is equivalent to the LP
minimizet,x
∑i ti
subject to −t ≤ x ≤ tGx ≤ hAx = b.
D. Palomar Convex Problems 22 / 48
![Page 23: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/23.jpg)
Linear-Fractional Programming
Linear-fractional programming:
minimizex
(cT x + d
)/(eT x + f
)subject to Gx ≤ h
Ax = b
with dom f0 ={x | eT x + f > 0
}.
It is a quasiconvex optimization problem (solved by bisection).Interestingly, the following LP is equivalent:
minimizey ,z
cT y + dz
subject to Gy ≤ hzAy = bzeT y + fz = 1z ≥ 0
D. Palomar Convex Problems 23 / 48
![Page 24: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/24.jpg)
Quadratic Programming (QP)
Quadratic programming:
minimizex
(1/2) xTPx + qT x + r
subject to Gx ≤ hAx = b
Convex problem (assuming P ∈ Sn � 0): convex quadratic objectiveand affine constraint functions.Minimization of a convex quadratic function over a polyhedron:
D. Palomar Convex Problems 24 / 48
![Page 25: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/25.jpg)
Quadratically Constrained QP (QCQP)
Quadratically constrained QP:
minimizex
(1/2) xTP0x + qT0 x + r0
subject to (1/2) xTPix + qTi x + ri ≤ 0 i = 1, . . . ,mAx = b
Convex problem (assuming Pi ∈ Sn � 0): convex quadratic objectiveand constraint functions.
D. Palomar Convex Problems 25 / 48
![Page 26: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/26.jpg)
Second-Order Cone Programming (SOCP)
Second-order cone programming:
minimizex
f T x
subject to ‖Aix + bi‖ ≤ cTi x + di i = 1, . . . ,mFx = g
Convex problem: linear objective and second-order cone constraintsFor Ai row vector, it reduces to an LP.For ci = 0, it reduces to a QCQP.More general than QCQP and LP.
D. Palomar Convex Problems 26 / 48
![Page 27: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/27.jpg)
Semidefinite Programming (SDP)
SDP:minimize
xcT x
subject to x1F1 + x2F2 + · · ·+ xnFn � GAx = b
Inequality constraint is called linear matrix inequality (LMI).Convex problem: linear objective and linear matrix inequality (LMI)constraints.Observe that multiple LMI constraints can always be written as asingle one.
D. Palomar Convex Problems 27 / 48
![Page 28: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/28.jpg)
Semidefinite Programming (SDP)
LP and equivalent SDP:
minimizex
cT x
subject to Ax ≤ bminimize
xcT x
subject to diag (Ax − b) � 0SOCP and equivalent SDP:
minimizex
f T x
subject to ‖Aix + bi‖ ≤ cTi x + di , i = 1, . . . ,m
minimizex
f T x
subject to[ (
cTi x + di)I Aix + bi
(Aix + bi )T cTi x + di
]� 0, i = 1, . . . ,m
D. Palomar Convex Problems 28 / 48
![Page 29: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/29.jpg)
Semidefinite Programming (SDP)
Eigenvalue minimization:
minimizex
λmax (A (x))
where A (x) = A0 + x1A1 + · · ·+ xnAn, is equivalent to SDP
minimizex
t
subject to A (x) � tI
It follows from
λmax (A (x)) ≤ t ⇐⇒ A (x) � tI
D. Palomar Convex Problems 29 / 48
![Page 30: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/30.jpg)
Outline
1 Optimization Problems
2 Convex Optimization
3 Quasi-Convex Optimization
4 Classes of Convex Problems: LP, QP, SOCP, SDP
5 Multicriterion Optimization (Pareto Optimality)
6 Solvers
![Page 31: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/31.jpg)
Scalarization for Multicriterion Problems
Consider a multicriterion optimization with q different objectives:
f0 (x) = (F1 (x) , . . . ,Fq (x)) .
To find Pareto optimal points, minimize the positive weighted sum:
λT f0 (x) = λ1F1 (x) + · · ·+ λqFq (x) .
Example: regularized least-squares:
minimizex
‖Ax − b‖22 + γ ‖x‖22 .
D. Palomar Convex Problems 31 / 48
![Page 32: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/32.jpg)
References on Convex Problems
Chapter 4 ofStephen Boyd and Lieven Vandenberghe, Convex Optimization.Cambridge, U.K.: Cambridge University Press, 2004.
https://web.stanford.edu/~boyd/cvxbook/
D. Palomar Convex Problems 32 / 48
![Page 33: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/33.jpg)
Outline
1 Optimization Problems
2 Convex Optimization
3 Quasi-Convex Optimization
4 Classes of Convex Problems: LP, QP, SOCP, SDP
5 Multicriterion Optimization (Pareto Optimality)
6 Solvers
![Page 34: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/34.jpg)
Solvers
A solver is an engine for solving a particular type of mathematicalproblem, such as a convex program.Every programming language (e.g., Matlab, Octave, R, Python, C,C++) has a long list of available solvers to choose from.Solvers typically handle only a certain class of problems, such as LPs,QPs, SOCPs, SDPs, or GPs.They also require that problems be expressed in a standard form.Most problems do not immediately present themselves in a standardform, so they must be transformed into standard form.
D. Palomar Convex Problems 34 / 48
![Page 35: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/35.jpg)
Solver Example: Matlab’s linprog
A program for solving LPs:
x = linprog( c, A, b, A_eq, B_eq, l, u )
Problems must be expressed in the following standard form:
minimizex
cT x
subject to Ax ≤ bAeqx = beql ≤ x ≤ u
D. Palomar Convex Problems 35 / 48
![Page 36: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/36.jpg)
Conversion to Standard Form: Common Tricks
Representing free variables as the difference of nonnegative variables:
x free =⇒ x+ − x−, x+ ≥ 0, x− ≥ 0
Elimininating inequality constraints using slack variables:
aT x ≤ b =⇒ aT x + s = b, s ≥ 0
Splitting equality constraints into inequalities:
aT x = b =⇒ aT x ≤ b, aT x ≥ b
D. Palomar Convex Problems 36 / 48
![Page 37: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/37.jpg)
Solver Example: SeDuMi
A program for solving LPs, SOCPs, SDPs, and related problems:
x = sedumi( A, b, c, K )
Solves problems of the form:
minimizex
cT x
subject to Ax = bx ∈ K , K1 ×K2 × · · · × KL
where each set Ki ⊆ Rni , i = 1, 2, . . . , L is chosen from a very shortlist of cones.The Matlab variable K gives the number, types, and dimensions of thecones Ki .
D. Palomar Convex Problems 37 / 48
![Page 38: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/38.jpg)
Solver Example: SeDuMi
Cones supported by SeDuMi:free variables: Rni
a nonnegative orthant: Rni+ (for linear inequalities)
a real or complex second-order cone:
Qn , {(x, y) ∈ Rn × R | ‖x‖2 ≤ y}Qn
c , {(x, y) ∈ Cn × R | ‖x‖2 ≤ y}
a real or complex semidefinite cone:
Sn+ ,{X ∈ Rn×n | X = XT , X � 0
}Hn+ ,
{X ∈ Cn×n | X = XH , X � 0
}The cones must be arranged in this order, i.e., the free variables first, thenthe nonnegative orthants, then the second-order cones, then thesemidefinite cones.
D. Palomar Convex Problems 38 / 48
![Page 39: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/39.jpg)
Example: Norm Approximation
Consider the norm approximation problem:
minimizex
‖Ax− b‖
An optimal value x? minimizes the residuals
rk = aTk x− bk , k = 1, 2, . . . ,m
according to the measure defined by the norm ‖·‖Obviously, the value of x? depends significantly upon the choice ofthat norm...... and so does the process of conversion to standard form
D. Palomar Convex Problems 39 / 48
![Page 40: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/40.jpg)
Example: Euclidean or `2-Norm
Norm approximation problem:
minimizex
‖Ax− b‖2 =(∑m
k=1(aTk x− bk
)2)1/2
No need to use any solver here: this is a least squares (LS) problem,with an analytic solution:
x? = (ATA)−1ATb
In Matlab or Octave, a single command computes the solution:>> x = A \ b
Similarly, in R:>> x = solve(A, b)
D. Palomar Convex Problems 40 / 48
![Page 41: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/41.jpg)
Example: Chebyshev or `∞-Norm
Norm approximation problem:
minimizex
‖Ax− b‖∞ = minimizex
max1≤k≤m
∣∣aTk x− bk∣∣
This can be expressed as a linear program:
minimizex,t
t
subject to −t1 ≤ Ax− b ≤ t1
or, equivalently,
minimizex,t
[0T 1
] [ xt
]subject to
[A −1−A −1
] [xt
]≤[
b−b
]
D. Palomar Convex Problems 41 / 48
![Page 42: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/42.jpg)
Example: Chebyshev or `∞-Norm
Recall the final formulation:
minimizex,t
[0T 1
] [ xt
]subject to
[A −1−A −1
] [xt
]≤[
b−b
]
Matlab’s linprog call:>> xt = linprog( [zeros(n,1); 1], ...
[A,-ones(m,1); -A,-ones(m,1)], ...[b; -b] )
>> x = xt(1:n)
D. Palomar Convex Problems 42 / 48
![Page 43: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/43.jpg)
Example: Manhattan or `1-Norm
Norm approximation problem:
minimizex
‖Ax− b‖1 = minimizex
∑mk=1
∣∣aTk x− bk∣∣
This can be expressed as a linear program:
minimizex,t
1T t
subject to −t ≤ Ax− b ≤ t
or, equivalently,
minimizex,t
[0T 1T
] [ xt
]subject to
[A −I−A −I
] [xt
]≤[
b−b
]
D. Palomar Convex Problems 43 / 48
![Page 44: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/44.jpg)
Example: Manhattan or `1-Norm
Recall the final formulation:
minimizex,t
[0T 1T
] [ xt
]subject to
[A −I−A −I
] [xt
]≤[
b−b
]
Matlab’s linprog call:>> xt = linprog( [zeros(n,1); ones(n,1)], ...
[A,-eye(m,1); -A,-eye(m,1)], ...[b; -b] )
>> x = xt(1:n)
D. Palomar Convex Problems 44 / 48
![Page 45: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/45.jpg)
Example: Constrained Euclidean or `2-Norm
Constrained norm approximation problem:
minimizex
‖Ax− b‖2subject to Cx = d
l ≤ x ≤ u
This is not a least squares problem, but it is QP and an SOCP.This can be expressed as
minimizex,y,t,sl ,su
t
subject to Ax− b = yCx = dx− sl = lx + su = usl , su ≥ 0‖y‖2 ≤ t
D. Palomar Convex Problems 45 / 48
![Page 46: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/46.jpg)
Example: Constrained Euclidean or `2-Norm
Equivalently: minimizex,y,t,sl ,su
[0T 0T 0T 0T 1
]x̄
subject to
A −ICI −II I
x̄ ≤
bdlu
x̄ ∈ Rn × Rn
+ × Rn+ × Qm
SeDuMi call:>> AA = [ A, zeros(m,n), zeros(m,n), -eye(m), 0;
C, zeros(p,n), zeros(p,n) zeros(p,n), 0;eye(n), -eye(n), zeros(n,n), zeros(n,n), 0;eye(n), zeros(n,n), eye(n), zeros(n,n), 0 ]
>> bb = [ b; d; l; u ]>> cc = [ zeros(3*n+m,1); 1 ]>> K.f = n; K.l = 2*n; K.q = m + 1;>> xsyz = sedumi( AA, bb, cc, K )>> x = xsyz(1:n)D. Palomar Convex Problems 46 / 48
![Page 47: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/47.jpg)
Modeling Frameworks: cvx
A modeling framework simplifies the use of a numerical technology byshielding the user from the underlying mathematical details.Examples: SDPSOL, YALMIP, cvx, etc.cvx is designed to support convex optimization or, more specifically,disciplined convex programming (available in Matlab, Python, R, Julia,Octave, etc.)People don’t simply write optimization problems and hope that theyare convex; instead they draw from a "mental library"of functions andsets with known convexity properties and combine them in ways thatconvex analysis guarantees will produce convex results.Disciplined convex programming formalizes this methodology.Links:
cvx: http://cvxr.comcvx user guide: http://web.cvxr.com/cvx/doc/CVX.pdfcvx for R (cvxr): https://github.com/cvxgrp/CVXR
D. Palomar Convex Problems 47 / 48
![Page 48: Convex Optimization Problems Prof. Daniel P. Palomar](https://reader033.vdocuments.site/reader033/viewer/2022051420/627e2c4ebc161a2e370ee167/html5/thumbnails/48.jpg)
Thanks
For more information visit:
https://www.danielppalomar.com