constructive convex analysis [0.5ex] and disciplined ...boyd/papers/pdf/cvx_dcp.pdf · constructive...
TRANSCRIPT
![Page 1: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/1.jpg)
Constructive Convex Analysisand Disciplined Convex Programming
Stephen Boyd Steven DiamondAkshay Agrawal Junzi Zhang
EE & CS DepartmentsStanford University
1
![Page 2: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/2.jpg)
Outline
Convex Optimization
Constructive Convex Analysis
Disciplined Convex Programming
Modeling Frameworks
Conclusions
2
![Page 3: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/3.jpg)
Outline
Convex Optimization
Constructive Convex Analysis
Disciplined Convex Programming
Modeling Frameworks
Conclusions
Convex Optimization 3
![Page 4: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/4.jpg)
Convex optimization problem — standard form
minimize f0(x)subject to fi (x) ≤ 0, i = 1, . . . ,m
Ax = b
with variable x ∈ Rn
I objective and inequality constraints f0, . . . , fm are convexfor all x , y , θ ∈ [0, 1],
fi (θx + (1− θ)y) ≤ θfi (x) + (1− θ)fi (y)
i.e., graphs of fi curve upwardI equality constraints are linear
Convex Optimization 4
![Page 5: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/5.jpg)
Convex optimization problem — conic form
cone program:
minimize cT xsubject to Ax = b, x ∈ K
with variable x ∈ Rn
I linear objective, equality constraints; K is convex coneI special cases:
I linear program (LP)I semidefinite program (SDP)
I the modern canonical formI there are well developed solvers for cone programs
Convex Optimization 5
![Page 6: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/6.jpg)
Other canonical formsI quadratic program (QP):
minimize 12xT Px + qT x
subject to l ≤ Ax ≤ uI smooth optimization:
minimize f (x)
where f : Rn → R is smoothI linearly constrained least squares:
minimize ‖Ax − b‖22subject to Fx = g
I prox-affine:
minimize∑N
i=1 fi (Hi xi )subject to
∑Ni=1 Ai xi = b.
Convex Optimization 6
![Page 7: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/7.jpg)
Why convex optimization?
I beautiful, fairly complete, and useful theoryI solution algorithms that work well in theory and practice
I convex optimization is actionableI many applications in
I controlI combinatorial optimizationI signal and image processingI communications, networksI circuit designI machine learning, statisticsI finance
. . . and many more
Convex Optimization 7
![Page 8: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/8.jpg)
How do you solve a convex problem?
I use an existing custom solver for your specific problem
I develop a new solver for your problem using a currentlyfashionable method
I requires workI but (with luck) will scale to large problems
I transform your problem into a cone program, and use astandard cone program solver
I can be automated using domain specific languages
Convex Optimization 8
![Page 9: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/9.jpg)
Outline
Convex Optimization
Constructive Convex Analysis
Disciplined Convex Programming
Modeling Frameworks
Conclusions
Constructive Convex Analysis 9
![Page 10: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/10.jpg)
Curvature: Convex, concave, and affine functions
I f is concave if −f is convex, i.e., for any x , y , θ ∈ [0, 1],
f (θx + (1− θ)y) ≥ θf (x) + (1− θ)f (y)
I f is affine if it is convex and concave, i.e.,
f (θx + (1− θ)y) = θf (x) + (1− θ)f (y)
for any x , y , θ ∈ [0, 1]I f is affine ⇐⇒ it has form f (x) = aT x + b
Constructive Convex Analysis 10
![Page 11: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/11.jpg)
Verifying a function is convex or concave
(verifying affine is easy)
approaches:
I via basic definition (inequality)I via first or second order conditions, e.g., ∇2f (x) � 0
I via convex calculus: construct f usingI library of basic functions that are convex or concaveI calculus rules or transformations that preserve convexity
Constructive Convex Analysis 11
![Page 12: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/12.jpg)
Convex functions: Basic examples
I xp (p ≥ 1 or p ≤ 0), e.g., x2, 1/x (x > 0)I ex
I x log xI aT x + bI xT Px (P � 0)I ‖x‖ (any norm)I max(x1, . . . , xn)
Constructive Convex Analysis 12
![Page 13: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/13.jpg)
Concave functions: Basic examples
I xp (0 ≤ p ≤ 1), e.g.,√
xI log xI√xy
I xT Px (P � 0)I min(x1, . . . , xn)
Constructive Convex Analysis 13
![Page 14: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/14.jpg)
Convex functions: Less basic examples
I x2/y (y > 0), xT x/y (y > 0), xT Y −1x (Y � 0)I log(ex1 + · · ·+ exn )I f (x) = x[1] + · · ·+ x[k] (sum of largest k entries)I f (x , y) = x log(x/y) (x , y > 0)I λmax(X ) (X = X T )
Constructive Convex Analysis 14
![Page 15: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/15.jpg)
Concave functions: Less basic examples
I log det X , (det X )1/n (X � 0)I log Φ(x) (Φ is Gaussian CDF)I λmin(X ) (X = X T )
Constructive Convex Analysis 15
![Page 16: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/16.jpg)
Calculus rules
I nonnegative scaling: f convex, α ≥ 0 =⇒ αf convex
I sum: f , g convex =⇒ f + g convex
I affine composition: f convex =⇒ f (Ax + b) convex
I pointwise maximum: f1, . . . , fm convex =⇒ maxi fi (x) convex
I composition: h convex increasing, f convex =⇒ h(f (x)) convex
. . . and similar rules for concave functions
(there are other more advanced rules)
Constructive Convex Analysis 16
![Page 17: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/17.jpg)
Examples
from basic functions and calculus rules, we can show convexity of . . .
I piecewise-linear function: maxi=1....,k(aTi x + bi )
I `1-regularized least-squares cost: ‖Ax − b‖22 + λ‖x‖1, with λ ≥ 0
I sum of largest k elements of x : x[1] + · · ·+ x[k]
I log-barrier: −∑m
i=1 log(−fi (x)) (on {x | fi (x) < 0}, fi convex)I KL divergence: D(u, v) =
∑i (ui log(ui/vi )− ui + vi ) (u, v > 0)
Constructive Convex Analysis 17
![Page 18: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/18.jpg)
A general composition rule
h(f1(x), . . . , fk(x)) is convex when h is convex and for each i
I h is increasing in argument i , and fi is convex, orI h is decreasing in argument i , and fi is concave, orI fi is affine
I there’s a similar rule for concave compositions(just swap convex and concave above)
I this one rule subsumes all of the othersI this is pretty much the only rule you need to know
Constructive Convex Analysis 18
![Page 19: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/19.jpg)
Example
let’s show that
f (u, v) = (u + 1) log((u + 1)/min(u, v))
is convex
I u, v are variables with u, v > 0I u + 1 is affine; min(u, v) is concaveI since x log(x/y) is convex in (x , y), decreasing in y ,
f (u, v) = (u + 1) log((u + 1)/min(u, v))
is convex
Constructive Convex Analysis 19
![Page 20: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/20.jpg)
Example
I log(eu1 + · · ·+ euk ) is convex, increasingI so if f (x , ω) is convex in x for each ω and γ > 0,
log((
eγf (x ,ω1) + · · ·+ eγf (x ,ωk))/k)
is convexI this is log E eγf (x ,ω), where ω ∼ U ({ω1, . . . , ωk})I arises in stochastic optimization via bound
log Prob(f (x , ω) ≥ 0) ≤ log E eγf (x ,ω)
Constructive Convex Analysis 20
![Page 21: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/21.jpg)
Constructive convexity verification
I start with function given as expressionI build parse tree for expression
I leaves are variables or constants/parametersI nodes are functions of children, following general rule
I tag each subexpression as convex, concave, affine, constantI variation: tag subexpression signs, use for monotonicity
e.g., (·)2 is increasing if its argument is nonnegativeI sufficient (but not necessary) for convexity
Constructive Convex Analysis 21
![Page 22: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/22.jpg)
Example
for x < 1, y < 1(x − y)2
1−max(x , y)is convex
I (leaves) x , y , and 1 are affine expressionsI max(x , y) is convex; x − y is affineI 1−max(x , y) is concaveI function u2/v is convex, monotone decreasing in v for v > 0
hence, convex with u = x − y , v = 1−max(x , y)
Constructive Convex Analysis 22
![Page 23: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/23.jpg)
Exampleanalyzed by dcp.stanford.edu (Diamond 2014)
Constructive Convex Analysis 23
![Page 24: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/24.jpg)
Example
I f (x) =√
1 + x2 is convex
I but cannot show this using constructive convex analysisI (leaves) 1 is constant, x is affineI x2 is convexI 1 + x2 is convexI but
√1 + x2 doesn’t match general rule
I writing f (x) = ‖(1, x)‖2, however, worksI (1, x) is affineI ‖(1, x)‖2 is convex
Constructive Convex Analysis 24
![Page 25: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/25.jpg)
Outline
Convex Optimization
Constructive Convex Analysis
Disciplined Convex Programming
Modeling Frameworks
Conclusions
Disciplined Convex Programming 25
![Page 26: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/26.jpg)
Disciplined convex programming (DCP)
(Grant, Boyd, Ye, 2006)
I framework for describing convex optimization problemsI based on constructive convex analysisI sufficient but not necessary for convexityI basis for several domain specific languages and tools for
convex optimization
Disciplined Convex Programming 26
![Page 27: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/27.jpg)
Disciplined convex program: Structure
a DCP has
I zero or one objective, with formI minimize {scalar convex expression} orI maximize {scalar concave expression}
I zero or more constraints, with formI {convex expression} <= {concave expression} orI {concave expression} >= {convex expression} orI {affine expression} == {affine expression}
Disciplined Convex Programming 27
![Page 28: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/28.jpg)
Disciplined convex program: Expressions
I expressions formed fromI variables,I constants/parameters,I and functions from a library
I library functions have known convexity, monotonicity, andsign properties
I all subexpressions match general composition rule
Disciplined Convex Programming 28
![Page 29: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/29.jpg)
Disciplined convex program
I a valid DCP isI convex-by-construction (cf. posterior convexity analysis)I ‘syntactically’ convex (can be checked ‘locally’)
I convexity depends only on attributes of library functions,and not their meanings
I e.g., could swap√· and 4
√·, or exp · and (·)+, since their
attributes match
Disciplined Convex Programming 29
![Page 30: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/30.jpg)
Canonicalization
I easy to build a DCP parser/analyzerI not much harder to implement a canonicalizer, which
transforms DCP to equivalent cone programI then solve the cone program using a generic solver
I yields a modeling framework for convex optimization
Disciplined Convex Programming 30
![Page 31: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/31.jpg)
Outline
Convex Optimization
Constructive Convex Analysis
Disciplined Convex Programming
Modeling Frameworks
Conclusions
Modeling Frameworks 31
![Page 32: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/32.jpg)
Optimization modeling languages
I domain specific language (DSL) for optimizationI express optimization problem in high level language
I declare variablesI form constraints and objectiveI solve
I long history: AMPL, GAMS, . . .I no special support for convex problemsI very limited syntaxI callable from, but not embedded in other languages
Modeling Frameworks 32
![Page 33: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/33.jpg)
Modeling languages for convex optimization
all based on DCP
YALMIP Matlab Lofberg 2004CVX Matlab Grant, Boyd 2005CVXPY Python Diamond, Boyd; Agrawal et al. 2013; 2018Convex.jl Julia Udell et al. 2014CVXR R Fu, Narasimhan, Boyd 2017
some precursors
I SDPSOL (Wu, Boyd, 2000)I LMITOOL (El Ghaoui et al., 1995)
Modeling Frameworks 33
![Page 34: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/34.jpg)
CVX
cvx_beginvariable x(n) % declare vector variableminimize sum(square(A*x-b)) + gamma*norm(x,1)subject to norm(x,inf) <= 1
cvx_end
I A, b, gamma are constants (gamma nonnegative)I variables, expressions, constraints exist inside problemI after cvx_end
I problem is canonicalized to cone programI then solved
Modeling Frameworks 34
![Page 35: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/35.jpg)
Some functions in the CVX library
function meaning attributesnorm(x, p) ‖x‖p, p ≥ 1 cvxsquare(x) x2 cvxpos(x) x+ cvx, nondecrsum_largest(x,k) x[1] + · · ·+ x[k] cvx, nondecrsqrt(x)
√x , x ≥ 0 ccv, nondecr
inv_pos(x) 1/x , x > 0 cvx, nonincrmax(x) max{x1, . . . , xn} cvx, nondecrquad_over_lin(x,y) x2/y , y > 0 cvx, nonincr in ylambda_max(X) λmax(X ), X = X T cvx
Modeling Frameworks 35
![Page 36: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/36.jpg)
DCP analysis in CVX
cvx_beginvariables x ysquare(x+1) <= sqrt(y) % acceptedmax(x,y) == 1 % not DCP...
Disciplined convex programming error:Invalid constraint: {convex} == {real constant}
Modeling Frameworks 36
![Page 37: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/37.jpg)
CVXPY
import cvxpy as cpx = cp.Variable(n)cost = cp.sum_squares(A*x-b) + gamma*cp.norm(x,1)prob = cp.Problem(cp.Minimize(cost),
[cp.norm(x,"inf") <= 1])opt_val = prob.solve()solution = x.value
I A, b, gamma are constants (gamma nonnegative)I variables, expressions, constraints exist outside of problemI solve method canonicalizes, solves, assigns value
attributes
Modeling Frameworks 37
![Page 38: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/38.jpg)
Signed DCP in CVXPY
function meaning attributes
abs(x) |x | cvx, nondecr for x ≥ 0,nonincr for x ≤ 0
huber(x)
{x2, |x | ≤ 12|x | − 1, |x | > 1
cvx, nondecr for x ≥ 0,nonincr for x ≤ 0
norm(x, p) ‖x‖p, p ≥ 1 cvx, nondecr for x ≥ 0,nonincr for x ≤ 0
square(x) x2 cvx, nondecr for x ≥ 0,nonincr for x ≤ 0
Modeling Frameworks 38
![Page 39: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/39.jpg)
DCP analysis in CVXPY
expr = (x − y)2
1−max(x , y)
x = cp.Variable()y = cp.Variable()expr = cp.quad_over_lin(x - y, 1 - cp.maximum(x,y))expr.curvature # CONVEXexpr.sign # POSITIVEexpr.is_dcp() # True
Modeling Frameworks 39
![Page 40: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/40.jpg)
Parameters in CVXPY
I symbolic representations of constantsI can specify signI change value of constant without re-parsing problem
I for-loop style trade-off curve:
x_values = []for val in numpy.logspace(-4, 2, 100):
gamma.value = valprob.solve()x_values.append(x.value)
Modeling Frameworks 40
![Page 41: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/41.jpg)
Parallel style trade-off curve
# Use tools for parallelism in standard library.from multiprocessing import Pool
# Function maps gamma value to optimal x.def get_x(gamma_value):
gamma.value = gamma_valueresult = prob.solve()return x.value
# Parallel computation with N processes.pool = Pool(processes = N)x_values = pool.map(get_x, numpy.logspace(-4, 2, 100))
Modeling Frameworks 41
![Page 42: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/42.jpg)
Convex.jl
using Convexx = Variable(n);cost = sum_squares(A*x-b) + gamma*norm(x,1);prob = minimize(cost, [norm(x,Inf) <= 1]);opt_val = solve!(prob);solution = x.value;
I A, b, gamma are constants (gamma nonnegative)I similar structure to CVXPYI solve! method canonicalizes, solves, assigns value
attributes
Modeling Frameworks 42
![Page 43: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/43.jpg)
Outline
Convex Optimization
Constructive Convex Analysis
Disciplined Convex Programming
Modeling Frameworks
Conclusions
Conclusions 43
![Page 44: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/44.jpg)
Conclusions
I DCP is a formalization of constructive convex analysisI simple method to certify problem as convex
(sufficient, but not necessary)I basis of several DSLs/modeling frameworks for convex
optimization
I modeling frameworks make rapid prototyping of convexoptimization based methods easy
Conclusions 44
![Page 45: Constructive Convex Analysis [0.5ex] and Disciplined ...boyd/papers/pdf/cvx_dcp.pdf · Constructive convexity verification I start with function given as expression I build parse](https://reader034.vdocuments.site/reader034/viewer/2022042911/5f4350f55c90a95d9d55a255/html5/thumbnails/45.jpg)
ReferencesI Disciplined Convex Programming (Grant, Boyd, Ye)I Graph Implementations for Nonsmooth Convex Programs
(Grant, Boyd)I Matrix-Free Convex Optimization Modeling
(Diamond, Boyd)I A Rewriting System for Convex Optimization Problems
(Agrawal, Verschueren, Diamond, Boyd)
I CVX: http://cvxr.com/I CVXPY: https://www.cvxpy.org/I Convex.jl: http://convexjl.readthedocs.org/I CVXR: https://cvxr.rbind.io/I DCP tools: https://dcp.stanford.edu/
Conclusions 45