decomposable optimisation methods
DESCRIPTION
Decomposable Optimisation Methods. LCA Reading Group, 12/04/2011 Dan-Cristian Tomozei. Convexity. Convex function Unique minimum over convex domain. Roadmap. (Sub)Gradient Method Convex Optimisation crash course NUM Basic Decomposition Methods Implicit Signalling. Roadmap. - PowerPoint PPT PresentationTRANSCRIPT
Decomposable Optimisation Methods
LCA Reading Group, 12/04/2011Dan-Cristian Tomozei
• Convex function
• Unique minimum over convex domain
Convexity
2
Roadmap• (Sub)Gradient Method• Convex Optimisation crash course• NUM• Basic Decomposition Methods• Implicit Signalling
3
Roadmap• (Sub)Gradient Method• Convex Optimisation crash course• NUM• Basic Decomposition Methods• Implicit Signalling
4
• Unconstrained convex optimisation problem
• If objective is differentiable,
• Else,
• Gain sequence – Constant– Diminishing
(Sub)gradient method
5
Roadmap• (Sub)Gradient Method• Convex Optimisation crash course• NUM• Basic Decomposition Methods• Implicit Signalling
6
• “Primal” formulation
• Convex constraints unique solution• Lagrangian
• “Dual” function – For all “feasible” points – lower bound
– Slater’s condition zero duality gap
Constrained Convex Optimisation
7
• “Primal” and “dual” formulations
• Karush-Kuhn-Tucker (KKT)
Optimality conditions
Primal variables Dual variables (i.e., Lagrange multipliers)
8
Optimum
Roadmap• (Sub)Gradient Method• Convex Optimisation crash course• NUM• Basic Decomposition Methods• Implicit Signalling
9
• Population of users • Concave utility functions (e.g., rates)• Typical formulation (e.g., [Kelly97]):– Network flows of rates– Physical links of max capacity– Routing matrix
– Dual variables = congestion shadow prices
Network Utility Maximisation
10
Roadmap• (Sub)Gradient Method• Convex Optimisation crash course• NUM• Basic Decomposition Methods• Implicit Signalling
11
• Coupling constraint
• To decouple – simply write the dual objective
• Iterative dual algorithm:– Each user computes – Use a gradient method to update dual variables, e.g.,
Dual Decomposition
12
• Coupling variable
• To decouple – consider fixed coupling variable• Iterative primal algorithm:– Solve individual problems and get partial optima
– Update primal coupling variable using gradient method
Primal Decomposition
13
Implementation issues• Certain problems can be decoupled• Dual decomposition dual algorithm– Primal vars (rates) depend directly on dual vars (prices) – Price adaptation relies on current rates– Always closed form?
• Primal decomposition– The other way around…
• Do we really need to keep track of both primal and dual variables? Can duals be “measured” instead?
14
Roadmap• (Sub)Gradient Method• Convex Optimisation crash course• NUM• Basic Decomposition Methods• Implicit Signalling
15
• Graph• Supported rate region • Network cost function – Unsupported rate allocation – Marginal cost positive and strictly increasing
• Source s wants to send data to receiver r at rate at minimum cost– Supported min-cut is at least
Multipath unicast min-cost live streaming
16
Optimisation formulation
• Write Lagrangian
• Primal-dual provably converges to optimum
17
Is it that hard?• Recall
• Dual variables have queue-like evolution!• We already queue packets!
18
Implicit Primal-Dual• Rate control via
• Rate on link (i,j)– Increase prop to backlog difference– Decrease prop to marginal cost (measurable – RTT, …)
• Influence of parameter s– Small closer optimal allocation, huge queue sizes– Large manageable queue sizes, optimality trade-off
19
Conclusion• Finding a fit-all recipe is hard• We can handle some cases• Specific formulations may lead to nice protocols
• See also– R. Srikant’s “Mathematics of Internet Congestion Control”– Kelly, Mauloo, Tan - ***– Palomar, Chiang - ***
20
Questions
21