submodular set function maximization a mini-survey chandra chekuri univ. of illinois,...

Submodular Set Function Maximization

A Mini-SurveyChandra Chekuri

Univ. of Illinois, Urbana-Champaign

Submodular Set Functions

A function f : 2N R is submodular if

f(A) + f(B) ≥ f(AB) + f(AB) for all A,B N

Equivalently,

f(A+j) – f(A) ≥ f(A+i+j) – f(A+i)

for all A N and i, j N\A

Submodular Functions

• Non-negative submodular set functions

f(A) ≥ 0 for all A

• Monotone submodular set functions

f(ϕ) = 0 and f(A) ≤ f(B) for all A B

• Symmetric submodular set functionsf(A) = f(N\A) for all A

Well-known Examples

• Cut functions in undirected graphs and hypergraphs (symmetric non-negative)

• Cut functions in directed graphs (non-negative)

• Rank functions of matroids (monotone)

• Coverage in set systems (monotone)

• many others ...

Maximizing Submodular Set Functions

Given f on a ground set N via a value oracle

max S N f(S)

S satisfies some constraints

Maximizing Submodular Set Functions

Given f on a ground set N via a value oracle

max S N f(S)


Motivation:

• Many non-trivial applications (easy to miss!)

• Generalize known results for modular functions

Unconstrained Problem

max S N f(S)

• Uninteresting for monotone f

• NP-Hard for non-negative f (Max-Cut is a special case)

• Very hard to approximate for arbitrary f (reduction from Set Packing)


[Feige-Mirrokni-Vondrak’07]

First O(1) approximation for non-negative f !

Easy O(1) algorithms, ½ for symmetric case

Non-trivial 2/5 = 0.4 approximation (slight improvement to 0.41 [Vondrak])



First O(1) approximation for non-negative f !

Easy O(1) algorithms, ½ for symmetric case

Non-trivial 2/5 = 0.4 approximation (slight improvement to 0.41 [Vondrak])

Better than ½ requires exponential # of value queries

Open Problem: Close gap between 0.41 and ½



Random set algorithm: • pick each i in N with prob ½, let R be random set• E[f(R)] ≥ OPT/4• E [f(R)] ≥ OPT/2 for symmetric f

Simple Local Search:• Initialize S to best singleton• S = local optimum for adding or deleting if improvement• Output better of S and N\S• 1/3 approx for non-negative f and ½ for symmetric f

Local Search Analysis

Lemma: If S is a local optimum then for any I S or S I, f(I) ≤ f(S).

Proof: Say S I and f(I) > f(S) then by submodularity there exists i in I\S s.t f(S+i) > f(S).

Corollary: Let S* be an optimum solution and S be a local opt. f(S S*) ≤ f(S) and f(S S*) ≤ f(S)

Local Search Analysis

f(S S*) ≤ f(S) and f(S S*) ≤ f(S)

f(S S*) + f(N\S) ≥ f(S*\S) + f(N) ≥ f(S*\S)

2f(S) + f(N\S) ≥ f(S*\S) + f(S* S) ≥ f(S*)

implies max (f(S), f(N\S)) ≥ f(S*)/3

S

N\S

S*

Maximizing Submodular Set Functions with Constraints

max S N f(S)


Question: what constraints?

For maximization probs, packing constraints natural

S I

I is a downward-closed: A I, B A implies B I

Matroid and Knapsack Constraints

• Combinatorial packing constraints• I is the intersection of some p matroids on N• Lemma: every downward-close family I on N

is the intersection of p matroids on N (for some p)

• Knapsack or matrix packing constraints• A is a m x n non-negative matrix, b is m x 1

vector• I = { x {0,1}n| A x ≤ b }

• Combination of matroid and knapsack constraints

Matroid Constraints

• Uniform matroid: I = { S : |S| ≤ k }

• Partition matroid: I = { S : |S Ni| ≤ ki, 1 ≤ i ≤ h } where N1, ..., Nh partition N, and ki are integers

• Laminar matroid: I = { S : |S U| ≤ k(U), U in F } for a laminar family of sets F

• Graphic matroid

Matroid polytope is integral and hence one can hope to capture constraints via relaxation in polytope

Monotone Functions

Cardinality Constraint

max f(S) such that |S| ≤ k

• Max k-Cover problem is special case

• Greedy gives (1-1/e) approximation [Nemhauser-Wolsey-Fisher’78]

• Unless P=NP no better approximation [Feige’98]

• Many applications, routinely used

Greedy Algorithm

1. S =

2. While |S| < k do

• i argmaxj fS(j)

• S S+i

3. Output S

Greedy Analysis

• Sj : first j elements picked by Greedy

• f(S) = δ1 + δ2... +δk

• δj ≥ (OPT – f(Sj-1))/k (monotonicity and submod)

• f(S) ≥ (1-1/k)k OPT

A Different Analysis

• S = {i1, i2, ..., ik } picked by Greedy in order

• S* = {i*1, i*2, ..., i*k } an optimum solution

• Form perfect matching between S and S* s.t elements in S S* are matched to themselves

S S*

A Different Analysis

• Form perfect matching between S and S* Renumber S* s.t ij is matched to i*j

• For each j, because Greedy chose i*j instead of i*j

δj = f(Sj-1+ ij) – f(Sj-1) ≥ f(Sj-1+ i*j) – f(Sj-1) ≥ fS(i*j)

• Summing up

f(S) ≥ Σj fS(i*j) ≥ fS(S*) = f(S S*) – f(S)

hence f(S) ≥ f(S S*)/2 ≥ f(S*)/2 if f is monotone

Why weaker analysis?

Let M=(N, I) be any matroid on N

Solve max f(S) such that S I

[Nemhauser-Wolsey-Fisher’78]

Theorem: Greedy is ½ approximation, and analysis is tight even for partition matroids

Many applications, unfortunately unknown, till recently, to approximation algorithms community!

Base Exchange Theorem

• B and B’ are distinct bases in a matroid M=(N, I)

Strong Base Exchange Theorem: There are elements i B\B’ and i’ B’\B such that B-i+i’ and B’-i’+i are both bases.

Corollary: There is a perfect matching between B\B’ and B’\B such that for each matched pair (i,i’), B-i+i’ is a base .

Proof for Greedy

• B is the solution that Greedy outputs

• B* is an optimum solution

• B and B* are bases of M by monotonicity of f

Same argument as before works by using perfect matching between B and B*

Multiple Matroids

max f(S) such that S is in intersection of p matroids

[Fisher-Nemhauser-Wolsey’78]

Theorem: Greedy is 1/(p+1) approximation

Generalize matching argument to match one element of Greedy to p elements of OPT

Also works for p-systems

Non-negative functions?

max f(S) such that S is in intersection of p matroids

[Lee-Mirrokni-Nagarajan-Sviridenko’08]

Theorem: For fixed p, local search based algorithm that achieves Ω(1/p) approximation.

[Gupta-Roth-Schoenbeck-Talwar’09]

For all p and with simple proof combining Greedy and unconstrained algorithm. Slightly worse constants.

Knapsack Constraints

Similar ideas can be used with standard guessing large items etc

Can we do better?

• ½ for one matroid, hardness is (1-1/e)

• 1/(p+1) for p matroids, hardness is (p/log p)

Can we do better?

[Calinescu-C-Pal-Vondrak’07]+[Vondrak’08]=[CCPV’09]

Theorem: There is a randomized (1-1/e) approximation for maximizing a monotone f subject to a matroid constraint.

Can we do better?

[Calinescu-C-Pal-Vondrak’07]+[Vondrak’08]=[CCPV’09]

Theorem: There is a randomized (1-1/e) approximation for maximizing a monotone f subject to a matroid constraint.

[Lee-Sviridenko-Vondrak’09]

Theorem: For fixed p≥2, there is a local-search based 1/(p+ε) approximation for intersection of p matroids

New useful insight for (two) matroid intersection

Multilinear Extension of f

Question: Is there a useful continuous relaxation of f such that it can be optimized? And can we round it effectively?


[CCPV’07] inspired by [Ageev-Sviridenko]

For f: 2N R+ define F:[0,1]N R+ as

x = (x1, x2, ..., xn) [0,1]|N|

F(x) = Expect[ f(x) ] = S N f(S) px(S)

= S N f(S) i S xi i N\S (1-xi)


For f: 2N R+ define F:[0,1]N R+ as

F(x) = S N f(S) i S xi i N\S (1-xi)

F is smooth submodular ([Vondrak’08])

• F/xi ≥ 0 for all i (monotonicity)

• 2F/xixj ≤ 0 for all i,j (submodularity)

Optimizing F(x)

[Vondrak’08]

Theorem: For any down-monotone polytope P [0,1]n max F(x) s.t x P can be optimized to within a (1-1/e) approximation if we can do linear optimization over P

Algorithm: Continuous-Greedy

Generic Approach

Want to solve: max f(S) s.t S I

Relaxation: max F(x) s.t x P(I )

• P(I) is a polytope that captures/relaxes S I

• Can solve to within (1-1/e) with continuous greedy

• How to round? F is a non-linear function

Rounding in Matroids

Let M=(S,I) be a matroid

• P(M) is the independent set polytope of M

• B(M) is the base polytope of M

Algorithm:

• Run continuous greedy to obtain a point x B(M) such that F(x) ≥ (1-1/e) OPT

• Round x to a vertex of B(M) (a base)

Rounding in Matroids

[CCPV’07]

Theorem: Given any point x in B(M), there is a polynomial time algorithm to round x to a vertex x* (hence a base of M) such that F(x*) ≥ F(x).

“Pipage” rounding technique of [Ageev-Sviridenko] adapted to matroids

New Rounding Method

[C-Vondrak-Zenklusen’09]

Randomized Swap-Rounding:

• Express x = mj=1 βi Bi (convex comb. of bases)

• B = B1 , β = β1

• For k = 2 to m do • Randomly Merge β B and βk Bk into (β + βk) B’

• Set B = B’, β = (β + βk)

• Output B

Merging two Bases

Merge B’ and B’’ into a random B that looks “half” like B’ and “half” like B’’

Merging two Bases

B’ B’’

B’B’’ B’B’’

i j

Base ExchangeTheorem:

B’-i+j and B’’-j+i are both bases

Merging two Bases

B’ B’’

B’B’’ B’B’’

i j

prob ½

prob ½

B’ B’’

B’B’’ B’B’’

i i

B’ B’’

B’B’’ B’B’’

j j

Swap Rounding for Matroids

Theorem: Swap-Merging with input x in B(M) outputs a random base B such that

1. E[f(B)] ≥ F(x) and

2. Pr[ f(B) < (1-δ) F(x)] ≤ exp(- F(x) δ2/8) (concentration for lower tail of submod functions) and

3. For any vector a [0,1]n , let μ = ax then • Pr[a 1B < (1-δ) μ] ≤ exp(- μ δ2/2)

• Pr[a 1B > (1+δ) μ] ≤ ( e δ / (1+δ) δ ) μ (concentration for linear functions)

Almost like independent random rounding of x

Applications

Can handle matroid constraint plus packing constraints

x P(M) and Ax ≤ b

• (1-1/e) approximation for submodular functions subject to a matroid plus O(1) knapsack/packing constraints (or many “loose” packing constraints)

• Simpler rounding and proof for “thin” spanning trees in ATSP application ([Asadpour etal’10])

• ...

Other Tools

On-going work [C-Vondrak-Zenklusen]

• Extension of continuous greedy to handle multiple submodular functions simultaneously

• Depending rounding via swap method for polyhedra• matroid intersection• matchings and b-matchings in non-bipartite graphs• ...

Many applications of dependent randomized rounding[Arora-Frize-Kaplan, Srinivasan’01, ...., Asadpour etal,..]

Illustrative Application

Maximum Bipartite Flow in Networks with Adaptive Channel Width [Azar-Madry-Moscibroda-Panigrahy-Srinivasan’09]

• Problem motivated by capacity allocation in wireless networks

• (1-1/e) approximation via a specialized LP and complicated analysis

• An easy O(log n) approximation

Problem Definition

Base stations

Clientsθ(B,C) : threshold capacity between B and C

α(C): max flow that C desires from base stns

β(B): total capacity of B to serve clients

For each base station B, decide an operating point τ(B) ≤ β(B)• If τ(B) > θ(B,C) then u(B,C) capacity of link (B,C) is 0, otherwise u(B,C) = τ(B)

• Maximize flow from base stations to clients

β(B)

α(C)

u(B,C)

Reduction

Base stations

Clients

β(B)

α(C)

t

B

Copies of B one for each τ(B) u(τ(B),C)

N : all copies on left

f : 2N R+ where f(S) = flow from S to t

Partition Matroid constraint: can pick only one copy for each B

Summary

• Substantial progress on submodular function maximization problems

• Increased awareness and more applications

• (New) tools and connections: continuous greedy, dependent rounding, local search, ...

• Several open problems still remain

Open Problems

• Unconstrained for non-neg f. Close gap bet 0.41 and ½

• max f(S) s.t S in intersection of p matroids• p=1 we have 1-1/e approx which is tight unless P=NP• for fixed p ≥ 2, 1/(p+ε) approx, otherwise 1/(p+1)• hardness is Ω (p/log p) for large p when f is modular• close gaps, most interesting for small p• How to round F(x) for more than 1 matroid? Don’t know

integrality gap for p=2!

• max f(x) s.t x in P(M) and Ax ≤ b, A is k-column-sparse. Want Ω(1/k) approx. Known without matroid constraint [Bansal-Korula-Nagarajan-Srinivisan’10]

submodular set function maximization a mini-survey chandra chekuri univ. of illinois,...

Documents