Matroids fromLossless Expander Graphs

Nick HarveyU. Waterloo

Maria-Florina BalcanGeorgia Tech

Matroids

• Ground Set V• Family of Independent Sets I• Axioms:• ; 2 I “nonempty”• J ½ I 2 I ) J 2 I “downwards closed”• J, I 2 I and |J|<|I| ) 9x2InJ s.t. J+x 2 I

“maximum-size sets can be found greedily”

• Rank function: r(S) = max { |I| : I2I and IµS }

Partition Matroid· 2 · 2

A1 A2

• This is a matroid• In general, if V = A1 [ [ Ak, then

is a partition matroid

V

. .

Intersecting Ai’s

a b c d e f g h i j k l

· 2 · 2

A1 A2

• Topic of This Talk:What if Ai’s intersect? Then I is not a matroid.

• For example, {a,b,k,l} and {f,g,h} are both maximal sets in I.

V

A fix

a b c d e f g h i j k l

· 2 · 2

A1 A2

• After truncating the rank to 3, then {a,b,k,l}I.• Checking a few cases shows that I is a matroid.

V

A general fix (for two Ai’s)

a b c d e f g h i j k l

· b1 · b2

A1 A2

• This works for any A1,A2 and bounds b1,b2

(unless b1+b2-|A1ÅA2|<0)

• Summary: There is a matroid that’s like a partition matroid, if bi’s large relative to |A1ÅA2|

V

The Main Question• Let V = A1[[Ak and b1,,bk2N• Is there a matroid s.t.• r(Ai) · bi 8i• r(S) is “as large as possible” for SAi (this is not formal)

• If Ai’s are disjoint, solution is partition matroid

• If Ai’s are “almost disjoint”, can we find a matroid that’s “almost” a partition matroid?

Next: formalize this

Lossless Expander Graphs

• Definition:G =(U[V, E) is a (D,K,²)-lossless expander if– Every u2U has degree D– |¡ (S)| ¸ (1-²)¢D¢|S| 8SµU with |S|·K,

where ¡ (S) = { v2V : 9u2S s.t. {u,v}2E }

“Every small left-set has nearly-maximalnumber of right-neighbors”

U V

Lossless Expander Graphs

• Definition:G =(U[V, E) is a (D,K,²)-lossless expander if– Every u2U has degree D– |¡ (S)| ¸ (1-²)¢D¢|S| 8SµU with |S|·K,

where ¡ (S) = { v2V : 9u2S s.t. {u,v}2E }

“Neighborhoods of left-vertices areK-wise-almost-disjoint”

Why “lossless”?Spectral techniques cannot obtain ² < 1/2.

U V

Trivial Example: Disjoint Neighborhoods

• Definition:G =(U[V, E) is a (D,K,²)-lossless expander if– Every u2U has degree D– |¡ (S)| ¸ (1-²)¢D¢|S| 8SµU with |S|·K,

where ¡ (S) = { v2V : 9u2S s.t. {u,v}2E }

• If left-vertices have disjoint neighborhoods, this gives an expander with ²=0, K=1

U V

Main Theorem: Trivial Case

• Suppose G =(U[V, E) has disjoint left-neighborhoods.• Let A={A1,…,Ak} be defined by A = { ¡(u) : u2U }.

• Let b1, …, bk be non-negative integers.

• Theorem:

is family of independent sets of a matroid.

I = f I : jI \ [ j 2 J A j j ·X

j 2 Jbj 8J gI = f I : jI \ A j j · bj 8j g

A1

A2

· b1

· b2U V

Main Theorem• Let G =(U[V, E) be a (D,K,²)-lossless expander• Let A={A1,…,Ak} be defined by A = { ¡(u) : u2U }

• Let b1, …, bk satisfy bi ¸ 4²D 8i

A1

· b1

A2

· b2

Main Theorem• Let G =(U[V, E) be a (D,K,²)-lossless expander• Let A={A1,…,Ak} be defined by A = { ¡(u) : u2U }

• Let b1, …, bk satisfy bi ¸ 4²D 8i

• “Wishful Thinking”: I is a matroid, whereI =f I : jI \ [ j 2 J A j j ·

X

j 2 Jbj 8J g

Main Theorem• Let G =(U[V, E) be a (D,K,²)-lossless expander• Let A={A1,…,Ak} be defined by A = { ¡(u) : u2U }

• Let b1, …, bk satisfy bi ¸ 4²D 8i

• Theorem: I is a matroid, whereI =f I : jI \ [ j 2 J A j j ·

X

j 2 Jbj ¡

³ X

j 2 JjA j j ¡ j [ j 2 J A j j

´

8J s.t. jJ j · K^ jI j · ²DK g

Main Theorem• Let G =(U[V, E) be a (D,K,²)-lossless expander• Let A={A1,…,Ak} be defined by A = { ¡(u) : u2U }

• Let b1, …, bk satisfy bi ¸ 4²D 8i

• Theorem: I is a matroid, where

• Trivial case: G has disjoint neighborhoods,i.e., K=1 and ²=0.

I =f I : jI \ [ j 2 J A j j ·X

j 2 Jbj ¡

³ X

j 2 JjA j j ¡ j [ j 2 J A j j

´

8J s.t. jJ j · K^ jI j · ²DK g

= 0

= 1

= 0

= 1

• Paving matroids can also be constructed by the main theorem

• A paving matroid is a matroid of rank D where every circuit has cardinality either D or D+1

Application: Paving Matroids

;

V

A1A2

A3

Ak

• Paving matroids can also be constructed by the main theorem

• A paving matroid is a matroid of rank D where every circuit has cardinality either D or D+1

• Sketch:– Let A={A1,...,Ak} be the circuits of cardinality D– A is a code of constant weight D and distance ¸ 4– This gives a (D,K,²)-expander with K=2 and ²=1-2/D

• Plugging this into the main theorem gives it(Actually, you need a more precise version from our paper)

Application: Paving Matroids

LB for Learning Submodular Functions

;

VA2

A1

• Similar idea to paving matroid construction,except we need “deeper valleys”

• If there are many valleys, the algorithm can’t learn all of them

n1/3

log2 n

LB for Learning Submodular Functions• Let G =(U[V, E) be a (D,K,²)-lossless expander, where Ai =

¡(ui) and– |V|=n − |U|=nlog n

– D = K = n1/3 − ² = log2(n)/n1/3

• Such graphs exist by the probabilistic method• Sketch:– Delete each node in U with prob. ½, then use main theorem to

get a matroid– If ui2U was not deleted then r(Ai) · bi = 4²D = O(log2 n)

– Claim: If ui deleted then Ai 2 I (Needs a proof) ) r(Ai) = |Ai| = D = n1/3

– Since # Ai’s = |U| = nlog n, no algorithm can learna significant fraction of r(Ai) values in polynomial time

I =f I : jI \ Cj · f (C) 8C 2 C gLemma: Let I be defined by

where f : C ! Z is some function. For any I 2 I, letT(I ) = f C : jI \ Cj = f (C) g

be the “tight sets” for I. Suppose thatC1 2 T(I ) ^ C2 2 T(I ) =) C1 [ C2 2 T(I )

Then I is independent sets of a matroid.

Proof: Let J,I 2 I and |J|<|I|. Must show 9x2InJ s.t. J+x 2 I.Let C be the maximal set in T(J). Then |IÅC| · f(C) = |JÅC|.Since |I|>|J|, 9x in In(C [ J).We must have J+x 2 I,because every C’3x has C’T(J).So |(J+x) Å C’|·f(C’). So J+x 2 I.

C JI

x

Concluding Remarks• A new family of matroids that give a common

generalization of partition & paving matroids

• Useful if you want...– a partition matroid, but the sets are not a partition– a paving matroid with deeper “valleys”

• Matroids came from analyzing learnability of submodular functions. – Imply a (n1/3) lower bound– Nearly matches O(n1/2) upper bound

Open Questions

• Other applications of these matroids?• n1/2 lower bound for learning submodular functions?• Are these matroids “maximal” s.t. |IÅAi|·bi?• Are these matroids linear?