on the capacity of information networks nick harvey collaborators: micah adler (umass), kamal jain...
Post on 20-Dec-2015
214 views
TRANSCRIPT
On the Capacity of Information Networks
Nick Harvey
Collaborators: Micah Adler (UMass), Kamal Jain (Microsoft), Bobby Kleinberg (MIT/Berkeley/Cornell), and April Lehman (MIT/Google)
s1 s2
Send items from s1t1 and s2t2
Problem: no disjoint paths
bottleneck edge
What is the capacity of a network?
t2 t1
b1⊕b2
An Information Networkb1 b2s1 s2
t2 t1
If sending information, we can do better Send xor b1⊕b2 on bottleneck edge
Network Coding New approach for information flow
problemsBlend of combinatorial optimization,
information theoryMulticast, k-Pairs
k-Pairs problems: Network coding when each commodity has one sinkAnalogous to multicommodity flow
Definitions for cyclic networks are subtle
MulticommodityFlow
Efficient algorithms for computing maximum concurrent (fractional) flow.
Connected with metric embeddings via LP duality.
Approximate max-flow min-cut theorems.
NetworkCoding
Computing the max concurrent coding rate may be: Undecidable Decidable in poly-time
No adequate duality theory.
No cut-based parameter is known to give sublinear approximation in digraphs.
Directed and undirected problems behave quite differently
Coding rate can be muchlarger than flow rate!
Butterfly: Coding rate = 1 Flow rate = ½
Thm [HKL’04,LL’04]: graphs G(V,E) whereCoding Rate = Ω( flow rate ∙ |V| )
Directed k-pairss1 s2
t2 t1
Thm: graphs G(V,E) whereCoding Rate = Ω( flow rate ∙ |E| ) And this is optimal Recurse on butterfly construction
Coding rate can be muchlarger than flow rate!
…and much larger than the sparsity(same example)
Directed k-pairs
Flow Rate Sparsity < Coding Rate
in some graphs
No known undirected instance where coding rate ≠ max flow rate!
(The undirected k-pairs conjecture)
Undirected k-pairs
Flow Rate Coding Rate Sparsity
Pigeonhole principle argumentGap can be Ω(log n) when G is an expander
Undirected k-Pairs Conjecture
Flow Rate Sparsity Coding Rate
< =? ?= <
Undirected k-pairs conjecture
Unknown until this work
Okamura-Seymour Graphs1
t1s2
t2s3
t3
s4 t4
Every cut has enough capacity to carry all commodities separated by the cut
Cut
Okamura-Seymour Max-Flows1
t1s2
t2s3
t3
s4 t4
Flow Rate = 3/4
si is 2 hops from ti.
At flow rate r, each commodity consumes 2r units of bandwidth in a graph with only 6 units of capacity.
The trouble with information flow… If an edge codes
multiple commodities, how to charge for “consuming bandwidth”?
We work around this obstacle and bound coding rate by 3/4.
s1
t1s2
t2s3
t3
s4 t4
At flow rate r, each commodity consumes at least 2r units of bandwidth in a graph with only 6 units of capacity.
Definition: A e if for every coding solution,the messages sent on edges of A uniquely determine the message sent on e.
Given A and e, how hard is it to determine whether A e? Is it even decidable?
Theorem: There is an algorithm tocompute whether A e in time O(k²m). Based on a combinatorial characterization
of informational dominance
Informational Dominance
i
i
i
What can we prove?
s1 t3
s2 t1
s3 t2
s4 t4
Combine Informational Dominance with Shannon inequalities for Entropy
Flow rate = coding rate for “Special Bipartite Graphs”: Bipartite Every source is 2 hops
away from its sink Dual of flow LP is optimized
by assigning length 1 to all edges
Next: show that proving conjecture for all graphs is quite hard
k-pairs conjecture & I/O complexity
I/O complexity model [AV’88]:A large, slow external memory consisting of
pages each containing p recordsA fast internal memory that holds 2 pagesBasic I/O operation: read in two pages from
external memory, write out one page
I/O Complexity of Matrix Transposition
Matrix transposition: Given a p×p matrix of records in row-major order, write it out in column-major order.
Obvious algorithm requires O(p²) ops.
A better algorithm uses O(p log p) ops.
I/O Complexity of Matrix Transposition
Matrix transposition: Given a p×p matrix of records in row-major order, write it out in column-major order.
Obvious algorithm requires O(p²) ops.
A better algorithm uses O(p log p) ops.
s1 s2
I/O Complexity of Matrix Transposition
Matrix transposition: Given a pxp matrix of records in row-major order, write it out in column-major order.
Obvious algorithm requires O(p²) ops.
A better algorithm uses O(p log p) ops.
s1 s2 s3 s4
I/O Complexity of Matrix Transposition
Matrix transposition: Given a pxp matrix of records in row-major order, write it out in column-major order.
Obvious algorithm requires O(p²) ops.
A better algorithm uses O(p log p) ops.
s1 s2 s3 s4
t1t3
I/O Complexity of Matrix Transposition
Matrix transposition: Given a pxp matrix of records in row-major order, write it out in column-major order.
Obvious algorithm requires O(p²) ops.
A better algorithm uses O(p log p) ops.
s1 s2 s3 s4
t1t3t2
t4
Matching Lower Bound Theorem: (Floyd ’72,
AV’88) A matrix transposition algorithm using only read and write operations (no arithmetic on values) must perform Ω(p log p) I/O operations.
s1 s2 s3 s4
t1t3t2
t4
Ω(p log p) Lower Bound
Proof: Let Nij denote the number of ops in which record (i,j) is written. For all j,
Σi Nij ≥ p log p.
Hence
Σij Nij ≥ p² log p.
Each I/O writes only p records. QED.
s1 s2 s3 s4
t1t3t2
t4
The k-pairs conjecture and I/O complexity
Definition: An oblivious algorithm is one whose pattern of read/write operations does not depend on the input.
Theorem: If there is an oblivious algorithm for matrix transposition using o(p log p) I/O ops, the undirected k-pairs conjecture is false.
s1 s2 s3 s4
t1t3t2
t4
The k-pairs conjecture and I/O complexity
Proof: Represent the algorithm
with a diagram as before.
Assume WLOG that each node has only two outgoing edges. s1 s2 s3 s4
t1t3t2
t4
The k-pairs conjecture and I/O complexity
Proof: Represent the algorithm
with a diagram as before.
Assume WLOG that each node has only two outgoing edges.
Make all edges undirected, capacity p.
Create a commodity for each matrix entry.
s1 s2 s3 s4
t1t3t2
t4
The k-pairs conjecture and I/O complexity
Proof: The algorithm itself is a
network code of rate 1. Assuming the k-pairs
conjecture, there is a flow of rate 1.
Σi,jd(si,tj) ≤ p |E(G)|.
Arguing as before, LHS is Ω(p² log p).
Hence |E(G)|=Ω(p log p).
s1 s2 s3 s4
t1t3t2
t4
Other consequences for complexity
The undirected k-pairs conjecture implies:A Ω(p log p) lower bound for matrix
transposition in the cell-probe model.
[Same proof.]A Ω(p² log p) lower bound for the running time
of oblivious matrix transposition algorithms on a multi-tape Turing machine.
[I/O model can emulate multi-tape Turing machines with a factor p speedup.]
Distance arguments Rate-1 flow solution implies Σi d(si,ti) ≤ |E|
LP duality; directed or undirected Does rate-1 coding solution imply
Σi d(si,ti) ≤ |E|?Undirected graphs: this is essentially the
k-pairs conjecture!Directed graphs: this is completely false
k commodities (si,ti)
Distance d(si,ti) = O(log k) i O(k) edges!
Recursive construction
s(1) s(2) s(3) s(4) s(5) s(6) s(7) s(8)
t(1) t(2) t(3) t(4) t(5) t(6) t(7) t(8)
Recursive Constructions1 s2
t1 t2
G (1):
Equivalent to:
s1s2
t1 t2
Edge capacity= 1
2 commodities7 edgesDistance = 3
Recursive Construction
G (n-1)G (n):
# commodities = 2n, |V| = O(2n), |E| = O(2n) Distance = 2n+1
s1 s2
t1 t2
s3 s4
t3 t4
s2n-1 s2n
t2n-1 t2n
Summary Directed instances:
Coding rate >> flow rate Undirected instances:
Conjecture: Flow rate = Coding rateProof for special bip graphsTool: Informational DominanceProving conjecture solves Matrix
Transposition Problem
Open Problems Computing the network coding rate in DAGs:
Recursively decidable? How do you compute a o(n)-factor approximation?
Undirected k-pairs conjecture:Stronger complexity consequences?Prove a Ω(log n) gap between sparsest cut
and coding rate for some graphs…or, find a fast matrix transposition algorithm.
Optimality
The graph G (n) proves:Thm [HKL’05]: graphs G(V,E) whereNCR = Ω( flow rate ∙ |E| )
G (n) is optimal:Thm [HKL’05]: graph G(V,E),NCR/flow rate = O(min {|V|,|E|,k})
s1 s2
t2 t1
A does not dominate B
Informational Dominance
Def: A dominates B if information in A determines information in Bin every network coding solution.
Informational Dominance
Def: A dominates B if information in A determines information in Bin every network coding solution.
s1 s2
t2 t1
A dominates B
Sufficient Condition: If no path fromany source B then A dominates B
Informational Dominance Examples1 s2
t1
t2
“Obviously” flow rate = NCR = 1 How to prove it? Markovicity? No two edges disconnect t1 and t2 from both sources!
Informational Dominance Examples1 s2
t1
t2
Cut A
Sufficient Condition: If no path from any source B then A dominates B
Informational Dominance Examples1 s2
t1
t2
Our characterization implies thatA dominates {t1,t2} H(A) H(t1,t2)
Cut A