communication complexity, information complexity and applications to privacy toniann pitassi...
Post on 25-Feb-2016
Embed Size (px)
DESCRIPTIONCommunication Complexity, Information Complexity and Applications to Privacy Toniann Pitassi University of Toronto. m 1. m 2. m 3. 2-Party Communication Complexity [Yao]. 2-party communication: each party has a dataset. Goal is to compute a function f(D A ,D B ). m k-1. m k. - PowerPoint PPT Presentation
Proof Complexity in the last Decade: Accomplishments, obstacles and open problems
Communication Complexity, Information Complexity and Applications to Privacy
University of TorontoMotivation for ic? (1) its an intrinsically interesting quantity;(2) Related to deep qs in complexity theory (direct sum)(3) Related to privacy (well see this)12-Party Communication Complexity[Yao]2-party communication: each party has a dataset. Goal is to compute a function f(DA,DB)m1m2m3mk-1mk
DAx1x2xnDBy1y2ymf(DA,DB)f(DA,DB)Communication complexity of a protocol for f is the number of bits exchanged between A and B.
In this talk, all protocols are assumed to be randomized.Deterministic ProtocolsA deterministic protocol specifies:Function of board contents:if the protocol is overif YES, the outputif NO, which player writes nextFunction of board contents and input available to player P:what P writesCost of = max number of bits written on the board over all inputsRandomized ProtocolsIn a randomized protocol , what player P writes is also a function of the (private and/or public) random string available to PProtocol allowed to err with probability over choice of random stringsThe cost of = max number of bits written on the board, over inputs and random stringsCommunication ComplexityFocus on randomized communication complexity: CC(F,) = the communication cost of computing F with error .A distributional flavor of randomized communication complexity: CC(F,,) = the communication cost of computing F with error with respect to .Yaos minimax: CC(F,)=max CC(F,,).5Stunning variety of applications of CC Lower BoundsLower Bounds for Streaming AlgorithmsData Structure Lower BoundsProof Complexity Lower BoundsGame TheoryCircuit Complexity Lower BoundsQuantum ComputationDifferential Privacy2-Party Information Complexity2-party communication: each party has a dataset. Goal is to compute a function f(DA,DB)m1m2m3mk-1mk
DAx1x2xnDBy1y2ymf(DA,DB)f(DA,DB)Information complexity of a protocol for f is the amount of information the players reveal to each other / or to an eavesdropper (Eve)Information Complexity[Chakrabarti,Shi,Wirth,Yao 01], [Bar-Yossef,Jayram,Kumar,Sivakumar 04]Entropy: H(X) = x p(x) log (1/p(x)Conditional entropy: H(X|Y) = y H(X|Y=y) p(Y=y)Mutual Information: I(X;Y) = H(X) - H(X|Y)
External IC: information about XY revealed to EveICext (,) = I(XY;) ICext (f,,) = max ICext(,)Internal IC: information revealed to Alice and Bob ICint (,) = I(X;|Y) + I(Y;|X)ICint (f,,) = max ICint (,)
CSWY defined external IC.Entropy: avg amt of bits to send an element from distribMutual info: how much information y reveals about x (on average)External IC: avg amount of info protocol reveals about XYInternal IC: avg amot of bits of info protocol reveals to Alice about y and to Bob about xChakrabarti-Shi-Wirth-Yao, Bar-Yossef-Jayram-Kumar-Sivakumar,Barak-Braverman-Chen-Rao8Why study information complexity?
Intrinsically interesting quantityRelated to longstanding questions in complexity theory (direct sum conjecture)Very useful when studying privacy, and quantum computation
Chakrabarti, Sen, Wirth, YaoJain Radhakrishnan, SenAnd then later paper does constant round case: Jain-et-al9Simple Facts about Information ComplexityExternal information cost is greater than internal: ICext(,) ICint (,) ICext() = I(XY;) = I(X;) + I(Y; | X) I(X;|Y) + I(Y; | X)= ICint ()
Information complexity lower bounds imply Communication Complexity lower bounds:CC(f,,) ICext(f,,) ICint (f,,)
Eve stands to learn a lot more since she knew nothing to begin withIC(XY; pi) = IC(X; pi) + IC(Y; pi,X) 10Do CC Lower Bounds imply IC Lower Bounds? (i.e., CC=IC?)
For constant-round protocols, IC and CC are basically equal [CSWY, JRS]Open for general protocols. Significant step for general case by [BBCR]
Chakrabarti, Sen, Wirth, YaoJain Radhakrishnan, SenAnd then later paper does constant round case: Jain-et-al11Compressing Interactive Communication[Barak,Braverman,Chen,Rao]Theorem 1For any distribution , any C-bit protocol of internal IC I can be simulated by a new protocol using O((CI) logC) bits.
Theorem 2For any product distribution , any C-bit protocol of internal IC I can be simulated by a protocol using O(I logC) bits.Barak, Braverman, Chen, RaoExplain main idea of Theorem 1. Each computes probability distribution over transcripts conditioned on what they know. Because IC is low, there arent many places where they differ by a lot. Find first place where they differ by a lot and correct this spot, continue.12Connection to the Direct Sum ProblemDoes it take m times the amount of resources to solve m instances?
Direct Sum Question for CC: CC(fm) m CC(f) for every f and every distribution? - Each copy should have error For search problems, the direct sum problem is equivalent to separating NC1 from P !
Want this to be true for every distributuion mu and for every boolean function fAlso for randomized protocols, the error should be at most eps for each copy.This was already known for product distributions13Connection to the Direct Sum Problem, 2 The direct sum property holds for information complexity: Lemma [Direct Sum for IC]: IC(fm) m IC(f)
Best general direct sum theorem known for cc: Theorem [Barak,Braverman,Chen,Rao]: CC(fm) m CC(f) ignoring polylog factors
The direct sum property for cc is equivalent to IC=CC! Theorem [Braverman,Rao]: IC(f,,) = limn CC(Fn, n,)/n
Want this to be true for every distributuion mu and for every boolean function fAlso for randomized protocols, the error should be at most eps for each copy.This was already known for product distributions14Methods for Proving CC and IC Lower BoundsJain and Klauck initiated the formal study of CC lower bound methods: all formalizable as solutions to (different) LPs Discrepancy Method, Smooth Discrepancy MethodRectangle Bound, Smooth Rectangle BoundPartition Bound
The Partition Bound [Jain, Klauck]Min z,R wz,R (x,y) R, (x,y) R wf(x,y),R 1- (x,y) R, (x,y) in R z wz,R = 1 z,R w z,R 0
Relaxed version: second line is n/2 + n 0 if HD(x,y) < n/2 n
Theorem. Any -DP protocol for Hamming distance must incur an additive error (n).Note: This lower bound is tight.Proof sketch: [Chakrabarti-Regev 2012] prove: CC(GHD,,1/3) = (n). Proof shows GHD has a smooth rectangle bound of 2(n). By Jain-Klauck, this implies that the partition bound for GHD is at least as large. Thus proof follows by DP Partition Theorem.
29Implications of Lower bound for Hamming Distance1. Separation between -DP protocols and computational -DP protocols [MPRV]:Hamming distance has an O(1) error computational -DP protocol, but any -DP protocol has error n. We also exhibit another function with linear separation. (Any -DP protocol has error n)
2. Pan Privacy: Our lower bound for Hamming Distance implies lower bounds for pan-private streaming algorithmsRelaxed notion of privacy: now prob over the transcripts for neighboring x,x is eps-indistinguishable to a polytime alg.Via fully homomorphic encryption, any low sensitivity f(x,y) has a O(1) error computational eps-DP protocol, including Hamming dist.
Important (although negative result) for game theoretic (mech design) applications. Example: vickrey auctions. Do we really want to trust a third party30Pan-Private Streaming Model [Dwork,P,Rothblum, Naor,Yekhanin]Data is a stream of items; each item belongs to a user. Sanitizer sees each item and updates internal state. Generates output at end of the stream (single pass). statePan-Privacy: For every two adjacent streams, at any single point in time, the internal state (and final output) are differentially private. Radical idea: data arriving continuously. Want internal state to be eps-DP!! Adjacent: can remove any user (even if interleaved)31What statistics have pan-private algorithms?We give pan-private streaming algorithms for:Stream density / number of distinct elementst-cropped mean: mean, over users, of min(t, #appearances)Fraction of users appearing exactly k times Fraction of users appearing exactly 0 times modulo k Fraction of heavy-hitters, users appearing at least k times Pan Privacy lower bounds via -DP lower boundsLower Bounds for -DP communication protocols imply pan privacy lower bounds for density estimation (via Hamming distance lower bound).Lower bounds also hold for multi-pass pan-private modelsAnalogy: 2-party communication complexity lower bounds imply lower bounds in streaming model.DP Protocols and CompressionSo back to Ones(x,y) and HD(x,y)...is DP the same as compressible?
Theorem. [BBCR] (Low Icost implies compression) For every product distribution , and protocol P, there exists a protocol Q (-approximating P) with comm. complexity Icost(P) x polylog(CC(P))/
Corollary. (DP protocols can be compressed) Let P be an -DP protocol P. Then there exists a protocol Q of cost 3n polylog(CC(P))/ and error .
DP almost implies low cc, except for this annoying polylog(CC(P)) factorMoreover, the low cc protocol can often be made DP (if the number of rounds is bounded.)
Differential Privacy andCompressionWe have seen that DP protocols have low information costBy BBCR this implies they can be compressed (and thus have low comm complexity)
What about the other direction? Can functions with low cc be made DP? Yes! (with some caveats..the error is proportional not only to the cc, but also the number of rounds.)Proof uses the exponential mechanism [MT] Applications of Information ComplexityDifferential Privacy
PAR37Approximate Privacy in Mechanism DesignTraditional goal of mechanism design: Incent agents to reveal private information that is needed to compute optimal results.