computing distributions using random walks on graphs guy kindler guy kindler dimacs dan romik dan...
Post on 19-Dec-2015
214 views
TRANSCRIPT
Computing Computing
Distributions using Distributions using
Random Walks on Random Walks on
GraphsGraphs
Computing Computing
Distributions using Distributions using
Random Walks on Random Walks on
GraphsGraphsGuy KindlerGuy Kindler
DIMACSDIMACS
Dan RomikDan Romik
Weizmann Institute Weizmann Institute
of Scienceof Science
Computing distributionsComputing distributionsComputing distributionsComputing distributions
[Knuth, Yao 76][Knuth, Yao 76]
Given a Given a source source of random bits,of random bits,
output a sample with givenoutput a sample with given distribution distribution DD. .
11
FormalizationFormalizationFormalizationFormalization
Example:Example: ComputeCompute D={(“0”,1/2), (“1”,1/4), D={(“0”,1/2), (“1”,1/4),
(“2”,1/4)}(“2”,1/4)} “H”=right “T”=left
1122“1”“1”
00Interpretation:Interpretation:
Computing a distribution using a Computing a distribution using a
random walk on a binary treerandom walk on a binary tree..
Infinite trees are sometimes Infinite trees are sometimes neededneeded
Infinite trees are sometimes Infinite trees are sometimes neededneeded
D={(“0”,2/3), (“1”,1/3)}D={(“0”,2/3), (“1”,1/3)}
00
00
11
o Requirement:Requirement:
Output is reached with Output is reached with
probability probability 11
o [Knuth, Yao 76][Knuth, Yao 76] Output can be Output can be
reached in expected timereached in expected time
Ent(D)+O(1)Ent(D)+O(1)
Tight!Tight!Tight!Tight!
Some other modelsSome other modelsSome other modelsSome other models
o [Romik ’99][Romik ’99] Generate dist. Generate dist. BB from dist. from dist. AA in optimal in optimal time.time.
o [von Neumann ’51][von Neumann ’51] Generate unbiased coins from Generate unbiased coins from biased ones (when bias is unknown).biased ones (when bias is unknown).
o [Keane & O’Brien ’94][Keane & O’Brien ’94] Generate Generate f(p)f(p)-biased coins -biased coins from from pp-biased ones.-biased ones.
o [Peres & Nacu ’03][Peres & Nacu ’03] Generate Generate f(p)f(p)-biased in “good -biased in “good time”.time”.
o [Mossel & Peres ’03][Mossel & Peres ’03] Generate Generate f(p)f(p)-biased coins -biased coins from from pp-biased, using a finite graph.-biased, using a finite graph.
Finite state generatorsFinite state generatorsFinite state generatorsFinite state generators
“0”“0”
“1”“1” “0”“0”
“1”“1”
“0”“0”
Output:Output:110…
[Knuth+Yao]:[Knuth+Yao]:
Finite state generatorsFinite state generatorsFinite state generatorsFinite state generators
“0”“0”
“1”“1” “0”“0”
“1”“1”
“0”“0”
o Interpretation – binary representation:Interpretation – binary representation:
Generating a random variable Generating a random variable
on on [0,1][0,1] using a using a random walk random walk
on a graphon a graph
o Definition:Definition: A distribution function A distribution function
is computable, if it is the is computable, if it is the
output distribution of some f.s.g.output distribution of some f.s.g.
o Question [Knuth+Yao]:Question [Knuth+Yao]: which which
distributions are computable?distributions are computable?smooth/analyticsmooth/analytic
Output:Output:110…
History of the problemHistory of the problemHistory of the problemHistory of the problem
o [Knuth+Yao ’76][Knuth+Yao ’76] Computable analytic density Computable analytic density
functions must be polynomials with rational functions must be polynomials with rational
coefficientscoefficients
o [Yao ’84] [Yao ’84] The roots of such functions must be The roots of such functions must be
rationalrationalo [this work] [this work]
1.1. All functions with above properties can be All functions with above properties can be
computedcomputed
2.2. Allowing smooth functions does not add Allowing smooth functions does not add
computable functions.computable functions.
We’ll discuss…We’ll discuss…We’ll discuss…We’ll discuss…
[Theorem][Theorem] Let Let DD be a be a
distribution with density distribution with density
function function ff. If. If
o ff is a non-negative is a non-negative
polynomiapolynomiall
o with rational coefficientswith rational coefficients
o and no irrational roots in and no irrational roots in
[0,1][0,1],,
then then DD is computable. is computable.
“0”“0”
“1”“1” “0”“0”
“1”“1”
“0”“0”
Generating some Generating some distributionsdistributions
Generating some Generating some distributionsdistributions
uniform distribution:uniform distribution:
Generating Generating max(X,Y)max(X,Y)::
o run two f.s.g’s “in run two f.s.g’s “in
parallel”parallel”
o output the maximumoutput the maximum
“0”“0”“1”“1”o All distributions with All distributions with
density of the form:density of the form:
f(x)=c xf(x)=c xmm(1-x)(1-x)nn
[Knuth+Yao ’76][Knuth+Yao ’76]
o All order statistics All order statistics
of independent of independent
uniform variablesuniform variables
More distributionsMore distributionsMore distributionsMore distributions
[Knuth+Yao ’76]:[Knuth+Yao ’76]:
uniform on uniform on [a,b][a,b], for , for aa,,bb
rational.rational.
All distributions with All distributions with
density of the form:density of the form:
f(x)=c (x-a)f(x)=c (x-a)mm(b-(b-
x)x)nn11[a,b][a,b](x)(x)Generating Generating max(X,Y)max(X,Y)::
o run two f.s.g’s “in run two f.s.g’s “in
parallel”parallel”
o output the maximumoutput the maximum
o easy:easy: if if ff1,..,1,..,ffkk are are
computable, then so iscomputable, then so is
aa11ff11+…a+…akkffkk
(for (for aaii rational) rational)
All distributionsAll distributionsAll distributionsAll distributions
Answer:Answer: all polynomialsall polynomialswith rational with rational coefficients, and no coefficients, and no irrational roots in irrational roots in
[0,1][0,1] !!
Answer:Answer: all polynomialsall polynomialswith rational with rational coefficients, and no coefficients, and no irrational roots in irrational roots in
[0,1][0,1] !!
All distributions with All distributions with
density of the form:density of the form:
f(x)=c (x-a)f(x)=c (x-a)mm(b-(b-
x)x)nn11[a,b][a,b](x)(x)
Question:Question: what is the what is the set of rational set of rational mixtures of such mixtures of such functions ?functions ?
Question:Question: what is the what is the set of rational set of rational mixtures of such mixtures of such functions ?functions ?
Proof:Proof:
1.1. Geometric in natureGeometric in nature
2.2. Non-constructiveNon-constructive
Proof:Proof:
1.1. Geometric in natureGeometric in nature
2.2. Non-constructiveNon-constructive
Q.E.D. !
Q.E.D. !
Q.E.D. !
Q.E.D. !
ConclusionsConclusionsConclusionsConclusions
o We solved the computability problem in the f.s.g. We solved the computability problem in the f.s.g. model, for smooth functions.model, for smooth functions.
o We have no good bounds on complexity (size of We have no good bounds on complexity (size of graph) in this model.graph) in this model.
Open problemsOpen problemsOpen problemsOpen problemso Solve for other computational models (stack Solve for other computational models (stack
automaton? automaton? [Yao84][Yao84]))o Solve the general computablity question (no Solve the general computablity question (no
smoothness restriction) smoothness restriction) o Solve the complexity questionSolve the complexity question
The End…The End…
The End…The End…