Data-Powered AlgorithmsData-Powered AlgorithmsData-Powered AlgorithmsData-Powered Algorithms

Bernard ChazelleBernard Chazelle

Princeton UniversityPrinceton University

Bernard ChazelleBernard Chazelle

Princeton UniversityPrinceton University

Linear ProgrammingLinear Programming Linear ProgrammingLinear Programming

N constraints and d variablesN constraints and d variables

N constraints and d variablesN constraints and d variables

Dimension ReductionDimension Reduction

1000010000 2525Images (face recognition)Images (face recognition) Signals (voice recognition)Signals (voice recognition)Text (NLP)Text (NLP). . . . . .

Nearest neighbor searchingNearest neighbor searchingClusteringClustering. . .. . .

Dimension reductionDimension reduction

All pairwise distances nearly preserved

Johnson-Lindenstrauss Transform (JLT)

c log nc log n22

dd

Random OrthogonalMatrix

vv dd

Friendly JLTFriendly JLT

c log nc log n22

dd

N(0,1)N(0,1) N(0,1)N(0,1) N(0,1)N(0,1)N(0,1)N(0,1)N(0,1)N(0,1)N(0,1)N(0,1) N(0,1)N(0,1)

N(0,1)N(0,1)

N(0,1)N(0,1) N(0,1)N(0,1) N(0,1)N(0,1)N(0,1)N(0,1)N(0,1)N(0,1)N(0,1)N(0,1) N(0,1)N(0,1)

N(0,1)N(0,1)

Friendlier JLTFriendlier JLT

c log nc log n22

dd

11++-- 11++-- 11++-- 11++--11++--11++--11++--11++--

11++--11++-- 11++--

11++-- 11++--11++-- 11++--

11++--

d log nd log n 22 = =

Sparse JLTSparse JLT? ?

c log nc log n22

11++--11++--11++--

11++-- 11++--11++--

11++--

00

00

00

00

00

00

0000

00

dd

11 dd

00

00

00

00

. .

..

. .

. .

..

. .

o(1)-Fraction non-o(1)-Fraction non-zeroszeros

Main Tool: Uncertainty Main Tool: Uncertainty PrinciplePrinciple

TimeTime

FrequencyFrequency

HeisenbergHeisenberg

Fast Johnson-Lindenstrauss Transform (FJLT)Fast Johnson-Lindenstrauss Transform (FJLT)

1+- 1+- 1+-

1+-

dd

DiscreteFourier

Transform

dddd

c log nc log n22

. . .

0N(0,1)

= =OO+ d log d + d + d log d + d loglog33 n n 22

dd

OptimalOptimal?? ??

theory experimentation

computation

theory experimentation

computation

theory experimentation

inputinput outputoutput

Most interestingMost interestingproblems areproblems are

too hard !!too hard !!

Most interestingMost interestingproblems areproblems are

too hard !!too hard !!

inputinput outputoutput

randomizationrandomization

approximationapproximation

So, we change So, we change the model…the model…

So, we change So, we change the model…the model…

inputinput outputoutput

randomizationrandomization

approximationapproximationPTAS for ETSPPTAS for ETSPPTAS for ETSPPTAS for ETSP

inputinput outputoutput

randomizationrandomization

approximationapproximation

Impossible toImpossible toapproximateapproximate chromatic chromatic

number withinnumber withina factor of… a factor of…

Impossible toImpossible toapproximateapproximate chromatic chromatic

number withinnumber withina factor of… a factor of…

inputinput outputoutput

randomizationrandomization

approximationapproximationProperty Property TestingTesting

[RS’96, [RS’96, GGR’96]GGR’96]

Property Property TestingTesting

[RS’96, [RS’96, GGR’96]GGR’96]

Berkeley “school”Berkeley “school”(program checking &(program checking &probabilistic proofs)probabilistic proofs)

Berkeley “school”Berkeley “school”(program checking &(program checking &probabilistic proofs)probabilistic proofs)

Distance is 3Distance is 3Distance is 3Distance is 3

Distance is 4Distance is 4Distance is 4Distance is 4

nononono

yesyesyesyes

bipartitebipartitebipartitebipartite

nononono

yesyesyesyesbipartitebipartitebipartitebipartite

anythinganythinganythinganything

[GR’97][GR’97][GR’97][GR’97]

Birthday paradox Birthday paradox Birthday paradox Birthday paradox

62626262

181818187777

polylog cyclespolylog cyclespolylog cyclespolylog cycles

17171717

MixingMixing casecaseMixingMixing casecase

[M’89[M’89

]][M’89[M’89

]]Nonmixing implies small cutsNonmixing implies small cutsNonmixing implies small cutsNonmixing implies small cuts

Non-mixingNon-mixing casecaseNon-mixingNon-mixing casecase

Dense graphsDense graphsDense graphsDense graphs

[GGR98, AK99][GGR98, AK99][GGR98, AK99][GGR98, AK99]

Hofstadter. Godel, Escher, Bach.

Is graph k-colorable?Is graph k-colorable?Is graph k-colorable?Is graph k-colorable?

1010001

0101011

1101100

1010011

1101101

0010110

1011001

Main Main tooltoolMain Main tooltool

Szemerédi’s Regularity Lemma Szemerédi’s Regularity Lemma Szemerédi’s Regularity Lemma Szemerédi’s Regularity Lemma

Far from k-colorableFar from k-colorableFar from k-colorableFar from k-colorable

Lots of Lots of witnesseswitnesses

Lots of Lots of witnesseswitnesses

Property Testing

Graph algorithms connectivity acyclicity k-way cuts clique

Distributions independence entropy monotonicity distances

Geometry convexity disjointness delaunay plane EMST

http://www.cs.princeton.edu/http://www.cs.princeton.edu/~chazelle/~chazelle/

http://www.cs.princeton.edu/http://www.cs.princeton.edu/~chazelle/~chazelle/