simple and deterministic matrix sketches[a simple algorithm for nding frequent elements in streams...
TRANSCRIPT
![Page 1: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/1.jpg)
Simple and Deterministic Matrix Sketches
Edo Liberty
Edo Liberty: Simple and Deterministic Matrix Sketches 1 / 38
![Page 2: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/2.jpg)
Data Matrices
Often our data is represented by a matrix.
Edo Liberty: Simple and Deterministic Matrix Sketches 2 / 38
![Page 3: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/3.jpg)
Data Matrices
But our data matrix is typically too large to work with on a single machine.
Data Columns Rows d n sparseTextual Documents Words 105 - 107 > 1012 yes
Actions Users Types 101 - 104 > 108 yes
Visual Images Pixels, SIFT 106 - 107 > 109 no
Audio Songs, tracks Frequencies 106 - 107 > 109 no
MachineLearning
Examples Features 102 - 104 > 105 no
Financial Prices Items, Stocks 103 -105 > 106 no
We think of A ∈ Rd×n as n column vectors in Rd and typically n� d .
Edo Liberty: Simple and Deterministic Matrix Sketches 3 / 38
![Page 4: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/4.jpg)
Streaming Matrices
Sometimes, we cannot store the entire matrix at all.
Edo Liberty: Simple and Deterministic Matrix Sketches 4 / 38
![Page 5: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/5.jpg)
Streaming Matrices
Example: can we compute the covariance matrix from the a stream?(enough for PCA for example).
AAT =n∑
i=1
AiATi
Naıve solution
Compute AAT in time O(nd2) and space O(d2).
Think about 1Mp images, d = 106. This solution requires 1012 operationsper update and 1T space.
Edo Liberty: Simple and Deterministic Matrix Sketches 5 / 38
![Page 6: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/6.jpg)
Streaming Matrices
Example: can we compute the covariance matrix from the a stream?(enough for PCA for example).
AAT =n∑
i=1
AiATi
Naıve solution
Compute AAT in time O(nd2) and space O(d2).
Think about 1Mp images, d = 106. This solution requires 1012 operationsper update and 1T space.
Edo Liberty: Simple and Deterministic Matrix Sketches 5 / 38
![Page 7: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/7.jpg)
Matrix Approximation
Matrix sketching or approximation
Efficiently compute a concisely representable matrix B such that
B ≈ A or BBT ≈ AAT
Working with B instead of A is often “good enough”.
Dimension reduction
Signal denoising
Classification
Regression
Clustering
Approximate matrix multiplication
Reconstruction
Recommendation
. . .Edo Liberty: Simple and Deterministic Matrix Sketches 6 / 38
![Page 8: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/8.jpg)
Matrix ApproximationColumn subset selection algorithms
Paper Space Time Bound
FKV04 O(k4/ε6 max(k4, ε−2)) O(k5/ε6 max(k4, ε−2)) P, εA
DV06 #C = O(k/ε + k2 log k)
O(n(k/ε + k2 log k))
O(nnz(A)(k/ε + k2 log k)+
(n+ d)(k2/ε2 + k3 log(k/ε) + k4 log2 k))
P, εR
DKM06“LinearTimeSVD”
#C = O(1/ε2)
O((n + 1/ε2)/ε4)
O((n + 1/ε2)/ε4 + nnz(A)) P, εL2
#C = O(k/ε2)
O((k/ε2)(n + k/ε2))
O((k/ε2)2(n + k/ε2) + nnz(A)) P, εA
DKM06“ConstantTimeSVD”
#C+R = O(1/ε4)
O(1/ε12 + nk/ε4)
O((1/ε12 + nk/ε4 + nnz(A)) P, εL2
#C+R = O(k2/ε4)
O(k6/ε12 + nk3/ε4)
O(k6/ε12 + nk3/ε4 + nnz(A)) P, εA
DMM08“CUR”
#C =O(k2/ε2)
#R = O(k4/ε6)
O(nd2) C, εR
MD09“ColumnSelect”
#C = O(k log k/ε2)
O(nk log k/ε2)
O(nd2) PO(k log k/ε2)
, εR
BDM11 #C = 2k/ε(1 + o(1)) O((ndk + dk3)ε−2/3) P2k/ε(1+o(1)), εR
[Relative Errors for Deterministic Low-Rank Matrix Approximations, Ghashami, Phillips 2013]
Sparsification and entry samplingPaper Space Time Bound
AM07 ρn/ε2 + n · polylog(n) nnz ρn/ε2 + nnz n · polylog(n) ‖A− B‖2 ≤ ε‖A‖2
AHK06 (nnz · n/ε2)1/2 nnz(nnz · n/ε2)1/2 ‖A− B‖2 ≤ ε‖A‖2
DZ11 ρn log(n)/ε2 nnz ρn log(n)/ε2 ‖A− B‖2 ≤ ε‖A‖2
AKL13 n ρ log(n)/ε2 +
(ρ log(n) nnz /ε2)1/2nnz ‖A− B‖2 ≤ ε‖A‖2
[Near-optimal Distributions for Data Matrix Sampling, Achlioptas, Karnin, Liberty, 2013]
Edo Liberty: Simple and Deterministic Matrix Sketches 7 / 38
![Page 9: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/9.jpg)
Matrix Approximation
Linear subspace embedding sketchesPaper Space Time Bound
DKM06LinearTimeSVD
#R = O(1/ε2)
O((d + 1/ε2)/ε4)
O((d + 1/ε2)/ε4 + nnz(A)) P, εL2
#R = O(k/ε2)
O((k/ε2)2(d + k/ε2))
O((k/ε2)2(d + k/ε2) + nnz(A)) P, εA
Sar06turnstile
#R = O(k/ε + k log k)O(d(k/ε + k log k))
O(nnz(A)(k/ε + k log k) + d(k/ε +
k log k)2))
PO(k/ε+k log k), εR
CW09 #R = O(k/ε) O(nd2 + (ndk/ε)) PO(k/ε), εR
CW09 O((n + d)(k/ε)) O(nd2 + (ndk/ε)) C, εR
CW09 O((k/ε2)(n + d/ε2)) O(n(k/ε2)2 + nd(k/ε2) + nd2) C, εR
Deterministic sketching algorithmsPaper Space Time Bound
FSS13 O((k/ε) log n) n((k/ε) log n)O(1) P2dk/εe, εR
Lib13 #R = O(ρ/ε)O(dρ/ε)
O(ndρ/ε) PO(ρ/ε), εL2
GP13 #R = dk/ε + keO(dk/ε)
O(ndk/ε) P, εR
[Relative Errors for Deterministic Low-Rank Matrix Approximations, Ghashami, Phillips 2013]
Edo Liberty: Simple and Deterministic Matrix Sketches 8 / 38
![Page 10: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/10.jpg)
Frequent Directions
Goal:
Efficiently maintain a matrix B with only ` = 2/ε columns s.t.
‖AAT − BBT‖2 ≤ ε‖A‖2f
Intuition:
Extend Frequent-items
[Finding repeated elements, Misra, Gries, 1982.]
[Frequency estimation of internet packet streams with limited space, Demaine, Lopez-Ortiz, Munro, 2002]
[A simple algorithm for finding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003]
[Efficient Computation of Frequent and Top-k Elements in Data Streams, Metwally, Agrawal, Abbadi, 2006]
(An algorithm so good it was invented 4 times.)
Edo Liberty: Simple and Deterministic Matrix Sketches 9 / 38
![Page 11: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/11.jpg)
Frequent Items
Obtain the frequency f (i) of each item in the stream of itemsEdo Liberty: Simple and Deterministic Matrix Sketches 10 / 38
![Page 12: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/12.jpg)
Frequent Items
With d counters it’s easy but not good enough (IP addresses, queries....)Edo Liberty: Simple and Deterministic Matrix Sketches 11 / 38
![Page 13: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/13.jpg)
Frequent Items
(Misra-Gries) Lets keep less than a fixed number of counters `.Edo Liberty: Simple and Deterministic Matrix Sketches 12 / 38
![Page 14: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/14.jpg)
Frequent Items
If an item has a counter we add 1 to that counter.Edo Liberty: Simple and Deterministic Matrix Sketches 13 / 38
![Page 15: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/15.jpg)
Frequent Items
Otherwise, we create a new counter for it and set it to 1Edo Liberty: Simple and Deterministic Matrix Sketches 14 / 38
![Page 16: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/16.jpg)
Frequent Items
But now we do not have less than ` counters.Edo Liberty: Simple and Deterministic Matrix Sketches 15 / 38
![Page 17: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/17.jpg)
Frequent Items
Let δ be the median counter value at time tEdo Liberty: Simple and Deterministic Matrix Sketches 16 / 38
![Page 18: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/18.jpg)
Frequent Items
Decrease all counters by δ (or set to zero if less than δ)Edo Liberty: Simple and Deterministic Matrix Sketches 17 / 38
![Page 19: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/19.jpg)
Frequent Items
And continue...Edo Liberty: Simple and Deterministic Matrix Sketches 18 / 38
![Page 20: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/20.jpg)
Frequent Items
The approximated counts are f ′
Edo Liberty: Simple and Deterministic Matrix Sketches 19 / 38
![Page 21: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/21.jpg)
Frequent Items
We increase the count by only 1 for each item appearance.
f ′(i) ≤ f (i)
Because we decrease each counter by at most δt at time t
f ′(i) ≥ f (i)−∑t
δt
Calculating the total approximated frequencies:
0 ≤∑i
f ′(i) ≤∑t
1− (`/2) · δt = n − (`/2) ·∑t
δt
∑t
δt ≤ 2n/`
Setting ` = 2/ε yields
|f (i)− f ′(i)| ≤ εn
Edo Liberty: Simple and Deterministic Matrix Sketches 20 / 38
![Page 22: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/22.jpg)
Frequent Directions
We keep a sketch of at most ` columns
Edo Liberty: Simple and Deterministic Matrix Sketches 21 / 38
![Page 23: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/23.jpg)
Frequent Directions
We maintain the invariant that some columns are empty (zero valued)
Edo Liberty: Simple and Deterministic Matrix Sketches 22 / 38
![Page 24: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/24.jpg)
Frequent Directions
Input vectors are simply stored in empty columns
Edo Liberty: Simple and Deterministic Matrix Sketches 23 / 38
![Page 25: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/25.jpg)
Frequent Directions
Input vectors are simply stored in empty columns
Edo Liberty: Simple and Deterministic Matrix Sketches 24 / 38
![Page 26: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/26.jpg)
Frequent Directions
When the sketch is ‘full’ we need to zero out some columns...
Edo Liberty: Simple and Deterministic Matrix Sketches 25 / 38
![Page 27: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/27.jpg)
Frequent Directions
Using the SVD we compute B = USV T and set Bnew = US
Edo Liberty: Simple and Deterministic Matrix Sketches 26 / 38
![Page 28: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/28.jpg)
Frequent Directions
Note that BBT = BnewBTnew so we don’t “lose” anything
Edo Liberty: Simple and Deterministic Matrix Sketches 27 / 38
![Page 29: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/29.jpg)
Frequent Directions
The columns of B are now orthogonal and in decreasing magnitude order
Edo Liberty: Simple and Deterministic Matrix Sketches 28 / 38
![Page 30: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/30.jpg)
Frequent Directions
Let δ = ‖B`/2‖2
Edo Liberty: Simple and Deterministic Matrix Sketches 29 / 38
![Page 31: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/31.jpg)
Frequent Directions
Reduce column `22-norms by δ (or nullify if less than δ)
Edo Liberty: Simple and Deterministic Matrix Sketches 30 / 38
![Page 32: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/32.jpg)
Frequent Directions
Start aggregating columns again...
Edo Liberty: Simple and Deterministic Matrix Sketches 31 / 38
![Page 33: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/33.jpg)
Frequent Directions
Input: `, A ∈ Rd×n
B ← all zeros matrix ∈ Rd×`
for i ∈ [n] doInsert Ai into a zero valued column of Bif B has no zero valued colums then
[U,Σ,V ]← SVD(B)δ ← σ2
`/2
Σ←√
max(Σ2 − I`δ, 0)B ← UΣ # At least half the columns of B are zero.
Return: B
Edo Liberty: Simple and Deterministic Matrix Sketches 32 / 38
![Page 34: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/34.jpg)
Bounding the error
We first bound ‖AAT − BBT‖
sup‖x‖=1
‖xA‖2 − ‖xB‖2 = sup‖x‖=1
n∑t=1
[〈x ,At〉2 + ‖xBt−1‖2 − ‖xBt‖2]
= sup‖x‖=1
n∑t=1
[‖xC t‖2 − ‖xBt‖2]
≤n∑
t=1
‖C tTC t − BtTBt‖ · ‖x‖2
=n∑
t=1
δt
Which gives:
‖AAT − BBT‖ ≤n∑
t=1
δt
Edo Liberty: Simple and Deterministic Matrix Sketches 33 / 38
![Page 35: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/35.jpg)
Bounding the error
We compute the Frobenius norm of the final sketch.
0 ≤ ‖B‖2f =
n∑t=1
[‖Bt‖2f − ‖Bt−1‖2
f ]
=n∑
t=1
[(‖C t‖2f − ‖Bt−1‖2
f )− (‖C t‖2f − ‖Bt‖2
f )]
=n∑
t=1
‖At‖2 − tr(C tTC t − BtTBt)
≤ ‖A‖2f − (`/2)
n∑t=1
δt
Which gives:n∑
t=1
δt ≤ 2‖A‖2f /`
Edo Liberty: Simple and Deterministic Matrix Sketches 34 / 38
![Page 36: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/36.jpg)
Bounding the error
We saw that:
‖AAT − BBT‖ ≤n∑
t=1
δt
and that:n∑
t=1
δt ≤ 2‖A‖2f /`
Setting ` = 2/ε yields
‖AAT − BBT‖ ≤ ε‖A‖2f .
The two proofs are (maybe unsurprisingly) very similar...
Edo Liberty: Simple and Deterministic Matrix Sketches 35 / 38
![Page 37: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/37.jpg)
Experiments
‖AAT − BBT‖ as a function of the sketch size `
0
2,000
4,000
6,000
8,000
10,000
12,000
14,000
16,000
18,000
20,000
10
20
30
40
50
60
70
80
90
100
110
120
130
140
150
160
170
180
190
200
Sketch accuracy
Number of rows in sketch
Naive Sampling Hashing Random ProjecBons Frequent DirecBons Bound Frequent DirecBons Brute Force
Synthetic input matrix with linearly decaying singular values.
Edo Liberty: Simple and Deterministic Matrix Sketches 36 / 38
![Page 38: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/38.jpg)
Experiments
Running time in second as a function of n (x-axis) and d (y-axis)
0
20
40
60
80
100
120
140
160
180
200
10,000
20,000
30,000
40,000
50,000
60,000
70,000
80,000
90,000
100,000
Runn
ing 'm
e in se
cond
s
Number of rows in matrix
10000 9000 8000 7000 6000 5000 4000 3000 2000 1000
The running time scales linearly in n, d and ` as expected.Edo Liberty: Simple and Deterministic Matrix Sketches 37 / 38
![Page 39: Simple and Deterministic Matrix Sketches[A simple algorithm for nding frequent elements in streams and bags, Karp, Shenker, Papadimitriou, 2003] ... (An algorithm so good it was invented](https://reader036.vdocuments.site/reader036/viewer/2022081404/5f0438bb7e708231d40ce97c/html5/thumbnails/39.jpg)
Thanks
Edo Liberty: Simple and Deterministic Matrix Sketches 38 / 38