simple and practical algorithm for the sparse fourier ...hashes correctly to one bucket, leaks to at...
TRANSCRIPT
![Page 1: Simple and Practical Algorithm for the Sparse Fourier ...Hashes correctly to one bucket, leaks to at most 1 bucket. Replace Gaussians with “Dolph-Chebyshev window functions”: factor](https://reader034.vdocuments.site/reader034/viewer/2022052012/6027da162e136a34dc33351e/html5/thumbnails/1.jpg)
Simple and Practical Algorithm for the SparseFourier Transform
Haitham Hassanieh Piotr Indyk Dina Katabi Eric Price
MIT
2012-01-19
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 1 / 19
![Page 2: Simple and Practical Algorithm for the Sparse Fourier ...Hashes correctly to one bucket, leaks to at most 1 bucket. Replace Gaussians with “Dolph-Chebyshev window functions”: factor](https://reader034.vdocuments.site/reader034/viewer/2022052012/6027da162e136a34dc33351e/html5/thumbnails/2.jpg)
Outline
1 Introduction
2 Algorithm
3 Experiments
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 2 / 19
![Page 3: Simple and Practical Algorithm for the Sparse Fourier ...Hashes correctly to one bucket, leaks to at most 1 bucket. Replace Gaussians with “Dolph-Chebyshev window functions”: factor](https://reader034.vdocuments.site/reader034/viewer/2022052012/6027da162e136a34dc33351e/html5/thumbnails/3.jpg)
The Dicrete Fourier Transform
Discrete Fourier transform: given x ∈ Cn, find
xi =∑
xjωij
Fundamental toolI Compression (audio, image, video)I Signal processingI Data analysisI ...
FFT: O(n log n) time.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 4 / 19
![Page 4: Simple and Practical Algorithm for the Sparse Fourier ...Hashes correctly to one bucket, leaks to at most 1 bucket. Replace Gaussians with “Dolph-Chebyshev window functions”: factor](https://reader034.vdocuments.site/reader034/viewer/2022052012/6027da162e136a34dc33351e/html5/thumbnails/4.jpg)
Sparse Fourier Transform
Often the Fourier transform is dominated by a small number of“peaks”
I Precisely the reason to use for compression.
If most of mass in k locations, can we compute FFT faster?
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 5 / 19
![Page 5: Simple and Practical Algorithm for the Sparse Fourier ...Hashes correctly to one bucket, leaks to at most 1 bucket. Replace Gaussians with “Dolph-Chebyshev window functions”: factor](https://reader034.vdocuments.site/reader034/viewer/2022052012/6027da162e136a34dc33351e/html5/thumbnails/5.jpg)
Previous work
Boolean cube: [KM92], [GL89]. What about C?[Mansour-92]: kc logc n.Long list of other work [GGIMS02, AGS03, Iwen10, Aka10]Fastest is [Gilbert-Muthukrishnan-Strauss-05]: k log4 n.
I All have poor constants, many logs.I Need n/k > 40,000 or ω(log3 n) to beat FFTW.I Our goal: beat FFTW for smaller n/k in theory and practice.I Result: n/k > 2,000 or ω(log n) to beat FFTW.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 6 / 19
![Page 6: Simple and Practical Algorithm for the Sparse Fourier ...Hashes correctly to one bucket, leaks to at most 1 bucket. Replace Gaussians with “Dolph-Chebyshev window functions”: factor](https://reader034.vdocuments.site/reader034/viewer/2022052012/6027da162e136a34dc33351e/html5/thumbnails/6.jpg)
Our result
Simple, practical algorithm with good constants.Compute the k -sparse Fourier transform in O(
√kn log3/2 n) time.
Get x ′ with approximation error
‖x ′ − x‖2∞ ≤1k‖x − xk‖22
If x is sparse, recover it exactly.Caveats:
I Additional ‖x‖2/nΘ(1) error.I n must be a power of 2.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 7 / 19
![Page 7: Simple and Practical Algorithm for the Sparse Fourier ...Hashes correctly to one bucket, leaks to at most 1 bucket. Replace Gaussians with “Dolph-Chebyshev window functions”: factor](https://reader034.vdocuments.site/reader034/viewer/2022052012/6027da162e136a34dc33351e/html5/thumbnails/7.jpg)
Structure of this section
If x is k -sparse with known support S, find xS exactly inO(k log2 n) time.
In general, estimate x approximately in O(√
nk) time.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 9 / 19
![Page 8: Simple and Practical Algorithm for the Sparse Fourier ...Hashes correctly to one bucket, leaks to at most 1 bucket. Replace Gaussians with “Dolph-Chebyshev window functions”: factor](https://reader034.vdocuments.site/reader034/viewer/2022052012/6027da162e136a34dc33351e/html5/thumbnails/8.jpg)
IntuitionOriginal signal (time) Original signal (freq)
Cutoff signal (time) Cutoff signal (freq)
Cutoff signal (time) Cutoff signal, subsampled (freq)
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 10 / 19
n-dimensional DFT
n-dimensional DFTof first B terms.
B-dimensional DFTof first B terms.
![Page 9: Simple and Practical Algorithm for the Sparse Fourier ...Hashes correctly to one bucket, leaks to at most 1 bucket. Replace Gaussians with “Dolph-Chebyshev window functions”: factor](https://reader034.vdocuments.site/reader034/viewer/2022052012/6027da162e136a34dc33351e/html5/thumbnails/9.jpg)
FrameworkCutoff signal (time) Cutoff subsampled signals (freq)
“Hashes” into B buckets in B log B time.Issues:
I “Hashing” needs a random hash functionF Access x ′t = ω−atxσt , so x ′t = xσ−1t+a [GMS-05]
I CollisionsF Have B > 4k , repeat O(log n) times and take median. [Count-Sketch,
CCF02]I LeakageI Finding the support. [Porat-Strauss-12], talk at 9:45am.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 11 / 19
![Page 10: Simple and Practical Algorithm for the Sparse Fourier ...Hashes correctly to one bucket, leaks to at most 1 bucket. Replace Gaussians with “Dolph-Chebyshev window functions”: factor](https://reader034.vdocuments.site/reader034/viewer/2022052012/6027da162e136a34dc33351e/html5/thumbnails/10.jpg)
LeakageCutoff signal (time) Cutoff, subsampled signals (freq)
Let Fi =
{1 i < B0 otherwise
be the “boxcar” filter. (Used in
[GGIMS02,GMS05])Observe
DFT(F ·x ,B) = subsample(DFT(F ·x ,n),B) = subsample(F∗x ,B).
DFT F of boxcar filter is sinc, decays as 1/i .Need a better filter F !
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 12 / 19
![Page 11: Simple and Practical Algorithm for the Sparse Fourier ...Hashes correctly to one bucket, leaks to at most 1 bucket. Replace Gaussians with “Dolph-Chebyshev window functions”: factor](https://reader034.vdocuments.site/reader034/viewer/2022052012/6027da162e136a34dc33351e/html5/thumbnails/11.jpg)
Filters
0 20 40 60 80 1000.0
0.5
1.0
1.5
2.0 Filter (time)
Bin0
5
10
15
20
25 Filter (freq)
Observe subsample(F ∗ x ,B) in O(B log B) time.Needs for F :
I supp(F ) ∈ [0,B]I |F | < δ = 1/nΘ(1) except “near” 0.I F ≈ 1 over [−n/2B,n/2B].
Gaussians:I Standard deviation σ = B/
√log n
B ·√
log n
I DFT has σ = (n/B)√
log n
(n/B)/√
log n
I Nontrivial leakage into O(log n) buckets.I But likely trivial contribution to correct bucket.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 13 / 19
![Page 12: Simple and Practical Algorithm for the Sparse Fourier ...Hashes correctly to one bucket, leaks to at most 1 bucket. Replace Gaussians with “Dolph-Chebyshev window functions”: factor](https://reader034.vdocuments.site/reader034/viewer/2022052012/6027da162e136a34dc33351e/html5/thumbnails/12.jpg)
Filters
0 20 40 60 80 1000.0
0.2
0.4
0.6
0.8
1.0 Filter (time)
Bin0
2
4
6
8
10
12
14
16
18 Filter (freq)
Observe subsample(F ∗ x ,B) in O(B log B) time.Needs for F :
I supp(F ) ∈ [0,B]I |F | < δ = 1/nΘ(1) except “near” 0.I F ≈ 1 over [−n/2B,n/2B].
Gaussians:I Standard deviation σ = B/
√log n
B ·√
log n
I DFT has σ = (n/B)√
log n
(n/B)/√
log n
I Nontrivial leakage into O(log n) buckets.I But likely trivial contribution to correct bucket.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 13 / 19
![Page 13: Simple and Practical Algorithm for the Sparse Fourier ...Hashes correctly to one bucket, leaks to at most 1 bucket. Replace Gaussians with “Dolph-Chebyshev window functions”: factor](https://reader034.vdocuments.site/reader034/viewer/2022052012/6027da162e136a34dc33351e/html5/thumbnails/13.jpg)
Filters
0 50 100 150 2000.0
0.2
0.4
0.6
0.8
1.0 Filter (time)
Bin0
10
20
30
40
50
60
70
80 Filter (freq)
Observe subsample(F ∗ x ,B) in O(B log B) time.Needs for F :
I supp(F ) ∈ [0,B log n]I |F | < δ = 1/nΘ(1) except “near” 0.I F ≈ 1 over [−n/2B,n/2B].
Gaussians:I Standard deviation σ =�����B/
√log n B ·
√log n
I DFT has σ =������(n/B)
√log n (n/B)/
√log n
I Nontrivial leakage into 0 buckets.I But likely trivial contribution to correct bucket.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 13 / 19
![Page 14: Simple and Practical Algorithm for the Sparse Fourier ...Hashes correctly to one bucket, leaks to at most 1 bucket. Replace Gaussians with “Dolph-Chebyshev window functions”: factor](https://reader034.vdocuments.site/reader034/viewer/2022052012/6027da162e136a34dc33351e/html5/thumbnails/14.jpg)
Filters
0 50 100 150 2000.0
0.2
0.4
0.6
0.8
1.0 Filter (time)
Bin0
10
20
30
40
50
60
70
80 Filter (freq)
Let G be Gaussian with σ = B√
log nH be box-car filter of length n/B.
Use F = G ∗ H.I F = G · H, so supp(F ) ⊂ [0,B log n].I |F | < 1/nΘ(1) outside −n/B,n/B.I |F | = 1± 1/nΘ(1) within n/2B,n/B.
Hashes correctly to one bucket, leaks to at most 1 bucket.Replace Gaussians with “Dolph-Chebyshev window functions”:factor 2 improvement.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 14 / 19
![Page 15: Simple and Practical Algorithm for the Sparse Fourier ...Hashes correctly to one bucket, leaks to at most 1 bucket. Replace Gaussians with “Dolph-Chebyshev window functions”: factor](https://reader034.vdocuments.site/reader034/viewer/2022052012/6027da162e136a34dc33351e/html5/thumbnails/15.jpg)
Filters
0 50 100 150 2000.02
0.00
0.02
0.04
0.06
0.08
0.10 Filter (time)
Bin0.0
0.2
0.4
0.6
0.8
1.0
1.2 Filter (freq)
Let G be Gaussian with σ = B√
log nH be box-car filter of length n/B.
Use F = G ∗ H.I F = G · H, so supp(F ) ⊂ [0,B log n].I |F | < 1/nΘ(1) outside −n/B,n/B.I |F | = 1± 1/nΘ(1) within n/2B,n/B.
Hashes correctly to one bucket, leaks to at most 1 bucket.Replace Gaussians with “Dolph-Chebyshev window functions”:factor 2 improvement.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 14 / 19
![Page 16: Simple and Practical Algorithm for the Sparse Fourier ...Hashes correctly to one bucket, leaks to at most 1 bucket. Replace Gaussians with “Dolph-Chebyshev window functions”: factor](https://reader034.vdocuments.site/reader034/viewer/2022052012/6027da162e136a34dc33351e/html5/thumbnails/16.jpg)
Algorithm to estimate xS
For O(log n) different permutations of x , computesubsample(F ∗ x ,B).Estimate each xi as median of values it maps to.
To find S: choose all that map to the top 2k values.nk/B candidates to update at each iteration: total
(nkB
+ B log n) log n =√
nk log3/2 n
time.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 15 / 19
![Page 17: Simple and Practical Algorithm for the Sparse Fourier ...Hashes correctly to one bucket, leaks to at most 1 bucket. Replace Gaussians with “Dolph-Chebyshev window functions”: factor](https://reader034.vdocuments.site/reader034/viewer/2022052012/6027da162e136a34dc33351e/html5/thumbnails/17.jpg)
Algorithm to estimate xS
For O(log n) different permutations of x , computesubsample(F ∗ x ,B).Estimate each xi as median of values it maps to.
To find S: choose all that map to the top 2k values.nk/B candidates to update at each iteration: total
(nkB
+ B log n) log n =√
nk log3/2 n
time.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 15 / 19
![Page 18: Simple and Practical Algorithm for the Sparse Fourier ...Hashes correctly to one bucket, leaks to at most 1 bucket. Replace Gaussians with “Dolph-Chebyshev window functions”: factor](https://reader034.vdocuments.site/reader034/viewer/2022052012/6027da162e136a34dc33351e/html5/thumbnails/18.jpg)
Algorithm to estimate xS
For O(log n) different permutations of x , computesubsample(F ∗ x ,B).Estimate each xi as median of values it maps to.
To find S: choose all that map to the top 2k values.nk/B candidates to update at each iteration: total
(nkB
+ B log n) log n =√
nk log3/2 n
time.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 15 / 19
![Page 19: Simple and Practical Algorithm for the Sparse Fourier ...Hashes correctly to one bucket, leaks to at most 1 bucket. Replace Gaussians with “Dolph-Chebyshev window functions”: factor](https://reader034.vdocuments.site/reader034/viewer/2022052012/6027da162e136a34dc33351e/html5/thumbnails/19.jpg)
Algorithm to estimate xS
For O(log n) different permutations of x , computesubsample(F ∗ x ,B).Estimate each xi as median of values it maps to.
To find S: choose all that map to the top 2k values.nk/B candidates to update at each iteration: total
(nkB
+ B log n) log n =√
nk log3/2 n
time.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 15 / 19
![Page 20: Simple and Practical Algorithm for the Sparse Fourier ...Hashes correctly to one bucket, leaks to at most 1 bucket. Replace Gaussians with “Dolph-Chebyshev window functions”: factor](https://reader034.vdocuments.site/reader034/viewer/2022052012/6027da162e136a34dc33351e/html5/thumbnails/20.jpg)
Algorithm in general
For O(log n) different permutations of x , computesubsample(F ∗ x ,B).Estimate each xi as median of values it maps to.
To find S: choose all that map to the top 2k values.nk/B candidates to update at each iteration: total
(nkB
+ B log n) log n =√
nk log3/2 n
time.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 15 / 19
![Page 21: Simple and Practical Algorithm for the Sparse Fourier ...Hashes correctly to one bucket, leaks to at most 1 bucket. Replace Gaussians with “Dolph-Chebyshev window functions”: factor](https://reader034.vdocuments.site/reader034/viewer/2022052012/6027da162e136a34dc33351e/html5/thumbnails/21.jpg)
Algorithm in general
For O(log n) different permutations of x , computesubsample(F ∗ x ,B).Estimate each xi as median of values it maps to.To find S: choose all that map to the top 2k values.
nk/B candidates to update at each iteration: total
(nkB
+ B log n) log n =√
nk log3/2 n
time.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 15 / 19
![Page 22: Simple and Practical Algorithm for the Sparse Fourier ...Hashes correctly to one bucket, leaks to at most 1 bucket. Replace Gaussians with “Dolph-Chebyshev window functions”: factor](https://reader034.vdocuments.site/reader034/viewer/2022052012/6027da162e136a34dc33351e/html5/thumbnails/22.jpg)
Algorithm in general
For O(log n) different permutations of x , computesubsample(F ∗ x ,B).Estimate each xi as median of values it maps to.To find S: choose all that map to the top 2k values.nk/B candidates to update at each iteration: total
(nkB
+ B log n) log n =√
nk log3/2 n
time.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 15 / 19
![Page 23: Simple and Practical Algorithm for the Sparse Fourier ...Hashes correctly to one bucket, leaks to at most 1 bucket. Replace Gaussians with “Dolph-Chebyshev window functions”: factor](https://reader034.vdocuments.site/reader034/viewer/2022052012/6027da162e136a34dc33351e/html5/thumbnails/23.jpg)
Empirical Performance: runtime
0.001
0.01
0.1
1
10
214
215
216
217
218
219
220
221
222
223
224
225
226
Ru
n T
ime
(se
c)
Signal Size (n)
Run Time vs Signal Size (k=50)
sFFT 1.0
sFFT 2.0
FFTW
FFTW OPT
AAFFT 0.9
0.01
0.1
1
10
100 1000
Ru
n T
ime
(se
c)
Number of Non-Zero Frequencies (k)
Run Time vs Sparsity (n=222
)
sFFT 1.0
sFFT 2.0
FFTW
FFTW OPT
AAFFT 0.9
Compare to FFTW, previous best sublinear algorithm (AAFFT).
Offer a heuristic that improves time to O(n1/3k2/3).I Filter from [Mansour ’92].I Can’t rerandomize, might miss elements.
Faster than FFTW for n/k > 2,000.Faster than AAFFT for n/k < 1,000,000.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 17 / 19
![Page 24: Simple and Practical Algorithm for the Sparse Fourier ...Hashes correctly to one bucket, leaks to at most 1 bucket. Replace Gaussians with “Dolph-Chebyshev window functions”: factor](https://reader034.vdocuments.site/reader034/viewer/2022052012/6027da162e136a34dc33351e/html5/thumbnails/24.jpg)
Empirical Performance: noise
1e-07
1e-06
1e-05
0.0001
0.001
0.01
0.1
1
-20 0 20 40 60 80 100
Ave
rage
L1
Err
or p
er E
nrty
SNR (dB)
Robustness vs SNR (n=222, k=50)
sFFT 1.0
sFFT 2.0
AAFFT 0.9
Just like in Count-Sketch, algorithm is noise tolerant.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 18 / 19
![Page 25: Simple and Practical Algorithm for the Sparse Fourier ...Hashes correctly to one bucket, leaks to at most 1 bucket. Replace Gaussians with “Dolph-Chebyshev window functions”: factor](https://reader034.vdocuments.site/reader034/viewer/2022052012/6027da162e136a34dc33351e/html5/thumbnails/25.jpg)
Conclusions
Roughly: fastest algorithm for n/k ∈ [2× 103,106].Recent improvements [HIKP12b?]
I O(k log n) for exactly sparse xI O(k log n
k log n) for approximation.I Beats FFTW for n/k > 400 (in the exact case).
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 19 / 19
![Page 26: Simple and Practical Algorithm for the Sparse Fourier ...Hashes correctly to one bucket, leaks to at most 1 bucket. Replace Gaussians with “Dolph-Chebyshev window functions”: factor](https://reader034.vdocuments.site/reader034/viewer/2022052012/6027da162e136a34dc33351e/html5/thumbnails/26.jpg)
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 20 / 19