offline adaptation using automatically generated heuristics
DESCRIPTION
Offline Adaptation Using Automatically Generated Heuristics. Frédéric de Mesmay , Yevgen Voronenko , and Markus Püschel Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, USA. Marius Fehr, December 7 th 2011. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Offline Adaptation Using Automatically Generated Heuristics](https://reader035.vdocuments.site/reader035/viewer/2022062222/568161dc550346895dd1ea0f/html5/thumbnails/1.jpg)
1
Offline Adaptation Using Automatically Generated Heuristics
Frédéric de Mesmay, Yevgen Voronenko, and Markus PüschelDepartment of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, USA
Marius Fehr, December 7th 2011
![Page 2: Offline Adaptation Using Automatically Generated Heuristics](https://reader035.vdocuments.site/reader035/viewer/2022062222/568161dc550346895dd1ea0f/html5/thumbnails/2.jpg)
Online vs. Offline Adaptive Libraries
2
Online Adaptive Offline Adaptive
Machine Learning
d = dft(n) d = dft(n)
Search generated at Installation
d(X,Y) d(X,Y)
– possibly unbounded initialization time
– impractical for constantly changing problem specifications
+ bounded initialization time
+ no search if input specifications change
![Page 3: Offline Adaptation Using Automatically Generated Heuristics](https://reader035.vdocuments.site/reader035/viewer/2022062222/568161dc550346895dd1ea0f/html5/thumbnails/3.jpg)
3
Motivation
Background
Adaptation Process
Results
![Page 4: Offline Adaptation Using Automatically Generated Heuristics](https://reader035.vdocuments.site/reader035/viewer/2022062222/568161dc550346895dd1ea0f/html5/thumbnails/4.jpg)
4
Statistical Classifier – C4.5
Source: Offline Adaptation Using Automatically Generated Heuristics, CMU
features
˃ Based on entropy, the measure of uncertainty
˃ Feature with smallest entropy becomes root
![Page 5: Offline Adaptation Using Automatically Generated Heuristics](https://reader035.vdocuments.site/reader035/viewer/2022062222/568161dc550346895dd1ea0f/html5/thumbnails/5.jpg)
5
DFT – Discrete Fourier Transforms
˃ The Fourier Transform is a Linear Transform:
XY DFTn= *
![Page 6: Offline Adaptation Using Automatically Generated Heuristics](https://reader035.vdocuments.site/reader035/viewer/2022062222/568161dc550346895dd1ea0f/html5/thumbnails/6.jpg)
6
FFT – Fast Fourier Transforms
˃ Divide and Conquer Algorithms
Source: Lecture Slides :“How to Write Fast Numerical Code “ 263-2300 (ETH, CS) , Markus Püschel
![Page 7: Offline Adaptation Using Automatically Generated Heuristics](https://reader035.vdocuments.site/reader035/viewer/2022062222/568161dc550346895dd1ea0f/html5/thumbnails/7.jpg)
7
FFTW – Search space
dft ( 128 )
These decisions can be optimized!
Radix 8
dft_strided (16, 8)
dft_scaled (8, 16)
Radix 4
No
˃ Features :Problem size and stride
˃ Decisions: Use of base cases and choice of radix
Source: Offline Adaptation Using Automatically Generated Heuristics, CMU
![Page 8: Offline Adaptation Using Automatically Generated Heuristics](https://reader035.vdocuments.site/reader035/viewer/2022062222/568161dc550346895dd1ea0f/html5/thumbnails/8.jpg)
8
FFTW – Advanced Implementations
Lots of decisions!
Source: Offline Adaptation Using Automatically Generated Heuristics, CMU
![Page 9: Offline Adaptation Using Automatically Generated Heuristics](https://reader035.vdocuments.site/reader035/viewer/2022062222/568161dc550346895dd1ea0f/html5/thumbnails/9.jpg)
9
Motivation
Background
Adaptation Process
Results
![Page 10: Offline Adaptation Using Automatically Generated Heuristics](https://reader035.vdocuments.site/reader035/viewer/2022062222/568161dc550346895dd1ea0f/html5/thumbnails/10.jpg)
10
Offline Adaptation Process
C4.5Searchx
x x x x
xx
Heuristicfeatures decisions
size stride use base case radix128 - no 816 - no 44 - yes -
16 8 no 4… … … …
Trainings Set
![Page 11: Offline Adaptation Using Automatically Generated Heuristics](https://reader035.vdocuments.site/reader035/viewer/2022062222/568161dc550346895dd1ea0f/html5/thumbnails/11.jpg)
11
Offline Adaptation Process
Exploration
Statistical Classification
Verification
Combination
Online Adaptive Library
Offline Adaptive Library
Source: Offline Adaptation Using Automatically Generated Heuristics, CMU
![Page 12: Offline Adaptation Using Automatically Generated Heuristics](https://reader035.vdocuments.site/reader035/viewer/2022062222/568161dc550346895dd1ea0f/html5/thumbnails/12.jpg)
12
Exploration
˃ “Creates a table for the statistical classifier to work with”
Exploration
Statistical Classification
Verification
Combination
Searchx
x x x x
xx
features decisionsSize Stride Use Base case Radix128 - No 816 - No 44 - Yes -
16 8 No 4… … … …
Trainings Set
![Page 13: Offline Adaptation Using Automatically Generated Heuristics](https://reader035.vdocuments.site/reader035/viewer/2022062222/568161dc550346895dd1ea0f/html5/thumbnails/13.jpg)
13
Statistical Classification
˃ Computes decision trees
˃ Uses a modified version of C4.5
˃ Provides “hints” based on library functionality
Exploration
Statistical Classification
Verification
Combination
Source: Offline Adaptation Using Automatically Generated Heuristics, CMU
![Page 14: Offline Adaptation Using Automatically Generated Heuristics](https://reader035.vdocuments.site/reader035/viewer/2022062222/568161dc550346895dd1ea0f/html5/thumbnails/14.jpg)
14
Hinting
˃ Problem: Sometimes C4.5 has not enough information to make wise decisions
˃ Choice of radix:˃ Performance depends strongly on the prime factorization
˃ Multiple of 2 and 3 show very different behavior but are heavily interleaved
˃ Solution: Provide additional features to C4.5
Exploration
Statistical Classification
Verification
Combination
12 16 27 32 48 56 64 66 72 81 90 99n
< 20 25 … 4820 … 24 49 … 58 > 59
Radix ? Radix ? Radix ? Radix ? Radix ?
![Page 15: Offline Adaptation Using Automatically Generated Heuristics](https://reader035.vdocuments.site/reader035/viewer/2022062222/568161dc550346895dd1ea0f/html5/thumbnails/15.jpg)
15
HintingExploration
Statistical Classification
Verification
Combination2 4 0 5 4 3 6 1 3 0 1 0
1 0 3 0 1 0 0 1 2 4 2 2
12 16 27 32 48 56 64 66 72 81 90 99
0 0 0
3 4 2
27 81 99
3 1
2 2
72 90
4
1
48
2 1
1 1
12 66
4 5 3 6
0 0 0 0
16 32 56 64
Radix 12
Radix 3
Radix 18
Radix 6
Radix 4
nfactor( 2 , n )
nfactor( 3, n )
n
![Page 16: Offline Adaptation Using Automatically Generated Heuristics](https://reader035.vdocuments.site/reader035/viewer/2022062222/568161dc550346895dd1ea0f/html5/thumbnails/16.jpg)
16
Exploration
Statistical Classification
Verification
Combination
Source: Offline Adaptation Using Automatically Generated Heuristics, CMU
![Page 17: Offline Adaptation Using Automatically Generated Heuristics](https://reader035.vdocuments.site/reader035/viewer/2022062222/568161dc550346895dd1ea0f/html5/thumbnails/17.jpg)
17
VerificationExploration
Statistical Classification
Verification
Combination
24 = 2 * 2 * 2* 3
Source: Offline Adaptation Using Automatically Generated Heuristics, CMU
![Page 18: Offline Adaptation Using Automatically Generated Heuristics](https://reader035.vdocuments.site/reader035/viewer/2022062222/568161dc550346895dd1ea0f/html5/thumbnails/18.jpg)
18
Combination
˃ Inserts decision trees into the library as heuristics
Exploration
Statistical Classification
Verification
Combination
Online Adaptive
d = dft(n)
Search
d(X,Y)
Offline Adaptive
![Page 19: Offline Adaptation Using Automatically Generated Heuristics](https://reader035.vdocuments.site/reader035/viewer/2022062222/568161dc550346895dd1ea0f/html5/thumbnails/19.jpg)
19
Motivation
Background
Adaptation Process
Results
![Page 20: Offline Adaptation Using Automatically Generated Heuristics](https://reader035.vdocuments.site/reader035/viewer/2022062222/568161dc550346895dd1ea0f/html5/thumbnails/20.jpg)
20
2-Powers – Competitiveness
Platform:
2 x dual core 3GHz Intel Xeon 5160
Source: Offline Adaptation Using Automatically Generated Heuristics, CMU
![Page 21: Offline Adaptation Using Automatically Generated Heuristics](https://reader035.vdocuments.site/reader035/viewer/2022062222/568161dc550346895dd1ea0f/html5/thumbnails/21.jpg)
21
Learning and Generating Heuristics
Source: Offline Adaptation Using Automatically Generated Heuristics, CMU
![Page 22: Offline Adaptation Using Automatically Generated Heuristics](https://reader035.vdocuments.site/reader035/viewer/2022062222/568161dc550346895dd1ea0f/html5/thumbnails/22.jpg)
22
Mixed Sizes - Competitiveness
Source: Offline Adaptation Using Automatically Generated Heuristics, CMU
![Page 23: Offline Adaptation Using Automatically Generated Heuristics](https://reader035.vdocuments.site/reader035/viewer/2022062222/568161dc550346895dd1ea0f/html5/thumbnails/23.jpg)
23
˃ Pro˃ Improvement of usability˃ Only small performance penalty (for DFT)˃ Entirely automatic method˃ Applicable to other problem domains / libraries˃ Computer generation of offline adaptive library, directly from algorithm specification
(together with Spiral)
˃ Contra˃ Performance depends strongly on choice of training set
Pro vs. Contra
Algorithm Specification SPIRAL this paper
Adaptive Offline Library
![Page 24: Offline Adaptation Using Automatically Generated Heuristics](https://reader035.vdocuments.site/reader035/viewer/2022062222/568161dc550346895dd1ea0f/html5/thumbnails/24.jpg)
24
Questions
?
![Page 25: Offline Adaptation Using Automatically Generated Heuristics](https://reader035.vdocuments.site/reader035/viewer/2022062222/568161dc550346895dd1ea0f/html5/thumbnails/25.jpg)
25
Questions
?
![Page 26: Offline Adaptation Using Automatically Generated Heuristics](https://reader035.vdocuments.site/reader035/viewer/2022062222/568161dc550346895dd1ea0f/html5/thumbnails/26.jpg)
Online vs. Offline Adaptive Libraries 3
Library Type Non – adaptive Online adaptive Offline adaptive
Prototype IPP FFTW this paper
Interface
Initialization cost
Computation cost
Adaptation mechanism
- online (planer at runtime)
offline (at installation time)
problem changes - rerun planer -
platform changes rebuy rerun planer reinstall
26Source: Offline Adaptation Using Automatically Generated Heuristics, CMU
![Page 27: Offline Adaptation Using Automatically Generated Heuristics](https://reader035.vdocuments.site/reader035/viewer/2022062222/568161dc550346895dd1ea0f/html5/thumbnails/27.jpg)
27
Math behind C4.5
˃ Entropy of event:
˃ Entropy of feature:
˃ Choose feature with smallest entropy