statistical inference for efficient microarchitectural and...
Post on 30-Jul-2021
6 Views
Preview:
TRANSCRIPT
Motivation & BackgroundMicroarchitectural Analysis
Application AnalysisConclusion
Statistical Inference for EfficientMicroarchitectural and Application Analysis
Benjamin C. Lee
www.deas.harvard.edu/∼bcleeDivision of Engineering and Applied Sciences
Harvard University
15 November 2006
Benjamin C. Lee 1 :: Supercomputing 2006
Motivation & BackgroundMicroarchitectural Analysis
Application AnalysisConclusion
Acknowledgements
David Brooks, Harvard UniversityBronis de Supinski, Lawrence Livermore National LaboratoryMartin Schulz, Lawrence Livermore National Laboratory
Benjamin C. Lee 2 :: Supercomputing 2006
Motivation & BackgroundMicroarchitectural Analysis
Application AnalysisConclusion
OutlineMotivation & Background
Parameter Space ExplorationExploration ParadigmStatistical Inference
Microarchitectural AnalysisMethodologyEvaluationCase Study
Application AnalysisMethodologyEvaluationCase Study
Conclusion
Benjamin C. Lee 3 :: Supercomputing 2006
Motivation & BackgroundMicroarchitectural Analysis
Application AnalysisConclusion
Parameter Space ExplorationExploration ParadigmStatistical Inference
OutlineMotivation & Background
Parameter Space ExplorationExploration ParadigmStatistical Inference
Microarchitectural AnalysisMethodologyEvaluationCase Study
Application AnalysisMethodologyEvaluationCase Study
Conclusion
Benjamin C. Lee 4 :: Supercomputing 2006
Motivation & BackgroundMicroarchitectural Analysis
Application AnalysisConclusion
Parameter Space ExplorationExploration ParadigmStatistical Inference
Parameter Space Exploration
Cycle-Accurate SimulationTracks instructions’ progress through microprocessorEstimates performance, power, temperature, . . .
Execution-Based ProfilingSelectively execute application with varying inputsEstimates performance
Exploration CostsNon-trivial simulation, profiling timesParameter space scales exponentially (mp)
p :: parameter countm :: parameter resolution
Benjamin C. Lee 5 :: Supercomputing 2006
Motivation & BackgroundMicroarchitectural Analysis
Application AnalysisConclusion
Parameter Space ExplorationExploration ParadigmStatistical Inference
Exploration Paradigm
Comprehensively understand parameter spaceSpecify large, high-resolution parameter spaceConsider all parameters simultaneously
Selectively measure modest number of pointsSample points randomly from space for measurementDecouple resolution of space and measurements
Efficiently leverage measured data with inferenceReveal trends, trade-offs from sparse samplingEnable predictions for metrics of interest
Benjamin C. Lee 6 :: Supercomputing 2006
Motivation & BackgroundMicroarchitectural Analysis
Application AnalysisConclusion
Parameter Space ExplorationExploration ParadigmStatistical Inference
Model Formulation
Notationn observations . {measured samples}Response :: ~y = y1, . . . , yn . {e.g., performance}Predictor :: ~xi = xi,1, . . . , xi,p . {e.g., L2 cache}
Regression Coefficients :: ~β = β0, . . . , βp
Random Error :: ~e = e1, . . . , en where ei ∼ N(0, σ2)Transformations :: f ,~g = g1, . . . , gp
Model
f (y) = β0 +p∑
j=1
βjgj(xj) + e
Benjamin C. Lee 7 :: Supercomputing 2006
Motivation & BackgroundMicroarchitectural Analysis
Application AnalysisConclusion
Parameter Space ExplorationExploration ParadigmStatistical Inference
Predictor Interaction
Modeling InteractionSuppose effects of predictors x1, x2 cannot be separatedConstruct predictor x3 = x1x2
y = β0 + β1x1 + β2x2 + β3x1x2 + ei
ExampleLet x1 be pipeline depth, x2 be L2 cache sizePerformance impact of pipelining affected by cache size
Speedup =Depth
1 + Stalls/Inst
Benjamin C. Lee 8 :: Supercomputing 2006
Motivation & BackgroundMicroarchitectural Analysis
Application AnalysisConclusion
Parameter Space ExplorationExploration ParadigmStatistical Inference
Predictor Non-Linearity
Restricted Cubic SplinesDivide predictor domain into intervals separated by knotsPiecewise cubic polynomials joined at knotsHigher order polynomials provide better fits
2Stone [SS’86]Benjamin C. Lee 9 :: Supercomputing 2006
Motivation & BackgroundMicroarchitectural Analysis
Application AnalysisConclusion
Parameter Space ExplorationExploration ParadigmStatistical Inference
Prediction
Expected Responseβ are known from least squaresxi,1, . . . , xi,p are known for a given query iExpected response is weighted sum of predictor values
E[y]
= E[β0 +
p∑j=1
βjxj
]+ E
[e]
= β0 +p∑
j=1
βjxj
Benjamin C. Lee 10 :: Supercomputing 2006
Motivation & BackgroundMicroarchitectural Analysis
Application AnalysisConclusion
MethodologyEvaluationCase Study
OutlineMotivation & Background
Parameter Space ExplorationExploration ParadigmStatistical Inference
Microarchitectural AnalysisMethodologyEvaluationCase Study
Application AnalysisMethodologyEvaluationCase Study
Conclusion
Benjamin C. Lee 11 :: Supercomputing 2006
Motivation & BackgroundMicroarchitectural Analysis
Application AnalysisConclusion
MethodologyEvaluationCase Study
Tools and Benchmarks
Simulation FrameworkTurandot :: a cycle-accurate trace driven simulatorPowerTimer :: power models derived from circuit analysesBaseline simulator models POWER4/POWER5 architecture
BenchmarksSPEC2kCPU :: compute-intensive benchmarksSPECjbb :: Java server benchmark
Statistical FrameworkR :: software environment for statistical computingHmisc and Design packages
Benjamin C. Lee 12 :: Supercomputing 2006
Motivation & BackgroundMicroarchitectural Analysis
Application AnalysisConclusion
MethodologyEvaluationCase Study
Predictors :: MicroarchitectureSet Parameters Measure Range |Si|
S1 Depth depth FO4 9::3::36 10S2 Width width insn b/w 4,8,16 3
L/S reorder queue entries 15::15::45store queue entries 14::14::42functional units count 1,2,4
S3 Physical general purpose (GP) count 40::10::130 10Registers floating-point (FP) count 40::8::112
special purpose (SP) count 42::6::96S4 Reservation branch entries 6::1::15 10
Stations fixed-point/memory entries 10::2::28floating-point entries 5::1::14
S5 I-L1 Cache i-L1 cache size log2(entries) 7::1::11 5S6 D-L1 Cache d-L1 cache size log2(entries) 6::1::10 5S7 L2 Cache L2 cache size log2(entries) 11::1::15 5
L2 cache latency cycles 6::2::14
Parameter space of 375,000 design points
Benjamin C. Lee 13 :: Supercomputing 2006
Motivation & BackgroundMicroarchitectural Analysis
Application AnalysisConclusion
MethodologyEvaluationCase Study
Validation Approach
FrameworkFormulate models with 1,000 samplesObtain 100 additional random samples for validationQuantify percentage error, 100 ∗ |yi − yi|/yi
ComparisonSimulator-reported performance, powerRegression-predicted performance, power
Benjamin C. Lee 14 :: Supercomputing 2006
Motivation & BackgroundMicroarchitectural Analysis
Application AnalysisConclusion
MethodologyEvaluationCase Study
Prediction Accuracy
Benjamin C. Lee 15 :: Supercomputing 2006
Motivation & BackgroundMicroarchitectural Analysis
Application AnalysisConclusion
MethodologyEvaluationCase Study
Multiprocessor Heterogeneity I
MotivationEvaluate trends in chip multiprocessor designMitigate penalties from single design compromise
ObjectiveIdentify efficient heterogeneous design compromises
ApproachSimulate 1K samples from design spaceFormulate regression models for performance, powerIdentify per benchmark optima (bips3/w) via regressionIdentify compromises via K-means clustering
Benjamin C. Lee 16 :: Supercomputing 2006
Motivation & BackgroundMicroarchitectural Analysis
Application AnalysisConclusion
MethodologyEvaluationCase Study
Multiprocessor Heterogeneity II
Benjamin C. Lee 17 :: Supercomputing 2006
Motivation & BackgroundMicroarchitectural Analysis
Application AnalysisConclusion
MethodologyEvaluationCase Study
OutlineMotivation & Background
Parameter Space ExplorationExploration ParadigmStatistical Inference
Microarchitectural AnalysisMethodologyEvaluationCase Study
Application AnalysisMethodologyEvaluationCase Study
Conclusion
Benjamin C. Lee 18 :: Supercomputing 2006
Motivation & BackgroundMicroarchitectural Analysis
Application AnalysisConclusion
MethodologyEvaluationCase Study
Platforms and Workloads
PlatformsBlue Gene/LIntel Xeon Clusters (ALC,MCR)
Numerical MethodsSemicoarsening Multigrid 2000 (SMG2k)High-Performance Linpack (HPL)
Statistical FrameworkR :: software environment for statistical computingHmisc and Design packages
Benjamin C. Lee 19 :: Supercomputing 2006
Motivation & BackgroundMicroarchitectural Analysis
Application AnalysisConclusion
MethodologyEvaluationCase Study
Predictors :: Numerical MethodsSemicoarsening Multigrid 2000
Set Parameters Measure Range |Si|S1 Nx x-dim working set grid points 10:20:510 26S2 Ny y-dim 10:20:510 26S3 Nz z-dim 10:20:510 26S4 Px x-dim processors processors 1,8,64,512 4S5 Py y-dim 1,8,64,512 4S6 Pz z-dim 1,8,64,512 4
Parameter space of ∼ 280,000 combinations
High-Performance LinpackSet Parameters Measure Range |Si|
S1 Matrix Size N sq. matrix dim 1000 1S2 Block Size NB sq. block dim 10:10:80 8S3 Processor rows (P) log2(procs) 0:1:9 10
Distribution columns (Q) log2(procs) 9-PS4 Panel Factor PFACT algorithm L,R,C 3S5 Recursive Factor RFACT algorithm L,R,C 3S6 Recursive Base NBMIN block size 1:1:8 8S7 Recursive Sub-Panels NDIV sub-panels 2:1:4 3S8 Broadcast BCAST algorithm 1rg, 1rM, 2rg, 6
2rM, Lng, LnM
Parameter space of ∼ 100,000 combinations
Benjamin C. Lee 20 :: Supercomputing 2006
Motivation & BackgroundMicroarchitectural Analysis
Application AnalysisConclusion
MethodologyEvaluationCase Study
Validation Approach
FrameworkFormulate models with 600 samplesObtain 100 additional random samples for validationQuantify percentage error, 100 ∗ |yi − yi|/yi
ComparisonProfiled execution timeRegression-predicted execution time
Benjamin C. Lee 21 :: Supercomputing 2006
Motivation & BackgroundMicroarchitectural Analysis
Application AnalysisConclusion
MethodologyEvaluationCase Study
Prediction Accuracy
Benjamin C. Lee 22 :: Supercomputing 2006
Motivation & BackgroundMicroarchitectural Analysis
Application AnalysisConclusion
MethodologyEvaluationCase Study
Performance Gradients I
MotivationModel performance empiricallyCircumvent analytical complexity
ObjectiveUnderstand performance topology, bottlenecks
ApproachMeasure 600 samples from parameter spaceFormulate regression models for performancePredict execution time for every pointCompute numerical gradients with local differences
Benjamin C. Lee 23 :: Supercomputing 2006
Motivation & BackgroundMicroarchitectural Analysis
Application AnalysisConclusion
MethodologyEvaluationCase Study
Performance Gradients II
Benjamin C. Lee 24 :: Supercomputing 2006
Motivation & BackgroundMicroarchitectural Analysis
Application AnalysisConclusion
OutlineMotivation & Background
Parameter Space ExplorationExploration ParadigmStatistical Inference
Microarchitectural AnalysisMethodologyEvaluationCase Study
Application AnalysisMethodologyEvaluationCase Study
Conclusion
Benjamin C. Lee 25 :: Supercomputing 2006
Motivation & BackgroundMicroarchitectural Analysis
Application AnalysisConclusion
Conclusion
Exploration ParadigmComprehensively understand parameter spaceSelectively measure modest number of pointsEfficiently leverage measured data with inference
Model Evaluation7.2%, 5.4% median errors for µ-arch performance, power5.1%, 3.1% median errors for SMG2K, HPL performance
Future DirectionsChip multiprocessors and on-chip interconnectAdditional applications and compiler parametersCombine microarchitecture, application models
Benjamin C. Lee 26 :: Supercomputing 2006
AppendixFurther ReadingReferencesExtra Slides
Further Readingwww.deas.harvard.edu/∼bclee
B.C. Lee and D.M. Brooks and B.R. de Supinski and M. Schulz and K. Singh and S.A. McKee.Methods of inference & learning for performance modeling of parallel applications.PPoPP’07: Symposium on Principles & Practice of Parallel Programming, March 2007.
B.C. Lee and D.M. Brooks.Illustrative design space studies with microarchitectural regression models.HPCA-13: International Symposium on High Performance Computer Architecture, Feb 2007.
B.C. Lee and D.M. Brooks.Accurate, efficient regression modeling for microarchitectural performance, power prediction.ASPLOS-XII: International Conference on Architectural Support for Programming Languages & OperatingSystems, Oct 2006.
B.C. Lee and M. Schulz and B. de Supinski.Regression strategies for parameter space exploration: A case study in semicoarsening multigrid & R.Technical Report UCRL-TR-224851, Lawrence Livermore National Laboratory, Sept 2006.
B.C. Lee and D.M. Brooks.Statistically rigorous regression modeling for the microprocessor design space.MoBS-2: Workshop on Modeling, Benchmarking, & Simulation, June 2006.
B.C. Lee and D.M. Brooks.Regression modeling strategies for microarchitectural performance & power prediction.Harvard University Technical Report TR-08-06, March 2006.
Benjamin C. Lee 27 :: Supercomputing 2006
AppendixFurther ReadingReferencesExtra Slides
References IY. Li, B.C. Lee, D. Brooks, Z. Hu, K. Skadron.CMP design space exploration subject to physical constraints.HPCA-12: International Symposium on High-Performance Computer Architecture, Feb 2006.
L. Eeckhout, S. Nussbaum, J. Smith, and K. DeBosschere.Statistical simulation: Adding efficiency to the computer designer’s toolbox.IEEE Micro, Sept/Oct 2003.
R. Liu and K. Asanovic.Accelerating architectural exploration using canonical instruction segments.In International Symposium on Performance Analysis of Systems and Software, Austin, Texas,March 2006.
T. Sherwood, E. Perelman, G. Hamerly, and B. Calder.Automatically characterizing large scale program behavior.ASPLOX-X: Architectural Support for Programming Languages and Operating Systems,October 2002.
B.C. Lee and D.M. Brooks.Effects of pipeline complexity on SMT/CMP power-performance efficiency.ISCA-32: Workshop on Complexity Effective Design, June 2005.
Benjamin C. Lee 28 :: Supercomputing 2006
AppendixFurther ReadingReferencesExtra Slides
References IIC. Stone.Comment: Generalized additive models.Statistical Science, 1986.
F. Harrell.Regression modeling strategies.Springer, New York, NY, 2001.
J. Yi, D. Lilja, and D. Hawkins.Improving computer architecture simulation methodology by adding statistical rigor.IEEE Computer, Nov 2005.
P. Joseph, K. Vaswani, and M. J. Thazhuthaveetil.Construction and use of linear regression models for processor performance analysis.In Proceedings of the 12th Symposium on High Performance Computer Architecture, Austin,Texas, February 2006.
S. Nussbaum and J. Smith.Modeling superscalar processors via statistical simulation.In PACT2001: International Conference on Parallel Architectures and Compilation Techniques,Barcelona, Sept 2001.
Benjamin C. Lee 29 :: Supercomputing 2006
AppendixFurther ReadingReferencesExtra Slides
Treatment of Missing Data
Missing Completely at Random (MCAR)Treat unobserved design points as missing dataSampling UAR ensures observations are MCARData is missing for reasons unrelated to characteristics orresponses of the configuration
Informative MissingData is more likely missing if their responses aresystematically higher or lower“Missingness” is non-ignorable and must also be modeledSampling UAR avoids such modeling complications
Benjamin C. Lee 30 :: Supercomputing 2006
AppendixFurther ReadingReferencesExtra Slides
Predictor Non-Linearity I
Polynomial TransformationsUndesirable peaks and valleysDiffering trends across regions
Linear SplinesPiecewise linear regions separated by knotsInadequate for complex, highly curved relationships
Restricted Cubic SplinesHigher order polynomials provide better fitsContinuous at knotsLinear constraint on tails
Benjamin C. Lee 31 :: Supercomputing 2006
AppendixFurther ReadingReferencesExtra Slides
Predictor Non-Linearity II
Location of KnotsLocation of knots less important than number of knotsPlace knots at fixed predictor quantiles
Number of KnotsFlexibility, risk of over-fitting increases with knot count5 knots or fewer are often sufficient 1
4 knots is a good compromise between flexibility, over-fittingFewer knots required for small data sets
1Stone [SS’86]Benjamin C. Lee 32 :: Supercomputing 2006
AppendixFurther ReadingReferencesExtra Slides
Derivation OverviewSpatial Sampling
Hierarchical ClusteringAssociation Analysis
qualitative scatterplots, quantitative ρ2
Model Specificationpredictor interaction, non-linearity
Assessing FitR2 statistic
Residual Analysisnormality (quantile-quantile), randomness (scatterplots)
Significance Testinghypothesis testing, F-statistic, p-values
4Lee[ASPLOS’06], Lee[PPoPP’07]Benjamin C. Lee 33 :: Supercomputing 2006
AppendixFurther ReadingReferencesExtra Slides
Variable Clustering
stal
l_ca
stst
all_
resv
dl1m
iss_
rate
base
_bip
sdl
2mis
s_ra
test
all_
dmis
sqst
all_
stor
eqbr
_mis
_rat
est
all_
reor
derq
fix_l
atct
l_la
tls
_lat
fpu_
lat
dept
hm
em_l
atw
idth
bips
phys
_reg
l2ca
che_
size
resv
il1m
iss_
rate
il2m
iss_
rate stal
l_in
fligh
tst
all_
rena
me
br_r
ate
br_s
tall
icac
he_s
ize
dcac
he_s
ize
1.0
0.8
0.6
0.4
0.2
0.0
Spe
arm
an ρ
2
Benjamin C. Lee 34 :: Supercomputing 2006
AppendixFurther ReadingReferencesExtra Slides
Performance Associations
bips
0.6 0.7 0.8 0.9
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
12131170 815 802
131813201362
11971179 802 822
11251247 818 810
4000
N
4 8
16
[ 9,18) [18,27) [27,33) [33,36]
[ 40, 70) [ 70,100) [100,120) [120,130]
[ 6, 9) [ 9,12)
[12,14) [14,15]
depth
width
phys_reg
resv
Overall
Pipeline
N=4000bips
0.74 0.78 0.82
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
799 828 807 805 761
774 794 773 837 822
825 776 823 780 796
4000
N
11 12 13 14 15
7 8 9
10 11
6 7 8 9
10
l2cache_size
icache_size
dcache_size
Overall
Memory Hierarchy
N=4000bips
0.5 0.8
●
●
●
●
●
●
●
●
●
●
●
●
●
1110 9191071 900
1103 9111103 883
1856 349 928 867
4000
N
0.02
[5.08e−06,3.91e−05) [3.91e−05,7.15e−04) [7.15e−04,3.85e−03) [3.85e−03,3.71e−02]
[0.02,0.09) [0.09,0.12) [0.12,0.18) [0.18,0.55]
[0.00,0.02)
[0.04,0.11) [0.11,0.24]
il1miss_rate
dl1miss_rate
dl2miss_rate
Overall
Cache
N=4000
Benjamin C. Lee 35 :: Supercomputing 2006
AppendixFurther ReadingReferencesExtra Slides
Performance Associations I
bips
0.6 0.7 0.8 0.9
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
12131170 815 802
131813201362
11971179 802 822
11251247 818 810
4000
N
4 8
16
[ 9,18) [18,27) [27,33) [33,36]
[ 40, 70) [ 70,100) [100,120) [120,130]
[ 6, 9) [ 9,12)
[12,14) [14,15]
depth
width
phys_reg
resv
Overall
Pipeline
N=4000bips
0.70 0.80 0.90
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
10101017 977 996
1505 728 953 814
2399 800 432 180 189
1036 973
1197 794
1298 707
1046 949
4000
N
4
1 2 3 4 5
1 2
5
[ 34, 56) [ 56, 77)
[ 77,124) [124,307]
[1, 4)
[5, 8) [8,19]
[3, 5) [5,13]
[ 2, 5)
[ 6,10) [10,24]
mem_lat
ls_lat
ctl_lat
fix_lat
fpu_lat
Overall
Latency
N=4000bips
0.74 0.78 0.82
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
799 828 807 805 761
774 794 773 837 822
825 776 823 780 796
4000
N
11 12 13 14 15
7 8 9
10 11
6 7 8 9
10
l2cache_size
icache_size
dcache_size
Overall
Memory Hierarchy
N=4000
Benjamin C. Lee 36 :: Supercomputing 2006
AppendixFurther ReadingReferencesExtra Slides
Performance Associations II
bips
0.6 0.8
●
●
●
●
●
●
●
●
●
1076
1060
934
930
1085
1055
960
900
4000
N
[0.01,0.05)
[0.05,0.10)
[0.10,0.16)
[0.16,0.29]
[0.005,0.034)
[0.034,0.072)
[0.072,0.093)
[0.093,0.165]
br_rate
br_mis_rate
Overall
Branch
N=4000bips
0.6 0.8
●
●
●
●
●
●
●
●
●
●
●
●
●
10751104 910 911
1114 933
1053 900
1135 876
1129 860
4000
N
[ 6225, 108733) [108733, 242619) [242619, 597291) [597291,2821539]
[ 193084,2.60e+06) [ 2599382,7.22e+06) [ 7222608,4.69e+07) [46879987,1.04e+08]
[ 949, 6037) [ 6037, 18667)
[ 18667, 254857) [254857,14883942]
stall_inflight
stall_dmissq
stall_cast
Overall
Stalls
N=4000bips
0.5 0.7 0.9
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
1093 933
1045 929
1096 907
1077 920
1127 922
1070 881
10991075 941 885
4000
N
[ 0, 9959) [ 9959, 97381)
[ 97381, 299112) [299112,1373099]
[ 0, 350) [ 350, 24448)
[ 24448,144305) [144305,923543]
[ 1814764,5.98e+06) [ 5984258,1.01e+07)
[10134921,2.42e+07) [24224471,1.13e+08]
[ 16188, 2013914) [ 2013914, 4131316)
[ 4131316,22315283) [22315283,45296951]
stall_storeq
stall_reorderq
stall_resv
stall_rename
Overall
Stalls
N=4000
Benjamin C. Lee 37 :: Supercomputing 2006
AppendixFurther ReadingReferencesExtra Slides
Performance Associations III
bips
0.5 0.8
●
●
●
●
●
●
●
●
●
●
●
●
●
1110 9191071 900
1103 9111103 883
1856 349 928 867
4000
N
0.02
[5.08e−06,3.91e−05) [3.91e−05,7.15e−04) [7.15e−04,3.85e−03) [3.85e−03,3.71e−02]
[0.02,0.09) [0.09,0.12) [0.12,0.18) [0.18,0.55]
[0.00,0.02)
[0.04,0.11) [0.11,0.24]
il1miss_rate
dl1miss_rate
dl2miss_rate
Overall
Cache
N=4000bips
0.5 0.7 0.9
●
●
●
●
●
1056
1122
913
909
4000
N
[0.309,0.92)
[0.920,1.21)
[1.208,1.35)
[1.347,1.58]
base_bips
Overall
Baseline Performance
N=4000
Benjamin C. Lee 38 :: Supercomputing 2006
AppendixFurther ReadingReferencesExtra Slides
Assessing FitMultiple Correlation Statistic
R2 is fraction of response variance captured by predictorsLarge R2 suggests better fit to observed dataR2 → 1 suggests over-fitting (less likely if p < n/20)
R2 = 1−∑n
i=1(yi − yi)2∑ni=1(yi − 1
n
∑ni=1 yi)2
Residual Distribution AssumptionsResiduals are normally distributed, ei ∼ N(0, σ2)No correlation between residuals and response, predictorsValidate by scatterplots and quantile-quantile plots
ei = yi − β0 −p∑
j=0
βjxij
Benjamin C. Lee 39 :: Supercomputing 2006
AppendixFurther ReadingReferencesExtra Slides
Significance Testing I
ApproachGiven two nested models, hypothesis H0 states additionalpredictors in larger model have no response associationTest H0 with F-statistics and p-values
ExamplePredictor interaction requires comparing nested modelsConsider a model y = β0 + β1x1 + β2x2 + β3x1x2.Test significance of x1 with null hypothesis H0 : β1 = β3 = 0
Benjamin C. Lee 40 :: Supercomputing 2006
AppendixFurther ReadingReferencesExtra Slides
Significance Testing IIF-Statistic
Compare two nested models using their R2 and F-statisticR2 is fraction of response variance captured by predictors
R2 = 1−∑n
i=1(yi − yi)2∑ni=1(yi − 1
n
∑ni=1 yi)2
F-statistic of two nested models follows F distribution
Fk,n−p−1 =R2 − R2
∗k
× n− p− 11− R2
P-ValuesProbability F-statistic greater than or equal to observedvalue would occur under H0
Small p-values cast doubt on H0
Benjamin C. Lee 41 :: Supercomputing 2006
AppendixFurther ReadingReferencesExtra Slides
Significance Testing IV
Microarchitectural PredictorsMajority of F-tests imply significance (p-values < 2.2E − 16)Several predictors were less significant
Control latency (p-value = 0.1247)Reservation station size (p-value = 0.1239)L1 instruction cache size (p-value = 0.02941)
Application-Specific PredictorsMajority of F-tests imply significance (p-values < 2.2E − 16)Pipeline stalls classified by structure are less significant
Completion and reorder queue stalls (p-values > 0.4)
Benjamin C. Lee 42 :: Supercomputing 2006
AppendixFurther ReadingReferencesExtra Slides
Related Work
Statistical Significance RankingYi :: Plackett-Burman, effect rankingsJoseph :: Stepwise regression, coefficient rankingsBound parameter values to improve tractabilityRequire simulation for estimation
Synthetic WorkloadsEeckhout :: Profile workloads to obtain synthetic tracesNussbaum :: Superscalar and SMP simulationObtain distribution of instructions and data dependenciesRequire simulation with smaller traces for estimation
Benjamin C. Lee 43 :: Supercomputing 2006
top related