dancing with uncertainty saša misailović stelios sidiroglou martin rinard mit csail
TRANSCRIPT
![Page 1: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/1.jpg)
Dancing With Uncertainty
Saša Misailović
Stelios Sidiroglou
Martin Rinard
MIT CSAIL
![Page 2: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/2.jpg)
ExampleWater: Simulates system of water molecules
HHO
HHO
HHO
H
HO
H
HO
HHO
HHO
![Page 3: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/3.jpg)
ExampleWater: Simulates system of water molecules
HHO
HHO
HHO
H
HO
H
HO
HHO
HHO
![Page 4: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/4.jpg)
ExampleWater: Simulates system of water molecules
HHO
HHO
HHO
H
HO
H
HO
HHO
HHO
![Page 5: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/5.jpg)
ExampleWater: Simulates liquid water molecules
HHO
HHO
HHO
H
HO
H
HO
HHO
HHO
![Page 6: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/6.jpg)
ExampleWater: Simulates system of water molecules
HHO
HHO
HHO
H
HO
H
HO
HHO
HHO
![Page 7: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/7.jpg)
ExampleWater: Simulates system of water molecules
HHO
HHO
HHO
H
HO
H
HO
HHO
HHO
![Page 8: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/8.jpg)
Dubstep
Explores the effects of selectively removing
synchronization
![Page 9: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/9.jpg)
Dubstep Highlights
1. Removing locks and opportunistic barrierstrade accuracy for performance
2. Automatically explores the tradeoff space induced by candidate transformations
3. Uses statistical analysis to characterize impact of transformations on accuracy
![Page 10: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/10.jpg)
Dubstep Workflow
Prepare
Find
Transform
Analyze
Navigate
![Page 11: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/11.jpg)
Dubstep Workflow
Prepare
Find
Transform
Analyze
Navigate
1. Prepare representative inputs
2. Prepare accuracy model– Output abstraction
(important parts of output)– Accuracy bound (amount of
tolerable error)
![Page 12: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/12.jpg)
Dubstep Workflow
Prepare
Find
Transform
Analyze
Navigate
Loops with parallel constructs• Profiling: performance &
memory
![Page 13: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/13.jpg)
Dubstep Workflow
Prepare
Find
Transform
Analyze
Navigate
Loops with parallel constructs• Profiling: performance &
memoryInterf (56.4%)
Poteng (43.4%)
![Page 14: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/14.jpg)
Dubstep Workflow
Removing synchronizationPrepare
Find
Transform
Analyze
Navigate
void scratchPad::updateForces (double R[3][3]) { mutex_lock(this->lock); this->H1force.vecAdd(R[0]); this->Oforce.vecAdd(R[1]); this->H2force.vecAdd(R[2]); mutex_unlock(this->lock);}
![Page 15: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/15.jpg)
Dubstep Workflow
Removing synchronizationPrepare
Find
Transform
Analyze
Navigate
void scratchPad::updateForces (double R[3][3]) { mutex_lock(this->lock); this->H1force.vecAdd(R[0]); this->Oforce.vecAdd(R[1]); this->H2force.vecAdd(R[2]); mutex_unlock(this->lock);}
![Page 16: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/16.jpg)
Dubstep Workflow
Removing synchronizationPrepare
Find
Transform
Analyze
Navigate
void scratchPad::updateForces (double R[3][3]) { this->H1force.vecAdd(R[0]); this->Oforce.vecAdd(R[1]); this->H2force.vecAdd(R[2]); }
![Page 17: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/17.jpg)
Dubstep Workflow
Opportunistic barriersPrepare
Find
Transform
Analyze
Navigate
void ensemble::interf(){ parallel_for( interf_body, 0, NumMol-1 ); }
![Page 18: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/18.jpg)
Dubstep Workflow
Opportunistic barriersPrepare
Find
Transform
Analyze
Navigate
void ensemble::interf(){ parallel_for( interf_body, 0, NumMol-1 ); }
![Page 19: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/19.jpg)
Dubstep Workflow
Opportunistic barriersPrepare
Find
Transform
Analyze
Navigate
void ensemble::interf(){ parallel_for( interf_body, 0, NumMol-1 ); }
• Schedule threads • Execute interf_body in
parallel• Wait for all threads to
complete
![Page 20: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/20.jpg)
Dubstep Workflow
Opportunistic barriersPrepare
Find
Transform
Analyze
Navigate
void ensemble::interf(){ parallel_for*( interf_body, 0, NumMol-1 ); }
• Schedule threads • Execute interf_body in
parallel• Wait for half of threads to
completeInstruct remaining threads to stop
[Rinard, OOPSLA 2007]
![Page 21: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/21.jpg)
Dubstep Workflow
Analyze transformed program:
• Criticality– Memory safety, integrity
• Performance – Speedup comparison
• Accuracy– Statistical analysis
Prepare
Find
Transform
Analyze
Navigate
![Page 22: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/22.jpg)
Dubstep Workflow
Prepare
Find
Transform
Analyze
Navigate
c
InputOriginal ProgramOutput
Output Abstraction(Application-Specific)
Transformed
Program
Difference Bound
δ<
![Page 23: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/23.jpg)
Dubstep Workflow
Navigate the tradeoff space:• Transform and analyze
one location at a time– 3 locations in water
• Transform multiple locations in the same candidate program– Guided by the results of the
previous step
Prepare
Find
Transform
Analyze
Navigate
![Page 24: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/24.jpg)
Search Space Exploration
0 0.01 0.02 0.03 0.04 0.05 0.06 0.071
1.05
1.1
1.15
1.2
1.25
Average Accuracy Loss vs. Speedup
LI BI
BRLI+BI
LI+BP
BI+BP
LI+BI+BP
Rela
tive
Sp
eed
up
Accuracy loss
LI – Synchronization InterfBI – Barrier InterfBP – Barrier Poteng
Baseline: original parallel program runs
6.2 times faster than sequential on 8 cores
![Page 25: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/25.jpg)
Search Space Exploration
0 0.01 0.02 0.03 0.04 0.05 0.06 0.071
1.05
1.1
1.15
1.2
1.25
Average Accuracy Loss vs. Speedup
LI BI
BRLI+BI
LI+BP
BI+BP
LI+BI+BP
Rela
tive
Sp
eed
up
Accuracy loss
LI – Synchronization InterfBI – Barrier InterfBP – Barrier Poteng
How confident can we be about these observations?
Baseline: original parallel program runs
6.2 times faster than sequential on 8 cores
![Page 26: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/26.jpg)
Execution Reliability
The probability p that the transformed program on the given
input produces the result with error less than
bound δ𝐩=𝐏𝐫 [|𝐑𝐞𝐬𝐎−𝐑𝐞𝐬𝐓
𝐑𝐞𝐬𝐎 |≤𝛅]While we cannot model p, we can specify
minimum acceptable reliability r
![Page 27: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/27.jpg)
Execution Reliability
• Repeat execution N times:• Observations: if , else 0
• Compute statistic p’
• Return Yes if p’ > r + • Return No otherwise
Determine if program’s reliability p > r
![Page 28: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/28.jpg)
Execution Reliability
• Repeat execution N times:• Observations: if , else 0
• Compute statistic p’
• Return Yes if p’ > r + • Return No otherwise
Determine if program’s reliability p > r
How to pick N?
![Page 29: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/29.jpg)
How Many Runs Are Enough?
Procedure that determines that p > r :
• Returns correct result most of the time– Wrong decision rate – Tolerance region
• Quickly determines extreme (very good or bad) transformations
![Page 30: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/30.jpg)
Statistical AnalysisSequential Probability Ratio Test
Two hypotheses:
H0: p > r +
H1: p < r
• Collects one observation in every iteration
• Updates likelihoods of H0 and H1 based on the
previous observation • Stops when wrong decision less than
specified • N is not fixed, depends on observations p
![Page 31: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/31.jpg)
Statistical AnalysisSequential Probability Ratio Test
• r (acceptable reliability) = 0.90• (wrong decision rate) = 0.10• ε (tolerance region) = 0.02
• If program always produces acceptable result, test says Yes after 100 runs
• If program never produces acceptable result, test says No after 10 runs
![Page 32: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/32.jpg)
Statistical AnalysisSequential Probability Ratio Test
• r (acceptable reliability) = 0.90• (wrong decision rate) = 0.10• ε (tolerance region) = 0.02
Bound (δ)Best
Transformation
0.01 LI
0.05 LI
0.10 LI+BI+BR
0.15 LI+BI+BR
![Page 33: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/33.jpg)
Statistical AnalysisSequential Probability Ratio Test
• r (acceptable reliability) = 0.90• (wrong decision rate) = 0.10• ε (tolerance region) = 0.02
Bound (δ)Best
Transformation
0.01 LI
0.05 LI
0.10 LI+BI+BR
0.15 LI+BI+BR
![Page 34: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/34.jpg)
Exploring Tradeoff Space
Start: Sequential program with for loopsTransformations:• Parallel loop introduction• Synchronization,
ReplicationQuickstep [MIT-TR-2010-38, TECS/PEC 2012]
Prepare
Find
Transform
Analyze
Navigate
![Page 35: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/35.jpg)
Exploring Tradeoff Space
Start: Program with for loopsTransformations:• Skip loop iterations
(multiple forms)
Loop Perforation[ICSE 2010, ONWARD 2010, SAS 2011, FSE 2011]
Prepare
Find
Transform
Analyze
Navigate
![Page 36: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/36.jpg)
Exploring Tradeoff Space
Start: Program with command line parametersTransformations:• Alternate function
versions activated by CL parameters
Dynamic Knobs [ASPLOS 2011]
Prepare
Find
Transform
Analyze
Navigate
![Page 37: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/37.jpg)
Exploring Tradeoff Space
Start: Program is a tree of Map-Reduce type tasksTransformations:• Function Substitution• Reduction Sampling
NapRed[POPL 2012]
Prepare
Find
Transform
Analyze
Navigate
![Page 38: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/38.jpg)
Exploring Tradeoff Space
Start: Parallel program with for loopsTransformations:• Removing Locks• Opportunistic Barriers
Dubstep[Today: RACES 2012]
Prepare
Find
Transform
Analyze
Navigate
![Page 39: Dancing With Uncertainty Saša Misailović Stelios Sidiroglou Martin Rinard MIT CSAIL](https://reader037.vdocuments.site/reader037/viewer/2022110400/56649db95503460f94aa91e7/html5/thumbnails/39.jpg)
Reasoning About Accuracy
Exploring levels of accuracy guarantees:• Logic-based• Probabilistic• Statistical• Empirical
Prepare
Find
Transform
Analyze
Navigate