faculty of engineering & information technologies a pareto frontier for optimizing data transfer...
TRANSCRIPT
![Page 1: FACULTY OF ENGINEERING & INFORMATION TECHNOLOGIES A Pareto Frontier for Optimizing Data Transfer vs. Job Execution in Grids Albert Y. Zomaya | Professor](https://reader036.vdocuments.site/reader036/viewer/2022062318/551af408550346f70d8b4f2a/html5/thumbnails/1.jpg)
FACULTY OFENGINEERING &INFORMATION TECHNOLOGIES
A Pareto Frontier for Optimizing Data Transfer vs. Job Execution in Grids
Albert Y. Zomaya | Professor and Director
Centre for Distributed and High Performance Computing School of Information Technologies
The University of Sydney, Sydney, Australia
Javid Taheri | Postdoctoral Research Fellow
![Page 2: FACULTY OF ENGINEERING & INFORMATION TECHNOLOGIES A Pareto Frontier for Optimizing Data Transfer vs. Job Execution in Grids Albert Y. Zomaya | Professor](https://reader036.vdocuments.site/reader036/viewer/2022062318/551af408550346f70d8b4f2a/html5/thumbnails/2.jpg)
2
› Introduction to Grid Computing
› Problem Statement: Data-Aware Job Scheduling
› GA-ParFnt
- Pareto Frontier
- Genetic Algorithm (GA)
› Simulation and Analysis of Results
› Conclusion
![Page 3: FACULTY OF ENGINEERING & INFORMATION TECHNOLOGIES A Pareto Frontier for Optimizing Data Transfer vs. Job Execution in Grids Albert Y. Zomaya | Professor](https://reader036.vdocuments.site/reader036/viewer/2022062318/551af408550346f70d8b4f2a/html5/thumbnails/3.jpg)
3
Grid Computing
![Page 4: FACULTY OF ENGINEERING & INFORMATION TECHNOLOGIES A Pareto Frontier for Optimizing Data Transfer vs. Job Execution in Grids Albert Y. Zomaya | Professor](https://reader036.vdocuments.site/reader036/viewer/2022062318/551af408550346f70d8b4f2a/html5/thumbnails/4.jpg)
4
Problem Statement
› Data Aware Job Scheduling (DAJS)
- (1) the overall execution time of a batch of jobs (NP-Complete)
- (2) transfer time of all datafiles to their dependent jobs (NP-Complete)
Storage
Nodes
Com
puta
tion
Nod
es
Job 1
Job 2
Job 3
Job N
...
File 1
File 2
File 3
...
File M
![Page 5: FACULTY OF ENGINEERING & INFORMATION TECHNOLOGIES A Pareto Frontier for Optimizing Data Transfer vs. Job Execution in Grids Albert Y. Zomaya | Professor](https://reader036.vdocuments.site/reader036/viewer/2022062318/551af408550346f70d8b4f2a/html5/thumbnails/5.jpg)
5
Problem Statement (cont.)
SN
SN
SN
CN
CN
CN
Scheduler
![Page 6: FACULTY OF ENGINEERING & INFORMATION TECHNOLOGIES A Pareto Frontier for Optimizing Data Transfer vs. Job Execution in Grids Albert Y. Zomaya | Professor](https://reader036.vdocuments.site/reader036/viewer/2022062318/551af408550346f70d8b4f2a/html5/thumbnails/6.jpg)
6
Preliminaries
› Pareto Front
› Genetic Algorithm
![Page 7: FACULTY OF ENGINEERING & INFORMATION TECHNOLOGIES A Pareto Frontier for Optimizing Data Transfer vs. Job Execution in Grids Albert Y. Zomaya | Professor](https://reader036.vdocuments.site/reader036/viewer/2022062318/551af408550346f70d8b4f2a/html5/thumbnails/7.jpg)
7
GA for Finding DAJS’ Pareto Front (GA-ParFnt)
![Page 8: FACULTY OF ENGINEERING & INFORMATION TECHNOLOGIES A Pareto Frontier for Optimizing Data Transfer vs. Job Execution in Grids Albert Y. Zomaya | Professor](https://reader036.vdocuments.site/reader036/viewer/2022062318/551af408550346f70d8b4f2a/html5/thumbnails/8.jpg)
8
Simulation
› Test-Grid-4-8
![Page 9: FACULTY OF ENGINEERING & INFORMATION TECHNOLOGIES A Pareto Frontier for Optimizing Data Transfer vs. Job Execution in Grids Albert Y. Zomaya | Professor](https://reader036.vdocuments.site/reader036/viewer/2022062318/551af408550346f70d8b4f2a/html5/thumbnails/9.jpg)
9
Discussion and Analysis
› The shape of Pareto Front
Test-Grid-8-4
![Page 10: FACULTY OF ENGINEERING & INFORMATION TECHNOLOGIES A Pareto Frontier for Optimizing Data Transfer vs. Job Execution in Grids Albert Y. Zomaya | Professor](https://reader036.vdocuments.site/reader036/viewer/2022062318/551af408550346f70d8b4f2a/html5/thumbnails/10.jpg)
10
Discussion and Analysis
› Scheduling Algorithms
![Page 11: FACULTY OF ENGINEERING & INFORMATION TECHNOLOGIES A Pareto Frontier for Optimizing Data Transfer vs. Job Execution in Grids Albert Y. Zomaya | Professor](https://reader036.vdocuments.site/reader036/viewer/2022062318/551af408550346f70d8b4f2a/html5/thumbnails/11.jpg)
11
Conclusion
› GA-ParFnt was effective in finding the Pareto Front of executing jobs vs Transfer time of Datafiles in Grids
› Such Pareto Front could be estimated by exponential funcitons
› Many scheduling algorithms are not optimal, despite their claim.
![Page 12: FACULTY OF ENGINEERING & INFORMATION TECHNOLOGIES A Pareto Frontier for Optimizing Data Transfer vs. Job Execution in Grids Albert Y. Zomaya | Professor](https://reader036.vdocuments.site/reader036/viewer/2022062318/551af408550346f70d8b4f2a/html5/thumbnails/12.jpg)
12
THANK YOUQuestions?