task programming
DESCRIPTION
Task Programming in CloudTRANSCRIPT
A Presentation on Task Programming
Introduction Computing categories Frameworks Task Based application models Aneka Task-Based programming
◦ Task programming model◦ Task Programming examples
Summary References
OVERVIEW
Task computing:◦ What is Task?
a distinct unit of code, or a program, that can be separated and executed in a remote runtime environment
◦ Difference between Multithreaded Computing and Task computing: Multithreaded programming:
mainly concerned with providing a support for parallelism within a single machine.
Task computing: provides distribution by harnessing the computer power of
several computing nodes
◦ Now clouds have emerged as an attractive solution.
Introduction
A Task computing Scenario
According to the specific nature of the problem, task computing is categorized into:
◦ High-performance computing (HPC)
◦ High-throughput computing (HTC)
◦ Many-task computing (MTC).
Computing Catogories
use of distributed computing facilities for solving problems that need large computing power.
Historically, supercomputers and clusters
are specifically designed to support HPC applications.
HPC
use of distributed computing facilities for applications requiring large computing power over a long period of time.
HTC systems need to be robust and to reliably operate over a long time scale.
Traditionally, computing grids composed of heterogeneous resources (clusters, workstations, and volunteer desktop machines) have been used to support HTC.
HTC
aims to bridge the gap between HPC and HTC.
MTC is similar to HTC, but it concentrates on the use of many computing resources over a short period of time to accomplish many computational tasks.
In brief, MTC denotes high-performance computations comprising multiple distinct activities coupled via file system operations.
MTC
Some popular software systems are ◦ Condor◦ Globus Toolkit ◦ Sun Grid Engine (SGE)◦ BOINC ◦ Nimrod/G ◦ Aneka.
Frameworks for task computing
Condor ◦ most widely used and long-lived middleware for
managing clusters, idle workstations, and a collection of clusters.
Sun Grid Engine (SGE)◦ Tool for managing heterogeneous resources and
constitutes middleware for grid computing.
Frameworks (contd……)
The Globus Toolkit ◦ collection of technologies that enable grid
computing.◦ provides a comprehensive set of tools for sharing
computing power, databases, and other services. Nimrod/G
◦ tool for automated modeling and execution of parameter sweep applications over global computational grids.
Contd…….
Berkeley Open Infrastructure for Network Computing (BOINC) ◦ framework for volunteer and grid computing. ◦ allows us to turn desktop machines into volunteer
computing nodes that are leveraged to run jobs when such machines become inactive.
Contd…………
Embarrassingly parallel applications Parameter sweep applications MPI applications Workflow applications with task
dependencies
Task-based application models
Constitute a collection of tasks that are independent from each other and that can be executed in any order.
The tasks might be of the same type or of different types, and they do not need to communicate among themselves.
Supporting frameworks are Globus Toolkit, BOINC, and Aneka.
Embarrassingly parallel applications
Parameter sweep applications are a specific class of embarrassingly parallel applications for which the tasks are identical in their nature and differ only by the specific parameters used to execute them.
Any distributed computing framework that provides support for embarrassingly parallel applica- tions can also support the execution of parameter sweep applications, since the tasks composing the application can be executed independently of each other.
Parameter sweep applications
Message Passing Interface (MPI) is a specification for developing parallel programs that communi- cate by exchanging messages. Compared to earlier models, MPI introduces the constraint of com- munication that involves MPI tasks that need to run at the same time.
MPI applications
Task Programming Model
Figure:- Task Programming Model Scenario
Developing Application with Task Model
Steps:
1. Define Classes With Itask Interface
2. Create a properly Configured AnekaApplication instance
3. Create ITask instances and wrap them into AnekaTask instances
4. Execute the application and wait for its completion
ITask Interface Namespace Aneka.Tasks
{
//Codes public interface ITask {
//Codes public void Execute(); }
}
ITask Interface Implementaion Using System;
Using Aneka.Tasks;
namespace GaussSample
{
[Serializable] public class GaussTask:ITask
{
private double x;
public double X{ get {return this.x; } set {this.x=value;} }
private double y;
public double Y{ get {return this.y; } set {this.y=value;} }
public void Execute()
{
this.y=Math.Exp(-this.x*this.x);
}
}
}
Wrapping an ITask into an Aneka Task Interface
GaussTask gauss=new GaussTask();
AnekaTask task=new AnekaTask(gauss);
task.AddFile("input.txt", FileDataType.Input, FileAttributes.Local);
task.AddFile("result.txt", FileDataType.Output, FileAttributes.Local);
Controlling Task Execution
Computation Logic : Task Classes, AnekaTask
Coordination logic : AnekaApplication Class
AnekaApplication Class Provides following operations:
1. Static and Dynamic task submission
2. Application state and task state monitoring
3. Event based notification of tasks completion or failure.
Task Submission Static Submission
◦ Creation of all tasks to be executed in one one loop and submission as single bag
Dynamic Submission◦ Submission as a result of event-based notification mechanism
Static Submission Configuration conf=new Configuration.GetConfiguration("conf.xml");
conf.SingleSubmission=true;
AnekaApplication<AnekaTask, TaskManager> app=new AnekaApplicatoin<Task, TaskManager>(conf); for(int i=0;i<400;i++)
{
GaussTask gauss=new GaussTask();
gauss.X=i;
AnekaTask task=new AnekaTask(gauss);
app.AddWorkUnit(task);
}
app.SubmitExecute();
Dynamic Submission Events:
WorkUnitFailed
WorkUnitCompleted
File ManagementFiles consists of
◦ Input Data◦ Result of Computation◦ Executable codes or library dependencies
WorkUnit and ApplicationBase Classes for file management
Using FileData Class, the user specifies the file dependencies of tasks and application.
Task Libraries Ready to use tasks
1. File Copy
2. Substitute Operation
3. File Deletion
4. Timed Delay
7. Task Composition
8. Legacy Application Execution
Web Services Integration
Integration with other technologies and applications by means of web services.
Operations supported through web service:
1. Local file copy on the remote node
2. File deletion
3. Parameter substitution
3. Legacy Application execution through the common shell services
ANEKA A PLATFORM FOR ENTERPRISE GRID/CLOUD COMPUTING 29
Anatomy of a Cloud
IAASData Centers
ClustersStorage
Other Grids/Clouds
Virtualization
VM Management & Deployment
Amazon S3, EC2
OpenNebula, Eucalyptus
Mosso
SAASWeb 2.0 Interface
Programming API
Scripting & ProgrammingLanguages
PAASGoogle AppEngine
Microsoft Azure
Manjrasoft Aneka
Google Apps (Gmail, Docs,…)
Salesforce.com
QoS
Met
erin
g
SLA
Neg
otiati
onAd
mis
sion
Con
trolPr
icin
g an
d Bi
lling
Acco
untin
g
Public Cloud
Private Cloud
Socia
l Net
wor
ks
Scie
ntific
Com
putin
g
Amus
emen
t
CDNs
Fina
ncia
l App
licati
ons
ANEKA A PLATFORM FOR ENTERPRISE GRID/CLOUD COMPUTING 30
Aneka & Clouds
Infrastructure as a Service
Platform as a Service
Software as a Service
Aneka fits into the cloud architecture at the platform layer. This means that it provides a programming based interface for developing distributed application and a virtual execution environment in which
the applications developed according to the published APIs can run.
Public CloudPrivate Cloud
Aneka
Aneka Architecture
ANEKA A PLATFORM FOR ENTERPRISE GRID/CLOUD COMPUTING 31
System Overview
Executor
Scheduler
Executor
Executor Executor
Manager
work units
internet
internet
Aneka enterprise Cloud
Manager
work units
Manager(s)
Client Applications
Workers
Aneka Container
Further on Aneka Aneka supports development of parameter sweep applications.
Aneka can be used to manage workflows through plug ins.
Summary Task based programming is most intuitive approach for distributing the computation of an application over set of nodes
Task Model Comprises set of services(directory, scheduling, execution and storage)
References 1. Buyya, Selvi, Vecchiola. Mastering Cloud Computing. 1st. New Delhi: Mc Graw Hill, 2013.