17 march 2008standards for interoperable grids 1 job execution standards for interoperable grids:...
TRANSCRIPT
17 March 2008 Standards for Interoperable Grids 1
Job Execution
Standards for Interoperable Grids: Experience from NextGRID and OMII-Europe
Clive Davenhall
National e-Science Centre, University of Edinburgh
Standards for Interoperable Grids 2
Job Execution: Introduction Running jobs on a Grid.
Topics:
Naming and addressing (brief introduction; needed by this talk and the next)
OGSA-EMS
OGSA-BES
JSDL
Standards for Interoperable Grids 3
NextGRID Naming and Addressing NextGRID uses WS-Addressing and WS-Naming.
WS-Addressing W3C standard. Way of addressing components, services, data items etc. De facto standard in the Web services community.
WS-Naming OGF Standard. Extends WS-Addressing. By adding optional naming elements.
NextGRID Has some additional extensions and restrictions.
Standards for Interoperable Grids 4
Job Execution
Topics:
Naming and addressing
OGSA-EMS
OGSA-BES
JSDL
Standards for Interoperable Grids 5
OGSA-EMS EMS: Execution Management Services
An architecture for initiating, managing and controlling the execution of ‘activities’ or ‘units of work.’
Units of work might be: OGSA applications. Legacy applications running in an OGSA harness.
An architecture not a standard So in the OGSA ‘informational’ rather than ‘recommendation’
track.
Standards for Interoperable Grids 6
Role and Status of OGSA-EMS OGSA uses EMS to identify and position standards that
address part of the ‘activity management’ problem. Tool or mechanism for identifying necessary standards and their
interrelations.
This effort continues.
Not all the necessary standards are yet in place.
Initial focus on fundamental job execution.
Where possible OGSA adopts or adapts existing standards, Rather than inventing new ones ab initio.
Standards for Interoperable Grids 7
Problems addressed by EMS Finding possible locations where activities can
execute.
Selecting the execution location for an activity.
Preparing for execution.
Initiating the execution.
Etc…
Standards for Interoperable Grids 8
EMS Standards BES
Basic Execution Service
JSDL Job Submission Description Language
CDDLM Deployment API Configuration, Description, Deployment and Lifecycle Management
Deployment API
CDDLM CDL Configuration, Description, Deployment and Lifecycle Management
Configuration Description Language
ACS Applications Content Service
RSS Resource Selection Services
Standards for Interoperable Grids 9
EMS Standards BES
Basic Execution Service
JSDL Job Submission Description Language
CDDLM Deployment API Configuration, Description, Deployment and Lifecycle Management
Deployment API
CDDLM CDL Configuration, Description, Deployment and Lifecycle Management
Configuration Description Language
ACS Applications Content Service
RSS Resource Selection Services
Standards for Interoperable Grids 10
Job Execution
Topics:
Naming and addressing
OGSA-EMS
OGSA-BES
JSDL
Standards for Interoperable Grids 11
OGSA-BES BES: Basic Execution Service
Defines Web service interfaces for creating, monitoring and controlling computational entities: Units of work. Activities.
Individual activities to be performed are defined in JSDL.
Activities execute on a suitable computational resource. Agnostic about the underlying environment on which the activity
executes.
BES interface is extensible.
Standards for Interoperable Grids 12
BES Port-types
BES-Management Allows clients to monitor and control a BES instance. Intended for system administrators.
BES-Factory Allows clients to create, monitor and control sets of
activities.
BES-Activity Allows clients to create, monitor and control individual
activities.
Standards for Interoperable Grids 13
Extensibility State Model
A set of defined states and the transitions between them. BES defines a basic mode and allows extensions.
Information Model Allows clients to examine aspects of a BES instance and its
applications. Allows mandatory, optional and user-defined attributes.
Resource Model Specifies basic operations, but allows additional ones. Specifically allows various OGSA interaction modes.
Standards for Interoperable Grids 14
Basic State Model
Pending Finished
Failed
Running
Terminated
System error/failure event
Request to terminate activity
Activity terminatessuccessfully
Standards for Interoperable Grids 15
Basic State Model
Allows extensibility by specialisation.
Pending Finished
Failed
Running
Terminated
System error/failure event
Request to terminate activity
Activity terminatessuccessfully
Standards for Interoperable Grids 16
BES-Factory Attributes IsAcceptingNewActivities CommonName LongDescription TotalNumberOfActivities ActivityReference TotalNumberOfContainedResources ContainedResource NamingProfile BESExtension LocalResourceManagerType ResourceName OperatingSystem CPUArchitecture CPUCount CPUSpeed PhysicalMemory VirtualMemory
Standards for Interoperable Grids 17
Operations Provided by Port-types
BES-Management StopAcceptingNewActivities StartAcceptingNewActivities
BES-Factory CreateActivity GetActivityStatuses TerminateActivities GetActivityDocuments GetFactoryAttributesDocument
Standards for Interoperable Grids 18
Operations Provided by Port-types
BES-Management StopAcceptingNewActivities StartAcceptingNewActivities
BES-Factory CreateActivity GetActivityStatuses TerminateActivities GetActivityDocuments GetFactoryAttributesDocument
Standards for Interoperable Grids 19
BES-Factory CreateActivity Used to request the creation of a new activity.
In its simplest form:
<bes:ActivityDocument><jsdl:JobDefinition>
…</jsdl:JobDefinition><xsd:any/>*
</bes:ActivityDocument>
Standards for Interoperable Grids 20
BES-Factory CreateActivity Used to request the creation of a new activity.
In its simplest form:
<bes:ActivityDocument><jsdl:JobDefinition>
…</jsdl:JobDefinition><xsd:any/>*
</bes:ActivityDocument>
Standards for Interoperable Grids 21
Job Execution
Topics:
Naming and addressing
OGSA-EMS
OGSA-BES
JSDL
Standards for Interoperable Grids 22
JSDL JSDL: Job Submission Description Language.
Not an OGSA standard; but adopted by OGSA.
Abstract description of the requirements for a job, activity or unit of work.
Can contain: Generic job identification information. Application description. Resource requirements (main focus is on computational jobs). Description of any data files required.
Describes what the job is to do, not how to do it.
Standards for Interoperable Grids 23
JSDL and BES JSDL is an abstract description of a job or activity.
It deliberately does not define a job submission interface.
And it is therefore independent of any submission interface.
BES provides a submission interface.
BES
Container JSDL
Standards for Interoperable Grids 24
Extensions
An extension for applications executing on a POSIX-compliant operating system is included in the basic specification.
Two additional extensions have subsequently been defined: JSDL HPC (High Performance Computing) extension. JSDL SPMD (Single Program Multiple Data) extension.
Additional extensions are anticipated.
Standards for Interoperable Grids 25
JSDL Document Structure
<JobDefinition><JobDescription>
<JobIdentification ... />?<Application ... />?<Resources... />?<DataStaging ...
/>*</JobDescription><xsd:any##other/>
</JobDefinition>
Note: None [1..1] ? [0..1] * [0..n] + [1..n]
Note: None [1..1] ? [0..1] * [0..n] + [1..n]
Standards for Interoperable Grids 26
JSDL Document Structure
<JobDefinition><JobDescription>
<JobIdentification ... />?<Application ... />?<Resources... />?<DataStaging ...
/>*</JobDescription> <xsd:any##other/>
</JobDefinition>
Job Identification
Application description
Resource requirements
Data files required
Note: None [1..1] ? [0..1] * [0..n] + [1..n]
Note: None [1..1] ? [0..1] * [0..n] + [1..n]
Standards for Interoperable Grids 27
Job Identification Element
<JobDefinition><JobDescription>
<JobIdentification ... />?<Application ... />?<Resources... />?<DataStaging ...
/>*</JobDescription> <xsd:any##other/>
</JobDefinition>
<JobIdentification><JobName ... />?<Description ... />?
<JobAnnotation ... />?<JobProject ... />? <xsd:any##other/>
</JobIdentification>
Standards for Interoperable Grids 28
Job Identification Example
<JobDefinition><JobDescription>
<JobIdentification ... />?<Application ... />?<Resources... />?<DataStaging ...
/>*</JobDescription> <xsd:any##other/>
</JobDefinition>
<JobIdentification><JobName>
My Job</JobName><Description>
Simple application</Description>…
</JobIdentification>
Standards for Interoperable Grids 29
Hello World in JSDL
<?xml version="1.0" encoding="UTF-8"?><jsdl:JobDefinition xmlns:jsdl=“http://schemas.ggf.org/2005/11/jsdl” xmlns:jsdl-posix= “http://schemas.ggf.org/jsdl/2005/11/jsdl-posix”><jsdl:JobDescription> <jsdl:Application> <jsdl-posix:POSIXApplication> <jsdl-posix:Executable> /bin/echo <jsdl-posix:Executable> <jsdl-posix:Argument>hello</jsdl-posix:Argument> <jsdl-posix:Argument>world</jsdl-posix:Argument> </jsdl-posix:POSIXApplication> </jsdl:Application> </jsdl:JobDescription></jsdl:JobDefinition>
Standards for Interoperable Grids 30
NextGRID Recommendations
BESReiterates that extensions to WS-Naming are
permitted, specifically the ones required by NextGRID naming.
JSDLDefines extensions to support data-staging
which is compatible with NextGRID data transfer.