fujitsu laboratories of europe © 2004introducing profiles to ogsaintroducing profiles to ogsa...

Fujitsu Laboratories of Europe Fujitsu Laboratories of Europe ©© 20042004

Current Status and Future Current Status and Future Direction of UNICORE and OGSADirection of UNICORE and OGSA

NAREGI SymposiumNAREGI SymposiumFebruary 23February 23--24, 200524, 2005Dr. David F. SnellingDr. David F. SnellingFujitsu Labs EuropeFujitsu Labs Europe


UniGrids ObjectivesUniGrids Objectives

Develop:Develop:An OGSA compliant Unicore ImplementationA framework and technology for Grid interoperabilityA framework for Grid economics

Create Generic Services forCreate Generic Services forVisualisation and steering of simulationsDevice monitoring and controlAccess to distributed data and databasesApplication instrumentation

Influence selected Grid standards and promote Influence selected Grid standards and promote European research resultsEuropean research results


What Does my Title Mean?What Does my Title Mean?

Current Status and Future Direction of Current Status and Future Direction of Unicore and OGSAUnicore and OGSA

1133OGSAOGSA

2244UnicoreUnicore

FutureFutureStatusStatus

Current Current StatusStatus and and FutureFuture Direction of Direction of UnicoreUnicore and and OGSAOGSA

44

22 33

11 22

4433

22

1133

11 44


OGSA: The FutureOGSA: The Future

OGSA Architecture V2.0 DocumentOGSA Architecture V2.0 DocumentSame scope, more refined than V1.0Possibly only a V1.5, due to effort on OGSA Profiles

OGSA ProfilesOGSA ProfilesBasic Profile

WSRF, WSN (Base Notification only), & WS-AddressingSecurity Profile

Transport Security, WS-Security, ???Basic Execution Profile

JSDL, Basic Execution Service, WS-Agreement?, WSDM?Naming ProfileCandidate Execution and Data Profiles


Introducing Profiles to OGSAIntroducing Profiles to OGSA

ProfilesProfilesRefine and clarify referenced specificationsGoals: Interoperability, Interoperability, InteroperabilityReference point for “claims” of OGSA compliance

No compliance suites plannedProfile TypesProfile Types

Full Profiles: GGF Recommendations TrackBased on stable standards

Candidate Profiles: GGF Informational TrackBased on proposed and evolving specifications


Full ProfileFull Profile

Required for Required for ““Proposed RecommendationProposed Recommendation”” StatusStatusContains specifications that are at least on a standards track

E.g. GGF Proposed Recommendation, OASIS Committee Draft, etc.

Commitments from multiple implementers to continue development of evolving specifications.

Required for Required for ““RecommendationRecommendation”” StatusStatusContains completed standard specifications only.

De facto standards permitted, e.g. WSDL 1.1.Two or more independently developed, interoperable implementationsExperience document must published


Candidate ProfilesCandidate Profiles

Informational Document in GGFInformational Document in GGF ProcessProcess““CandidateCandidate”” Status isStatus is

A function of status of any referenced specificationsE.g. with respect to the standardization process

Not a function of the importance of the profile to OGSA.

Restrictions on Referenced SpecificationsRestrictions on Referenced SpecificationsOnly references specifications are in standards development.At least one reference prototype of each specification exists.Specifications may be evolving.Commitments for multiple implementations of the final specification.


Unicore FutureUnicore Future


1133OGSAOGSA

2244UnicoreUnicore



Unicore PrinciplesUnicore Principles

Seamless ComputingSeamless ComputingHigh SecurityHigh SecurityPortable ServerPortable ServerComplex WorkflowsComplex WorkflowsFile ManagementFile ManagementAdvanced Client InterfaceAdvanced Client InterfaceExtensibilityExtensibility

RetainTheseIdeals


Unicore: The FutureUnicore: The Future

OGSA ProfilesOGSA ProfilesBasic ProfileSecurity ProfileBasic Execution ProfileNaming Profile

Unicore Grid Programming EnvironmentUnicore Grid Programming EnvironmentVirtual Resource FactoryJob and File ManagementWS Based File Transfer

Globus InteroperabilityGlobus InteroperabilityIssues: WSRF Version and GSI Interfaces


Unicore: The Future ContinuedUnicore: The Future Continued

Virtual OrganizationsVirtual OrganizationsDynamic Trust Management

SAML and XACML BasedDistributed Trust Base

New Gateway DesignNew Gateway DesignTentative ideas only, Please feedbackHttps as core protocol, not Java only UPLWS-Addressing enabledGateway “Plug-in” support improved


Unicore: GPE ArchitectureUnicore: GPE Architecture

WSRF-enabledServers

WSRF-enabledStorage

WSRF-enabledNetwork

Web Service ResourceFramework (WSRF)

Open Grid Services Architecture

U-GPEHigh-level Grid API

OperationsJob managementFile transfersBrokeringSteering, etc.

DescriptionsResources (CIM)Jobs (JSDL)Workflows (BPEL)

Unicore Atomic Services

Grid Programming Library

Grid SDK Grid Beans Client Framework

Applications


Unicore Atomic ServicesUnicore Atomic Services

Atomic Services

Job Management

Service(JMS)

ImplementationW

S-*

Managejobs on target system

Grid Programming LibraryTSS

ClientJMS

ClientSMS

ClientFIS

ClientFES

Client

U-GPE (or other) Clients

Target System Service (TSS)

Implementation

Storage Management

Service(SMS)

Implementation

FileImport Service

(FIS)

Implementation

FileExport Service(FES)

Implementation

Manage target system

Managefiles on storage

Manageimports to storage

Manageexports from storage


Unicore GPE Grid ServicesUnicore GPE Grid Services

Target System FactoryTarget System FactoryCreate a virtual compute resource for a VO.

Target System ServiceTarget System ServiceJob Submit (JSDL and AJO)Resource Description Properties

Job Management ServiceJob Management ServiceBasic control and status properties

File Management ServiceFile Management ServiceCopy, Delete, List Directory, ...

File Import/Export ServicesFile Import/Export ServicesWS Based File Transfer Capability


Atomic Services DetailAtomic Services DetailOperations Service Resource Type Properties

Job Management Service (JMS)

Extends WS-RP/LTstart()abort()hold()resume()

File Import Service (FIS)Extends WS-RP/LT

putChunk(offset)

File Export Service (FES)

Extends WS-RP/LTgetChunk(offset, size)

Target System Service (TSS)

Extends WS-RP/LTJob-EPR submit(JSDL, initialTT)

Job3Job2

Status=RunningOriginalJSDL=<JSDL..>ExecutionJSDL=<JSDL..>etc.Job1

create

FileImport 1

FileImport 2

Sourcefile=c:/tmp/test.txtDestFile=/tmp/test.txtTransferred=300kbetc.

FileExport 1

FileExport 2

Sourcefile=/tmp/output.txt DestFile=c:/tmp/output.txtTransferred=12kbetc.

create

Target System Factory (TSF)

Target System

Memory=1GbRunning Jobs=23Storage=Temp, Rootetc.

create

Storage Management Service (SMS)

Extends WS-RP/LTlistDirectory(path)listFile(path)createDirectory(path)changePermissions(path, perm)copy(path, path)rename(path, path)delete(path)import(path, protocol, isPipe)export(path, protocol, isPipe)

HomeRoot

TS-Property

Temp

AvailableFileSpace=127kbSupportedProtocols=

{GridFTP, UPL, scp}etc.

Work

create

TSR-EPR create(type, config)


OGSA StatusOGSA Status


1133OGSAOGSA

2244UnicoreUnicore



OGSA-WG: Current StatusOGSA-WG: Current Status

OGSA Version 1.0 and Glossary PublishedOGSA Version 1.0 and Glossary PublishedOGSA WG Focus on ProfilesOGSA WG Focus on Profiles

Base and Security Profiles - Drafts on GridForgeContribute NOW!

Base Execution Profile Draft after GGF13

OGSA Spawned OGSA Spawned WGsWGsByteIO, Basic Execution Services, and Naming

Attend BoFs at GGF13

OGSA Candidate Profiles Starting Work NowOGSA Candidate Profiles Starting Work NowAdvanced Execution Services (SuperScheduler, Broker, ...)Base and Advanced Data Services Profiles


WSRF/WSN/WSDM: StatusWSRF/WSN/WSDM: Status

WSRFWSRFResource Properties and Lifetime ready for Committee DraftService Group in processRenewable References moved out of scope

WSNWSNBase Notification stable & seeking synergy with WS-EventingIncludes “pull” based Notification

Important to NAREGI and UniGrids because of Unicore modelWSDMWSDM

Committee Draft(Based of a different version of WSRF than the CD)

HP/Globus/Apache Incubator AnnouncementHP/Globus/Apache Incubator AnnouncementApollo, Hermes, Muse incubatorshttp://wiki.apache.org/incubator/<name>Proposal


JSDL: StatusJSDL: Status

Job Submission Description LanguageJob Submission Description LanguagePre-stage, Execute, Post-stage.Uniformity across batch subsystems.Support for multiple file transfer protocols.

Not a Job Management Interface!Not a Job Management Interface!A job description document with a processing semantics

CapabilitiesCapabilitiesOptional data sources/sinks, with different credentials possibleSeamless or restricted resource descriptionSuitable for resource brokers and schedulers

Act as JSDL to JSDL processorsRestrictionsRestrictions

No support for “Multi-Jobs” or work-flow.


Schema OverviewSchema Overview

<<JobDefinitionJobDefinition>>

<<JobDescriptionSectionJobDescriptionSection>>

<<JobIdentification JobIdentification ... />?... />?

<User <User id="xsdid="xsd::uriuri””... /> ?... /> ?

<Application <Application id="xsdid="xsd::uri" uri" ... /> ?... /> ?

<Resource <Resource id="xsdid="xsd::uriuri””... />*... />*

<<DataStaging id="xsdDataStaging id="xsd::uriuri””... />*... />*

<Limits <Limits id="xsdid="xsd::uriuri””... />?... />?

<</JobDescriptionSection/JobDescriptionSection>>

<Profile><Profile>

<Description ... />?<Description ... />?

<User <User id="xsdid="xsd::uriuri””... />?... />?

<Application <Application id="xsdid="xsd::uri" uri" ... />?... />?

<Resource <Resource id="xsdid="xsd::uriuri””... />*... />*

<<DataStaging id="xsdDataStaging id="xsd::uriuri””... />*... />*

<Limits <Limits id="xsdid="xsd::uriuri””... />?... />?

</Profile>*</Profile>*

<</JobDefinition/JobDefinition>>


JSDL ExampleJSDL Example<<JobDefinitionJobDefinition>>

<<JobDescriptionSectionJobDescriptionSection>>

<<JobIdentification/JobIdentification/>>

<Application id=<Application id=””Daves Daves App">App">

<<ApplicationName ApplicationName ““NanoCodesRSNanoCodesRS””/>/>

<<ApplicationVersion ApplicationVersion ““V1.7aV1.7a””/>/>

</Application></Application>

<</JobDescriptionSection/JobDescriptionSection>><Profile id=<Profile id=““Profile1Profile1””>>

<Resource id=<Resource id=”” LargeSystemLargeSystem””>>

<<ResourceCount ResourceCount ““6464””/>/>

<<CPUCount CPUCount ““88””/>/>

</Resource></Resource>

</Profile></Profile>

<Profile id=<Profile id=““Profile2Profile2””>>

<Resource id=<Resource id=”” SmallSystemSmallSystem””>>

<<ResourceCount ResourceCount ““6464””/>/>

<<CPUCount CPUCount ““88””/>/>

</Resource></Resource>

</Profile></Profile>

<</JobDefinition/JobDefinition>>


OGSA Specification OverviewOGSA Specification Overview

SYSTEMSMANAGEMENT

UTILITYCOMPUTING

GRIDCOMPUTING

WS-Addressing

Privacy

WS-Base Notification

CIM/JSIM

WSRF-RAP

WSDM

WS-Security

Naming

OGSA-EMSOGSA Self Mgmt

GFD-C.16

GGF-UR

Data Model

HTTP(S)/SOAP

GRID Computing, Distributed Computing and Utility Computingare different views of the same important problem domain.

Discovery

SAML/XACML

WSDL

WSRF-RL

Trust

WS-DAI

VO Management

Information

Distributed query processing

ASP

Data CentreUse Cases &Applications Collaboration Multi MediaPersistent Archive

Data Transport

WSRF-RP

X.509

Core Services

InfrastructureSpecifications


Status LegendStatus Legend

StandardStandardExisting, adopted specification from a recognized Standards Development Organization.

EvolvingEvolvingEmerging specifications with reference implementations and standardization in process

GapGapIdentified architectural component with no existing specifications complete, but work is in progress somewhere.

HoleHoleIdentified architectural component without apparent activity.


OGSA Architecture: StatusOGSA Architecture: Status

SYSTEMSMANAGEMENT

UTILITYCOMPUTING

GRIDCOMPUTING

Core Services

InfrastructureSpecifications

WS-Addressing

Privacy

WS-BaseNotification

CIM/JSIM

WSRF-RAP

WSDM

WS-Security

Naming

OGSA-EMSOGSA Self Mgmt

GFD-C.16

GGF-UR

Data Model

HTTP(S)/SOAP

Discovery

SAML/XACML

WSDL

WSRF-RL

Trust

WS-DAI

VO Management

Information

Distributed query processing

ASP

Data CentreUse Cases &Applications Collaboration Multi MediaPersistent Archive

Data Transport

WSRF-RP

X.509

StandardEvolvingGapHole


Unicore StatusUnicore Status


1133OGSAOGSA

2244UnicoreUnicore



Unicore: Current StatusUnicore: Current Status

Security InfrastructureSecurity InfrastructureExplicit Trust Delegation Framework - Done

WS InfrastructureWS InfrastructurePrototype WSRF Plus job and file management interfaces

Shipped in NovemberV1.0 WSRF by end March 2005V1.0 Unicore Atomic Services by end March 2005

Includes JSDL based job description

WS Based File TransferWS Based File TransferV1.0 by mid AprilV2.0 next year


ConclusionsConclusions

Progress is GoodProgress is GoodOGSA is moving aggressively, backed by GGF.WSRF is approaching completionNAREGI/UniGrids WSRF Infrastructure ready soonJSDL’s progress is due to NAREGI/UniGrids Team

Aim for a repeat performance on Basic Execution ServiceNew Tools TooNew Tools Too

Basic execution services for NAREGI requirementsWS Based File Transfer and New Gateway DesignVO support coming (Tell us your requirements)

High Degree of CollaborationHigh Degree of CollaborationNAREGI, UniGrids, Globus, EGEE, Condor, ...

fujitsu laboratories of europe © 2004introducing profiles to ogsaintroducing profiles to ogsa...

Documents