The Open Grid Services Architecture, Version 1.0I. Foster, H. Kishimoto, A. Savva, D. Berry, A. Djaoui, A. Grimshaw, B. Horn, F. Maciel,

F. Siebenlist, R. Subramaniam, J. Treadwell, J. Von Reich

Presented by Oscar Valdivia

OGSA Introduction

GridGrid systems and applications aim to integrate, virtualize, and manage resources and services within distributed, heterogeneous, dynamic “virtual organizations”

Items needed Computers, application services, data, and other resources need to be accessed

within different organizations Standardization

Open Grid Services Architecture (OGSA)Is a service-oriented architecture (SOA), that addresses the need for standardization by defining a set of core capabilities and behaviors that address key concerns in Grid systems

SOA:A perspective of software architecture that defines the use of services to support the requirements of software users. Enables the creation of applications that are built by combining loosely coupled and interoperable services

wikipedia.com

OGSA Document Outline

RequirementsBased on requirements, use cases, technical challenges, previous experience, state of the art in related work.

CapabilitiesTranslates the requirements into a coherent set of capabilities that collectively define OGSA.

1. Describe infrastructure services and assumptions that constrain development of the OGSA design.

2. Refinement of the required functionality into capabilities: Execution Management, Data, Resource Management, Security, Self-Management and Information services.

Requirements

Base input on the following Use Cases:

Commercial Data Center (CDC) Virtual Organization (VO) Grid Portal

Severe Storm Modeling Persistent Archive

Online Media and Entertainment Mutual Authorization

National Fusion Collaboratory (NFC) Resource Usage Service

Service-Based Distributed Query Processing IT Infrastructure and Management

Grid Workflow Application Use Cases

Grid Resource Reseller Reality Grid

Inter Grid The Learning GRID

Interactive Grids HLA-based Distributed Simulation

Grid Lite GRID based ASP for Business

Grid Monitoring Architecture

http://www.gridforum.org/documents/GFD.29.pdf

Requirements - Characteristics

Interoperability and Support for Dynamic and Heterogeneous Environments

Resource virtualization.

Resource Sharing Across Organizations Common management capabilities

Optimization

Quality of Service (QoS) Assurance Resource discovery and query

Job Execution

Data Services Standard protocols and schemas

Security

Administrative Cost Reduction

Scalability

Availability

Ease of Use and Extensibility

Interoperability and Support for Dynamic and Heterogeneous Environments

Resource Sharing Across Organizations

Optimization

Quality of Service (QoS) Assurance

Job Execution

Data Services

Security

Administrative Cost Reduction

Scalability

Availability

Ease of Use and Extensibility

Interoperability and Support for Dynamic and Heterogeneous Environments

Global name space

Resource Sharing Across Organizations Metadata services

Optimization

Quality of Service (QoS) Assurance Site autonomy

Job Execution

Data Services Resource usage data

Security

Administrative Cost Reduction

Scalability

Availability

Ease of Use and Extensibility

Analysis of the previous use cases led to identify the following characteristics

Interoperability and Support for Dynamic and Heterogeneous Environments

Resource Sharing Across Organizations Suppliers-Side Optimization

Optimization

Quality of Service (QoS) Assurance Consume-Side Optimization

Job Execution

Data Services

Security

Administrative Cost Reduction

Scalability

Availability

Ease of Use and Extensibility

Interoperability and Support for Dynamic and Heterogeneous Environments

Service level agreement

Resource Sharing Across Organizations Service level attainment

Optimization

Quality of Service (QoS) Assurance Migration

Job Execution

Data Services

Security

Administrative Cost Reduction

Scalability

Availability

Ease of Use and Extensibility

Interoperability and Support for Dynamic and Heterogeneous Environments

Support for various job types

Resource Sharing Across Organizations Scheduling

Optimization

Quality of Service (QoS) Assurance Resource provisioning

Job Execution

Data Services

Security

Administrative Cost Reduction

Scalability

Availability

Ease of Use and Extensibility

Interoperability and Support for Dynamic and Heterogeneous Environments

Data access

Resource Sharing Across Organizations Data consistency

Optimization

Quality of Service (QoS) Assurance Data persistency

Job Execution

Data Services Data integration

Security

Administrative Cost Reduction Data location management

Scalability

Availability

Ease of Use and Extensibility

Interoperability and Support for Dynamic and Heterogeneous Environments

Authentication and authorization

Resource Sharing Across Organizations Multiple security infrastructures

Optimization

Quality of Service (QoS) Assurance Isolation

Job Execution

Data Services Delegation

Security

Administrative Cost Reduction Security policy exchange

Scalability

Availability Intrusion detection, protection, and secure logging

Ease of Use and Extensibility

Interoperability and Support for Dynamic and Heterogeneous Environments

Policy-based management

Resource Sharing Across Organizations Application contents management

Optimization

Quality of Service (QoS) Assurance Problem determination

Job Execution

Data Services

Security

Administrative Cost Reduction

Scalability

Availability

Ease of Use and Extensibility

Interoperability and Support for Dynamic and Heterogeneous Environments

Management architecture

Resource Sharing Across Organizations

Optimization High-throughput computing

Quality of Service (QoS) Assurance

Job Execution

Data Services

Security

Administrative Cost Reduction

Scalability

Availability

Ease of Use and Extensibility

Interoperability and Support for Dynamic and Heterogeneous Environments

Disaster recovery

Resource Sharing Across Organizations

Optimization Fault management

Quality of Service (QoS) Assurance

Job Execution

Data Services

Security

Administrative Cost Reduction

Scalability

Availability

Ease of Use and Extensibility

Interoperability and Support for Dynamic and Heterogeneous Environments

Mask Complexity

Resource Sharing Across Organizations Users can choose level of interaction

Optimization

Quality of Service (QoS) Assurance Extensible policies and mechanisms

Job Execution

Data Services Extensible and replaceable core system

Security

Administrative Cost Reduction

Scalability

Availability

Ease of Use and Extensibility

Capabilities - OGSA Overview

OGSA is intended to facilitate the seamless use and management of distributed, heterogeneous resources.

Three major logical and abstract tiers.

HardwareData

Storage

LicensesApplicationServices

Sensors

Networks

Operating Systems

Software

Information Management

SLA Management

Security Framework

Unified Interface

Resource Management

Optimization Framework

Monitoring & Analytics

Grid

Infrastructure

Value Add Software

User Domain Applications

User Frameworks

Conceptual view of Grid infrastructure

HardwareData

Storage

LicensesApplicationServices

Sensors

Networks

Operating Systems

Software

Information Management

SLA Management

Security Framework

Unified Interface

Resource Management

Optimization Framework

Monitoring & Analytics

Grid

Infrastructure

Value Add Software

User Domain Applications

User Frameworks

OGSA FOCUS

HardwareData

Storage

LicensesApplicationServices

Sensors

Networks

Operating Systems

Software

Information Management

SLA Management

Security Framework

Unified Interface

Resource Management

Optimization Framework

Monitoring & Analytics

Grid

Infrastructure

Value Add Software

User Domain Applications

User Frameworks

OGSA FOCUS

OGSA RELEVANCE

HardwareData

Storage

LicensesApplicationServices

Sensors

Networks

Operating Systems

Software

Information Management

SLA Management

Security Framework

Unified Interface

Resource Management

Optimization Framework

Monitoring & Analytics

Grid

Infrastructure

Value Add Software

User Domain Applications

User Frameworks

OGSA FOCUS

OGSA RELEVANCE

Macro (System Level) Quality of Service

Capabilities - OGSA Framework

Capabilities - OGSA Infrastructure

Define a coherent and integrated set of components that collectively address the requirements identified previously. Within SOA context.

OGSA Builds and contribute Web Services Architecture [WS-Architecture].

Service interfaces are defined by the Web Services Description Language (WSDL).

XML for description and representation SOAP as the primary message exchange format for OGSA services WS-I for service definition WS-Security WS Resource Framework (WSRF) for state representation and

manipulation WS-Notification

OGSA Capabilities

Security• Cross-organizational users• Trust nobody• Authorized access only

Security• Cross-organizational users• Trust nobody• Authorized access only

Information Services• Registry• Notification• Logging/auditing

Information Services• Registry• Notification• Logging/auditing

Execution Management• Job description & submission• Scheduling• Resource provisioning

Execution Management• Job description & submission• Scheduling• Resource provisioning

Data Services• Common access facilities• Efficient & reliable transport• Replication services

Data Services• Common access facilities• Efficient & reliable transport• Replication services

Self-Management• Self-configuration• Self-optimization• Self-healing

Self-Management• Self-configuration• Self-optimization• Self-healing

Resource Management• Discovery• Monitoring• Control

Resource Management• Discovery• Monitoring• Control

OGSAOGSA

OGSA “profiles”OGSA “profiles”

Web services foundation Web services foundation

Source: OGSA keynotes 20060510 H. Kishimoto

CDLCDL

3. Select from or deployrequired resources

3. Select from or deployrequired resources

Execution Management The basic problem

Execute and manage jobs/services in the grid Select from or provision required resources

The basic problem Execute and manage jobs/services in the grid Select from or provision required resources

2. Submit the job2. Submit the job

1. Describe the job1. Describe the job

JSDLJSDL

Job

4. Manage the job4. Manage the job

Source: OGSA keynotes 20060510 H. Kishimoto

EMS Services

Resources Service Container Persistent State Handle Service (PSHS)

Job management and monitoring services Job Job Manager

Resource selection services Execution Planning Services (EPS) Candidate Set Generator (CSG) Reservation services

EMS Services

Interactions with the rest of OGSA Deployment & Configuration Service Naming Information Service Monitoring Fault-Detection and Recovery Services Auditing, billing and logging services Accounting

IssuesIssues

Find Describe

Access

DataDataDataData

Formats

ProtocolsProtocols

Use casesUse cases

DataDataDataData

DataData

DataDataMove/Copy/ReplicateMove/Copy/Replicate

Metadata Metadata

DataData

ManageManage

Common

accessCommon

access

Data Services

The basic problem Manage, transfer and access distributed data services and resources

The basic problem Manage, transfer and access distributed data services and resources

Derived dataCatalog

Sensor Data stream

Text file

Relational database

Source: OGSA keynotes 20060510 H. Kishimoto

Data Services

Data Service n

Data Service n

Data Service 1

Data Service 1

Data Service 2

Data Service 2

Composite Data Services

ReplicationReplication

CacheCache

FederationFederation

Source: OGSA keynotes 20060510 H. Kishimoto

Data Services

Functional Capabilities Transparency and

Virtualization Client APIs Extensible data type support

and operation Data Location Management Simple Access Queries (Structured Access) Transformation Data Update Security Mapping Extensions Data Resource Configuration Metadata Provenance

Properties Scalability Quality of Service Coherency Performance Availability Legal and Ethical Restrictions

Data Services

Interactions with the rest of OGSA Transactions Logging Execution Management Services Workflow Provisioning Resource Reservation Discovery Security Network management Naming Notification

Resource Management Services

Management of the resources themselves Management of the resources on Grid Management of the OGSA infrastructure

Model

Resource Management Services

Interactions with the rest of OGSA Information Services Execution Management Services Data Services Self-Management Services Infrastructure Security

Properties Scalability Interoperability Security Reliability

Security ServicesObjectives

Enforcement of security-related policy in a VO Span multiple administrative domains Integrate and make interoperable unify popular security models,

mechanisms, protocols, platforms, and technologies Implementation-agnostic Extensible and Integratable

Security Services

Model Security policies are statements about entities, interaction mechanisms and

contexts The model defines the security services as entities with interaction patterns

that facilitate the administration, expression, publishing, discovery, communication, verification, enforcement and reconciliation of the security policy

Security Services

Functional Requirements Authentication Identity mapping Authorization Credential conversion Audit and secure logging Privacy

Interactions with the rest of OGSA All of OGSA services Can be a consumer of other services (E.g. Data Services)

Self-Management

Self-

Management

Self-

Management

MonitoringMonitoring

ProjectionProjection

AnalysisAnalysisActionAction

PolicyPolicyPolicyPolicyPolicyPolicy

PolicyPolicyPolicyPolicySLASLA

Self-configuration: Automatically adapt to changes in the environment: e.g. Deploy/undeploy resources as load changes

Self-configuration: Automatically adapt to changes in the environment: e.g. Deploy/undeploy resources as load changes

Self-optimization: Automatically tune system to best meet user or business needs Uses service-level agreements (SLAs)

Self-optimization: Automatically tune system to best meet user or business needs Uses service-level agreements (SLAs)

Self-healing: Automatically detect & correct problems

Component failures Security violations etc.

Self-healing: Automatically detect & correct problems

Component failures Security violations etc.

Source: OGSA keynotes 20060510 H. Kishimoto

Basic Properties Service Level Agreement Policy Service Level Manager Model

Self Management Services

Functional Requirements Service level management

Monitoring Analysis and projection Action

Policy and Model based Management Entitlement Planning Capacity Management Provisioning

Properties Availability Security Performance

Self Management Services

Interactions with the rest of OGSA Discovery Logging and monitoring Resource reservation Workflow Composition Security Resource management

Information Services

Executionmanagement

Resourcereservation

Problemdetermination

Accounting

Applicationmonitoring

Loadbalancing

Servicediscovery

Consumers ConsumersInformationServices

InformationServices

• Reliable

• Secure• Efficient

Provide management and access facilities for information about applications and resources in the grid environment

Provide management and access facilities for information about applications and resources in the grid environment

ProducersProducers

Asynchronous notification

Retrieval

RegistryRegistry

LoggerLogger

Source: OGSA keynotes 20060510 H. Kishimoto

Information Services

Functional Requirements Naming scheme

Human-oriented name Abstract name Address

Discovery Message delivery Logging Monitoring General Information and Monitoring service

Information Services

Interactions with the rest of OGSA Standard event data models Notification mechanisms Security services Replication

Agnostics Questions1. How does OGSI relate to IGSA? http://www-scf.usc.edu/~igsa

2. QoS is stated as a requirement, but isn’t this an implicit requirement for all computer systems. So how is it addressed in OGSA that makes it important to single out?

3. Keeping tabs on resources using XML messages sounds good, but what happens when the resource state information is too complex and makes the messages too big and cumbersome? As you know parsing and working with large SOAP messages is not efficient at all. Specifically, any ideas on how to make this a more efficient process?

4. Service discovery is mentioned as an existing facility, but service discovery is limited to the facilities each organization provides at this moments. Any ideas on how his could be streamlined and standardized?

5. In capabilities (section 3) the specification mentions self-management. Does this imply that OGSA also specifies a standard for all Grid Services to be Autonomic Systems?

6. Why where those specific Use Cases chosen?

7. Is this really an Architecture or more like a grouping of many different services?