1

Grid Computing Middleware

Presenter:Xon Xay

Nguyen Tuan VietTram The Phien

2

Outline

Introduction Definition of Grid Middleware Purpose of Middleware Grid layered architecture Technology

Globus GLite

3

Outline

Introduction Definition of Grid Middleware Purpose of Middleware Grid layered architecture Technology

Globus GLite

4

Grid environment

5

Introduction

Three main issues to confont in Grid environment: Heterogeneity – multiplicity of

resources Scalability Adaptability - Resource managers or

applications dynamic to extract the maximum performance from the available resources and services.

6

Outline

Introduction Definition of Grid Middleware Purpose of Middleware Grid layered architecture Technology

Globus GLite

7

Grid Middleware System software between

applications and operating system Provide services to application

Discovery, storage, execution, information, service integration, resource monitoring, failure detection and recovery,…

Hide heterogeneous of the Grid environment

Provide standardised interfaces to services.

8

Outline

Introduction Definition of Grid Middleware Purpose of Middleware Grid layered architecture Technology

Globus GLite

9

Purposes of Middleware Open, general-purpose and standard Standard protocols

Defines the contents and sequence of message exchanges used to request remote operation

Important and essential to achieve the interoperability that Grid depends on

Standard APIs Interfaces to code libraries Facilitate construction of Grid components by

allowing code components to be reused

10

Outline

Introduction Definition of Grid Middleware Purpose of Middleware Grid layered architecture Technology

Globus GLite

11

Grid layered architecture

- The neck: a small set of core abstractions and protocols - Top of hourglass: many different high-level behaviors can be mapped - Base of hourglass: can be mapped onto many different underlying technologies

12

Component of Middleware

Connectivity layer Resource layer Collective layer

13

Connectivity layer Communicating easily and securely Communication protocol: exchange data Authentication protocol: cryptographically

secure mechanism for verifying the identity of users and resources. Single sign-on Delegation Integration with various local security solutions User-based trust relationships

14

Resource layer Sharing single resources Define protocols (and APIs and SDKs) for

the secure negotiation, initiation, monitoring, control, accounting, and payment of sharing operations on individual resources

Information protocols - obtain information about the structure and state of a resource

Management protocols - negotiate access to a shared resource

15

Collective layer Coordinating multiple resources Implement a wide variety of sharing

behaviors without placing new requirements on the resources being shared

Example – Directory services, co-allocation, scheduling, brokering services, Monitoring and diagnostics services, Data replication services, Grid-enabled programming systems,…

16

Collective layer- Example

Application use co-reservation service protocol to request end-to-end network reservation

We define a co-reservation service protocol

Implement co-reservation service

Call co-allocation API to implement co-allocation operation

17

Outline

Introduction Definition of Grid Middleware Purpose of Middleware Grid layered architecture Technology

Globus GLite

18

Globus Toolkit

Evolution Globus Toolkit version 2 Globus Toolkit version 3 Globus Toolkit version 4

19

Globus evolution

1997

GT2.0

2003

GT3.0OGSA

2005

GT4.xWSRF

Proof of concept Standardization

20

Globus Toolkit version 2 Community-based, open-architecture,

open source set of services and software libraries that support Grids and Grid applications.

De facto standard for Grid computing Focus on usability and interoperability Middleware

GT2 – Connectivity layer GT2 - Resource layer GT2 - Collective layer

21

GT2 – ConnectivityAuthentication protocol Public-key based GSI protocols

GSI - Grid Security Infrastructure authentication, communication protection,

authorization GSI builds on and extends the TLS

protocols TLS - Transport Layer Security TLS - defines a protocol to provide privacy

and data integrity between two communicating applications.

22

GT2 – Connectivity layerCommunication protocol

Internet layered protocol architecture Internet layer - IP and ICMP Transport - TCP, UDP Application layer - DNS, OSPF, RSVP,

…

23

GT2 - Resource layer A Grid Resource Information Protocol –

base on Lightweight Directory Access Protocol (LDAP)

HTTP-based GRAM protocol GRAM - Grid Resource Access and Management allocation of computational resources monitoring and control of computation on

resources GridFTP - a management protocol for data

access An extended version of the File Transfer

Protocol

24

GT2 - Collective layer build on Globus Connectivity and Resource

protocols Meta Directory Service

introduces Grid Information Index Servers (GIISs) LDAP information protocol

Access resource-specific GRISs to obtain resource state GRRP used for resource registration.

DUROC resource co-allocation library provides an SDK and API for resource co-allocation

25

Globus Toolkit version 3 Build on and significantly extend GT2

concepts and technologies Base on Open Grid Services Architecture

Factoring of component behaviors – identify essential Grid functions and express

Service orientation – enable a uniform treatment of all network entities

Align with Web services – WSDL (Web Services Definition Language)

26

Globus Toolkit version 4

developed by The Globus Alliance OGSA requires “stateful services” WSRF – Web Service Resources

Framework developed by OASIS (http://www.oasis-

open.org)

WSRF provides the stateful services that OGSA needs

27

Relationship between OGSA, GT4, WSRF, and Web Services

28

gLite Toolkit The Enabling Grids for E-sciencE

project (EGEE) Middleware stack that combines

components developed in various related projects.

29

gLite middleware A layer between

services and resources

gLite follows a Service Oriented Architecture

30

gLite - guiding principles Service oriented approach

Allow for multiple interoperable implementations

Lightweight (existing) services Easily and quickly deployable Use existing services where possible

Condor, EDG, Globus, LCG, … Portable

Being built on Scientific Linux and Windows Security

Sites and Applications

31

gLite - guiding principles Performance/Scalability &

Resilience/Fault Tolerance Comparable to deployed infrastructure

Co-existence with deployed infrastructure Co-existence with LCG-2 and OSG (US) are

essential for the EGEE Grid services Site autonomy

Reduce dependence on ‘global, central’ services

Open source license

32

gLite Architecture

33

gLite - Access

All of the gLite services are accessible via APIs and CLIs.

API - Application Programming Interface

CLI - Command Line Interface

34

gLite - Security Services

35

gLite - Security Services Authentication

identify entities PKI (X.509) infrastructure with CA third party for

signature Authorization

allows or denies access to specific services three basic authorization models, classified as agent,

push and pull. Dynamic Connectivity Service

tackle the problems that arise when connectivity is restricted by the resource owner.

Auditing monitoring and providing for post-mortem analysis of

security related events.

36

gLite - Infomation and Monitoring Services

37

gLite - Infomation and Monitoring Services

Infomation and Monitoring Services a vital low-level component of any grid

38

gLite - Infomation and Monitoring Services

Job Monitoring use the APIs offered by the apache logging

services (log4j) Service Discovery

a facility offered to both end users and to other services to locate suitable services.

Network Performance Monitoring define a standard interface to monitoring

frameworks. NPM Publisher for use by gLite components.

39

gLite - Helper Services

40

gLite - Helper Services Configuration and Instrumentation Service

common, standard-based configuration and instrumentation functionality to the gLite services.

Bandwidth Allocation and Reservation Service control and balance the usage of the network and

categorise and prioritise traffic flows. Agreement Service

implements the communication protocol used to exchange information about Service Level Agreements (SLAs) and defines the SLA structure.

41

gLite - Data Services

42

gLite - Data Services Catalog

store information about the data and metadata Storage Element

is the Grid service responsible for saving/retrieving files to/from some data store which can provide a wide range of quality of service to the user.

Data movement provide scalable and robust managed data

transfer between Grid sites

43

gLite - Job Management Services

44

gLite - Job Management Services Accounting

accumulates information about the usage of Grid resources

Computing element representing a computing resource job management and other capabilities

Workload Management comprises a set of Grid middleware

components responsible for the distribution and management of tasks across Grid resources

45

gLite - Job Management Services

Job Provenance keep track of the definition of submitted

jobs, execution conditions and environment. Package Manager

is a helper service that automates the process of installing, upgrading configuring, and removing software packages from a shared area

46

Comparison

Globus Toolkit - faster and more efficient

gLite - sophisticated and developer friendly

Document Globus documents - sketchy

and missing gLite documents - sufficient,

available

47

Summary

Definition of Grid Middleware Purpose of Middleware Grid layered architecture Technology

Globus GLite

48

Reference I. Foster and C. Kesselman, The Grid: Blueprint for a

New Computing Infrastructure. Morgab Kaufmann Publishers, 1999.

Fran Berman, Anthony J. G. Hey and Geoffrey C. Fox, Grid computing: Making the Global Infrastructure a Reality. John Wiley & Sons Ltd, 2003.

Globus project: http://www.globus.org/alliance/ Grid Computing Slides – Nguyen Tuan Anh Grid Computing Slides – Pham Tran Vu vonLaszewski--grid-middleware 03-grid-concepts-architecture-middleware-adler-vortrag Websites of gLite (http://www.eu-egee.org)

49

Q&A