session: application deployment on grids/escience

36
Session: Application Deployment Session: Application Deployment on Grids/eScience on Grids/eScience The 28 The 28 th th Open Grid Open Grid Forum Forum München, Germany München, Germany

Upload: damon

Post on 19-Mar-2016

51 views

Category:

Documents


1 download

DESCRIPTION

The 28 th Open Grid Forum München, Germany. Session: Application Deployment on Grids/eScience. Agenda. The presentation of “Application Deployment on Grids/eScience” from Lanzhou University. (25min.) - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Session: Application Deployment on Grids/eScience

Session: Application Deployment on Session: Application Deployment on

Grids/eScienceGrids/eScience

The 28The 28thth Open Grid Forum Open Grid Forum

München, GermanyMünchen, Germany

Page 2: Session: Application Deployment on Grids/eScience

| 2

Agenda

The presentation of “Application Deployment on Grids/eScience” from Lanzhou University. (25min.)

Question and Answer. (10min.)

Discussing with the topic of “How to make the developments and the Operations on Grids/eScience easier?” (20min.)

Page 3: Session: Application Deployment on Grids/eScience

Dr. YI YANGDr. YI YANG

Lanzhou University, ChinaLanzhou University, China

[email protected]@lzu.edu.cn

Application Deployment on Application Deployment on

Grids/eScienceGrids/eScience

Page 4: Session: Application Deployment on Grids/eScience

| 4

Outlines

Background

Our team and Project

Application Deployment Interface Framework (ADIF)

Current work

Further work

Publications

Page 5: Session: Application Deployment on Grids/eScience

| 5

Background

Grids technologies are developed sophisticatedTo be the infrastructure of eScience Too many Grids Different countries

Globus, U.S.AEGEE, EUCNGrid, ChinaCGSP, China……

Does this met the objectives of the initial Grid?

Different purposes

Bioinformatic

Computer Chemistry

Environment

Physics

……

Different specifications

OGSI

Webservices

WSRF

Different imple-mentation of WSRF

……

Page 6: Session: Application Deployment on Grids/eScience

| 6

Background

Objectives of Grid The usage is as convenient as the electricity to a

general customer. Standard, open, general-purpose protocols and

interfaces. A hardware and software integration. Sharing the resources on the internet. Co-operations among distributed areas and

organizations.

?

?

√√ !

?

Page 7: Session: Application Deployment on Grids/eScience

| 7

Background

Current Grid Integration the software and hardware. Sharing the resources as much as more. To be as the infrastructures of many applications. BUT

The sharing resources is flat organized.

The usage of applications is always for special purposes.

The usage of applications is difficult for a general customer.

Why?

Page 8: Session: Application Deployment on Grids/eScience

| 8

Background

Current development mode of Grid Applications Developed by the developers of Grid. Developers must familiar with both the

specifications of Grid and the domainknowledge of special purposes.

The development and maintenance is hard.

Applications tightly coupled with itsrelated Grid.

The leverage between applications andGrid is not clear.

The migration of applications to other gridsseems unable.

Applications

Gird

Page 9: Session: Application Deployment on Grids/eScience

| 9

background

Idea The “abstraction” is always an important methodology to improve

computer science.

Why we don’t abstract the grid application develop interface (ADVI) to higher levels so that it can be used to develop applications by a domain user or specialist.

COMPUTERabstract Assembly

Language

abstractC/C++/Java

GridArchitecture

abstractADVI

abstract??

Page 10: Session: Application Deployment on Grids/eScience

| 10

Background

GridApplications

Grid platform

DeployMiddleware

DeployMiddleware

DeployMiddleware

The “deployment” of grid applications on Grid Abstract the ADVI Separate the development of

Grid and Grid applications Make them loosely coupled Application Deployment Interface

Framework (ADIF)

Page 11: Session: Application Deployment on Grids/eScience

| 11

Our Team

A single research group, which are composed of three departments of two Universities. ICCM and SC Lab in Lanzhou University GCD in Tsinghua University

Group Leader : Prof. Li Lian, [email protected]

Team Member : Prof. Ruishen Zhang, 2 lectors, 3 Ph.Ds, 8 Masters.

Page 12: Session: Application Deployment on Grids/eScience

| 12

Project

Basic Research on Grid Specifications Supported by National Natural Science Foundation of China. 12 teams in total who are in charge of different parts of the whole

specifications respectively It is hoped to become the National Grid Specifications in

China after finished Started from Jan. 2008 and will end in Jun. 2010. Funding is 3 million chinese yuan

Page 13: Session: Application Deployment on Grids/eScience

| 13

Page 14: Session: Application Deployment on Grids/eScience

| 14

Application Deployment Interface Framework (ADIF)

Abstract the Application Deployment interface from Application Development interface

It is XML documents, and according to BNF.

Developers just deed to generate these documents by web wizards.

Reference from WSRF 、 WSDL 、 JSDL etc.

Page 15: Session: Application Deployment on Grids/eScience

| 15

The Conception of Application Deploy

Grid platform and grid application can be developed independently.

Here, the basic architecture is the grid platform, and applications in different areas will be built on this basic architecture through the standard, uniform interfaces, which make their surface minimal, seeming like a ‘sandglass’.

Page 16: Session: Application Deployment on Grids/eScience

| 16

The Advantages of Application Deploy

The grid middleware will be the core of gird applications and the shell of the grid middleware

The grid application will provide an easier and more custom user experiences

The standardizing of grid platform and grid application will be promoted rapidly.

It enhances the scalability and extensibility of grid applications

It also provides the possibility of deploying several applications on alternative grid platforms, especially the heterogeneous grid platforms.

Page 17: Session: Application Deployment on Grids/eScience

| 17

Current Work

ADIF v1.0 A general specification Four sub specifications

Three levels Grid application level Grid application

deployment level Grid architecture level

Four interfaces Resource management Workflow management Logging management Notification managements

Page 18: Session: Application Deployment on Grids/eScience

| 18

The general specification V1.0

A general introduction to ADIF

Terminologies and Definitions

Introduction about four sub specifications and their relationship.

Page 19: Session: Application Deployment on Grids/eScience

| 19

Application Deployment Resource Management V1.0

The grid is to integrate the Internet into a giant super-computer to achieve the comprehensive sharing.

Unfortunately, collections of heterogeneously and geographically distributed resources in grid system are assembled loosely and disorderly.

Only limited resources are useful for individual users or communities.

It is intractable to hunt for the proper resources from such huge system according to user’s requirements.

The specification for discovering intended resources, from a common pool collecting all kinds of resources, becomes more crucial for many users.

Page 20: Session: Application Deployment on Grids/eScience

| 20

Application Deployment Resource Management V1.0

At present, the Resource Management includes: Resource Deploy Here, resources deploying is allowed, which makes

individuation requirements customizing possible. Actually, only a document, called deployList, in XML format, is necessary.

Resource Undeploy Under specified circumstances, some resources in User Resource List MAY become obsolete. In order to make sure the practicability of the User Resource List, users are authorized to give these resources away absolutely according to their wishes.

Resource Destroy Usually, some resources should be discarded from the User Resource List when they have become unavailable. For security, under this operation, the unavailable resources are merely deleted from user logic view, and never destroyed in the grid system.

Page 21: Session: Application Deployment on Grids/eScience

| 21

Application Deployment Resource Management V1.0

Grid Resources RDD User Resourcest

Example of Resource Deploy

Page 22: Session: Application Deployment on Grids/eScience

| 22

Application Deployment Workflow Management V1.0

In order to support complex research experiments, distributed resources need to be orchestrated while the application workflow operations being managed under Grid environments.

The grid workflow management system thus should provide sets of tools to facilitate building grid application services by orchestrating web services.

From grid perspective, workflow is a coarsely granular unit which is supervised and controlled by the job management system of grid. The focus of workflow is mainly what to do and how to do.

At present, the user not only regards the logic of the whole processes, but also involves different requirements towards heterogeneous environments to submit their computational jobs.

Page 23: Session: Application Deployment on Grids/eScience

| 23

Application Deployment Workflow Management V1.0

Therefore, a set of standard interfaces is proposed as following: Definition for the process control structure; Definition for the workflow designer, which addresses some conventions

on the structure of the designer; The Requirement Document Definition, which provides an XML

grammar document to implement the users’ requirements about computational jobs, in order to identify an environment to support reliable executions towards workflow.

Page 24: Session: Application Deployment on Grids/eScience

| 24

Application Deployment Workflow Management V1.0

Page 25: Session: Application Deployment on Grids/eScience

| 25

Application Deployment Logging Management V1.0

Logging management plays an important part in the grid platform. The logging can absolutely record all the information of each operation in the grid platform, which can provide all the necessary information for any possible analyzing of the grid administrator.From the perspective of users, the information, which should be acquainted, is not only the state information of grid platform, but also the record of the information of personal behaviors in the grid platform. So it is important to manage and maintain the records of the user’s behaviors in the grid system. However, for each user, what they concern about just is their own relevant information among the grid platform. So the interfaces to extend and shield the logging of grid SHOULD be provided.

Page 26: Session: Application Deployment on Grids/eScience

| 26

Application Deployment Logging Management V1.0

Therefore, in this document the map relationship of logging, between the application system and the grid, is defined. What’s more, the flexible logging definition is provided, and the personnel operation for the logging model is supported. the logging for Job Result, the logging for submitting the Atomic Job, the logging for submitting the Workflow Job, the logging for packing the Application for gird platform, the logging for Destroying Resource in User Resource List, the logging for user Login in platform the logging for user Logout from grid platform

Page 27: Session: Application Deployment on Grids/eScience

| 27

Application Deployment Logging Management V1.0

Example of logging deployment

User logging Mng.RDD

User logging

Page 28: Session: Application Deployment on Grids/eScience

| 28

Application Deployment Notification Mechanism V1.0

There are multitudinous resources in Grid System, but the resources, meeting users’ requirements, MAY be only a small part.

So it turns necessary that the resources COULD be subscribed according to users’ requirements.

However, the resources among the Grid System are changing frequently and dynamically

It turns more urgent to provide an interaction pattern between the user and the grid, which is used to inform the users the current statuses of the grid resource.

Page 29: Session: Application Deployment on Grids/eScience

| 29

Application Deployment Notification Mechanism V1.0

Page 30: Session: Application Deployment on Grids/eScience

| 30

Current Results

ADIF documents V1.0 ADIF v1.0 ADIF-WorkflowManagement v1.0 ADIF-ResourceManagement v1.0 ADIF-LoggingManagement v1.0 ADIF-NotificationMechanism v1.0

A reference implementation on CGSP ( China Grid Support Platform)

Page 31: Session: Application Deployment on Grids/eScience

| 31

Future Works

Improve the ADIF

Make the ADIF more implacable

Deploy eSoc on CGSP using ADIF

Interoperation between CGSP and CNGrid using ADIF

Page 32: Session: Application Deployment on Grids/eScience

| 32

Publications

Jiazao Lin, Wanbing Shi, Yi Yang , Lian Li , A novel model for grid resources management based on user customization, ChinaGrid Annual Conference, 2008-9-12 [EI检索 ].

Wanbing Shi, Yi Yang, Chunyan Zhang, Lian Li , A Study on ADIF Based on e-Science Deployment Architecture, GCC2007 IEEE Computer Society,2007.8 [EI检索 ].

Lian Li, Ruishen Zhang, Yongwei Wu, Yi Yang, deploy eSOC on CGSP, Tech. Report. Lanzhou University.

Jiazao Lin, Zhili Zhao, Shoubo Li, Huajian Zhang, Lifen Li, Lian Li, Caihong Li , Application-objected Workflow Management System based on Abstract Service , The 8th international conference on grid and cooperative computing (GCC2009) , 2009 , 8(已发表)Jiazao Lin, Zhili Zhao, Lei Liu, Huarong Sun, Shoubo Li, Caihong Li, Li Liu, Lian Li, MDMF: A Comprehensive Framework for Managing Large-scale Heterogeneous Data in eSoC Collaborative Environment , SWS2009, 2009,pp:138-143

Chenyang Zhao, Shoubo Li, Zhiwei Liu, Li Liu, Lian Li. A Publish/Subscribe System based on Distributed Broker Management in e-Science environment. SWS2009, 2009, pp:20-25

Shoubo Li, Jiazao Lin, Chenyang Zhao, Zhili Zhao, Li Liu, Lian Li. GCP:A Load-balancing Tool in e-Science Environment. SWS2009, 2009. pp:90-95.

Page 33: Session: Application Deployment on Grids/eScience

| 33

“Although we can see the limit steps we have made, but we still have a lot of works to do.”

Page 34: Session: Application Deployment on Grids/eScience

Thank YouQuestions & Answers

Page 35: Session: Application Deployment on Grids/eScience

| 35

How to make the developments and the Operations on Grids/eScience easier?

Is the idea of ADIF correct?

Does the ADIF useful?

IF it does, How to improve? Suggestions and Advices are welcome

Any Other sub specification?

……

Page 36: Session: Application Deployment on Grids/eScience

Thank You