cngrid: progress, achievements, and prospects prof. qian depei director, sino-german joint software...

CNGrid: Progress, achievements, and prospects

Prof. Qian DepeiDirector, Sino-German Joint Software Institute (JSI)

Beihang Universitydepeiq@buaa.edu.cn

Beijing, June 21, 2005

Overview of CNGrid A key project supported by the

National High-Tech R&D Program (the 863 program), continued effort to the previous HPCE project

4-year project 100 million fund from the Chinese

government (MOST) Two to three times more associated

funds from local governments, application organizations, and industry

Major tasks of the project Developing a test-bed for the new information in

frastructure—China National Grid (CNGrid) About 8 grid nodes across the country Interconnected by available networks

Developing a 4TFlops high performance computer for the grid

Developing a set of grid software to support grid applications and grid operation and management

Developing several application grids Productive systems Demonstration of the future grid applications

Fund distribution Testbed: 10 million HPC: 40 million Software: 12 million Application: 27 million Strategic study and integration: 5 million Supporting center: 3 million Other: 3 million

Guidelines of the project Application driven

The most important driving force to the research

Technological leaping Emphasizing patents and

standards

Guidelines and strategies of the project (con.) Multilateral cooperation

Funds from 863 program Local government Application organizations Industry

Cooperation of Research teams Application teams Management people

Focus on the grid Complementary research efforts Integration into the grid testbed

The R&D supported 25 projects are supported so far

The strategic and integration technology study

testbed(9 teams) Grid software (1 team) Application grids (11 teams) High performance computers (2 teams) Testing of software and HPC (2 teams)

Task 1: Grid test-bed development Building the China National Grid (CDGrid) Eight nodes across the country

CAS Network Center (Beijing, major node) Shanghai Supercomputer Center (Shanghai, major node Tsinghua University (Beijing) Institute of Applied Physics and computational science (Beiji

ng) University of Science and Technology of China (Hefei, Anhui) Xi’an Jiaotong University (Xi’an, Shaanxi) NUDT (Changsha) Hong Kong University (Hongkong)

CNGrid operation and supporting center (CAS Network Center)

Resource Requirement Major nodes

>1 Tflops Equipped with the HPC developed by t

he project Ordinary nodes

>100GFlops Machines provided by the organization

supporting the node

grid nodes across China

Hefei

Xi’an

Beijing

Changsha

Shanghai

Hong Kong

CNGrid operation & support center Based on the Computer Network & Infor

mation Center of CAS Major tasks

Operational management of CNGrid User management Security management User training Technical support Software maintenance

Task 2: HPC development Two >4TFlops machines have been

developed Dawning 4000

AMD 64-bit Opteron processor 11 Tflops peak performance

Lenovo DeepComp 6800 Intel Itanium 2 processor 5.324 Tflops peak performance

Lenovo DeepComp 6800 Completed in Nov. 2003 2.6TB momory 81TB disks 4.183 Linpack performance (78.5%

efficiency) Ranked at 14 in Top500 ranking Installed at CAS Network Center in J

an. 2004

Lenovo 6800

Dawning 4000A Completed in Apr. 2004 8TFlops Linpack performance (70% effi

ciency) Installed at the Shanghai Supercomput

er Center in Nov. 2004

HPC measurement & evaluation center

Based on Tsinghua University Major tasks

Study of measurement methodology Benchmark development Services for system evaluation China Top 100 ranking jointly with ano

ther organization

Task 3: Grid software development

Grid system software Interfacing to heterogeneous systems Basic resource management &

scheduling Communication

Grid application development environment

Grid user environment Grid browser GSML

Grid software layerComp

Application

System

Resource

Securityservice

Jobservice

Usermanage

Dataservice

Gridmonitor

Information service

Grid resources (host, programs, instruments, file systems, database) wrapped as WS or GS

Data Transact

Grid S/W

Grid manage

UserEnvir.

Other

Grid software deployment Version 1.0 in Feb. 2004 Starting deployment in March

2004 on the test-bed

Deployment of grid software 1.0

Web 浏览器

GT3 core

网格服务器

Web Portal

GT3 core

网格服务器

网格应用

CNGrid GOS APIs

网格应用服务器

GT3

WS GS

GS

OGSA/OGSIWebSphere,

.Net

GS or wrappers

网格结点 A

本地资源驱动

网格结点 B

本地资源驱动

CNGrid 信息管理和资源管理

资源外包

CNGrid 信息管理和资源管理

Internet

CNGrid GOS APIs

基于网格软件的应用

GS orwrappers

其它APIs

注册

注册

网格操作环境

RB1

RB2

RB3

RA1

RA3

RA2

图7 网格软件部署结构

网格管理工具

Major features of the CNGrid software v2

V 1.1 V 2.0

Architecture

Simple virtualization

Resource routing

Advanced virtualization, grid proc, community

Loosely coupled: easy to change

Lower the appl. programming cost

User man

Life cycle man (reg., login, use, maintain, logout) Supporting roles

Security man.

Single login and authentication

Supporting authorization (community-based multi-grain access control)

Sys. Man.

Monitoring, accounting Monitoring, accounting, log file

Resource man.

Virtual & physical resource man., dynamic deploy

Effective, virtual resource management (strategy-based resource selection)

Data man.

Multi-point (not user transparent), physic view

Unified virtual view, location independent, access control

CNGrid GOS Client API

CNGrid GOS Core API

Grid application logicManagement tools for grid software, grid use environment

DynamicDeployment services

Nameservices

Data services

PKI based grid securitymechanism

GridApplication

Gridaddressbuses

Gridresources

Resource management

Application servers (i.e. Tomcat...)

Grid node hosts and storage resources

Resources encapsulation

Strategies and Access control engine

Grid file management Resource schedule strategy

Authorize strategy

Access control at service operational level

Grid process container and grid process

Securitymechanism of grid App

Access mechanism of general services

Securitymechanism of grid nodes

Grid resources driver

Grid router network

User management

Gridcommunity

Gridprocesscontainer

Community Service

Grid community

grid applications grid process container grid community services Forming Virtual organization Self-organizing and managing

Grid process runtime entity for accessing resources

on behalf of users running in the grid process container. Simplifies client programming

interface Hiding resource access details Result buffering Decouple grid applications from

resources

Grid process container provides execution and

management environment for grid process

Dynamically deployable

Grid address bus

Community is built upon grid address bus.

Grid address implements the binding of community resources and grid resources The location change of the grid

resources will not affect the grid community and grid applications

Grid use environment GSML is the application development language

for the end user XML based Easy to use flexible, robust access to grid resources Interactive GSML page composed of information cells

GSML runtime system GSML browser GSML editor Tools

Roadmap of grid software

Compatible to v1.1: sys.interfaces, Hosting Environment

Time table Feb. 2005: internal testing version May 2005: beta version, support demo applic

ations June 2005: v2.0

Task 4: Application grids development

Applications from selected areas Resource and Environment

GSG, DFG, SeisGrid Research

SDG, BioGrid, DDG, ChinaGrid Services

MSG, ITG Manufacturing

AviGrid, SimGrid

Geological Survey Grid Undertaken by China National

Geologic Survey Bureau Nation-wide productive grid Data exchange and sharing Resource evaluation services

Shorten the evaluation time from 3 years to 3 months

Delivering achievements in North China ground water resources

evaluation Mineral resources evaluation

Grid environment and resource deployment

Digital Forestry Grid Undertaken by the Institute of

Forest Resources Information Support turning the farmland

back to forests potential evaluation project planning project evaluation

Potential Analysis (major rivers and lakes ）

Buffer region analysis result(A)

Farmland, unused land, and mountain (B)

Overlap result: A+B ＝ C

Grade >25 （ D ）

Second overlap result ： B ＋ D ＝ E

The third time overlap: (C ＋ E ）

Meteorological Service Grid Undertaken by China National

Meteorological Center Multiple sites to form the research

platform for new weather forecasting model research

Provide new weather forecast services (time and location-specific)

Application development platform

·

CA·Myproxy·MDS GIIS·Globus 2.4

Node A（ Oscar cluster ）

Node B（ Sunway cluster ）

Node C（ Galuxy III ）

Man node

·Globus 2.4·OpenPBS.GRIS

·Globus 2.4·OpenPBS.GRIS

·Globus 2.4·OpenPBS.GRIS

Web server

·

CMAG Portal·Tomcat, Java SOAP·Globus, CoG·Myproxy client

Grid resource

32 dual CPU node 16 dual CPU node

MM5 mode （ 5km ） : simulation of rainfall of 30-36h, simulation result is 12h later than reality.

GRAPES_MESO mode （ 6km ） rain belt, center, and time close to real situation!

Result from GRAPES for forecasting snow (rain) in Dec. 21, 2004 （ grid ： 30km ）

Scientific Data Grid Undertaken by the Network &

Information Center of CAS Integration of more than 200

scientific databases from more than 40 institutes across the country by means of metadata management

Virtual astronomical observatory

Software module structure

Scientific Data Grid Portal

General metadata management tool

MDIS(LDAP)

interim XML

MD schema

Userpage

Process(Java bean)

XML engine

install & configure

universal, extensible

customizable

-metadata is tree-like and more flexible than fix-column tables, difficult to deal with on web UI

-use xml files to store interim results

New drug discovery grid Undertaken by Shanghai Institute of Mat

eria Medica CAS Compound screening for new drug discov

ery Speed up the process by computer simulatio

n Higher accuracy

Using HPC in P2P mode New drug for diabetes is under developm

ent and will enter clinic testing by the end of 2005

New Drug Discovery Grid Platform

神威机群Shanghai SCC

神威PC机群

PCPC

神威PC机群

曙光4000A

Beijing Medical Institute

Shanghai Institute of Materia Medica CAS

主服务器

Three layers of the grid

Internet

Main server

Sub-server

term term term

Internet

CN

CN

CN

CN

CN

CN

CN

CN

CN

Sub-server Sub-server

DDG Portal

Screening task allocation on different clusters

Bio-informatics Grid Undertaken by Genomics &

Bioinformatics Institute, CAS Provide computing, data, and

information grids for bio-information research in the country

Manufacturing Grid Undertaken by AVIC II of China Support distributed design,

simulation, and manufacturing CAD software sharing Data exchange and sharing Computing resource sharing

哈飞

洪都

北京 628

Manufacturing Grid

Aviation grid platform composed of three clusters in different locations

Manufacturing Grid

WindowsWindows

Windows Windows

Linux

Linux

LinuxWindowsAIX

　　 Automatically locating resource when submitting jobs

网格内的所有结点的硬件共享

WAN

MultiClusterMultiCluster

submit

Licence poolLicence pool

Acquire Token

GLBGLBrun

Manufacturing Grid

Software sharing

After optimizationbefore optimization

Added skin Reduced skin

Manufacturing Grid

How to judge the success of Grid?

Application is the most important criteria Improves the quality? Increase the efficiency? Lower the cost? Reduce the complexity?

International Cooperation MoU with UK e-Science Planned connection with EGEE IBM and Platform participate CNGri

d project Exchange research results with Glo

bus team Looking forward to broad internatio

nal cooperation

What is going on? CNGrid may become part of the CNGI effort National Network Research Environment will be built upon C

NGrid The next five-year R&D plan is being designed. It is very possi

ble that Grid will be one of the key issues. Grid applications are emerging. We need successful applicati

ons to show the feasibility of Grid technology New trends in Grid research is emerging

Ubiquitous and pervasive Grid Mobile grid applications New operating systems, programming models, languages, and t

ools for Grid Business models and economical models for Grid services

Thank you!