adopting and setting standards the naregi strategy satoshi matsuoka professor, global scientific...

Adopting and Setting Standards The NAREGI

Strategy

Satoshi MatsuokaProfessor, Global Scientific Information and Comp

uting Center, Deputy Director, NAREGI Project

Tokyo Institute of Technology / NII

GGF16 Thursday Keynote, Athenswith Dave Snelling

National Research Grid Infrastructure (NAREGI) 2003-2007

• Petascale Grid Infrastructure R&D for Future Deployment– $45 mil (US) + $16 mil x 5 (2003-2007) = $125 mil total– PL: Ken Miura (FujitsuNII)

• Sekiguchi(AIST), Matsuoka(Titech), Shimojo(Osaka-U), Aoyagi (Kyushu-U)…– Participation by multiple (>= 3) vendors, Fujitsu, NEC, Hitachi, NTT, etc.– NOT AN ACADEMIC PROJECT, ~100FTEs– Follow and contribute to GGF Standardization, esp. OGSA

AIST

Grid Middleware Grid Middleware R&DR&D

SuperSINETSuperSINET

Grid R&D Infrastr.Grid R&D Infrastr.15 TF => 100TF15 TF => 100TF

National AAA Infr.National AAA Infr.

““NanoGrid”NanoGrid”IMS ~10TFIMS ~10TF

(BioGrid(BioGridRIKEN)RIKEN)

OtherOtherInst.Inst.

National ResearchNational ResearchGrid Middleware R&DGrid Middleware R&D

NanotechNanotechGrid AppsGrid Apps

(Biotech(BiotechGrid Apps)Grid Apps)

(Other(OtherApps)Apps)

Titech

Fujitsu

NECOsaka-U

U-Kyushu Hitachi

Focused “Grand Challenge” Grid Apps Areas

IMS

Ind

us

try

/So

cie

tal F

ee

db

ac

k

Inte

rna

tio

na

l In

fra

str

uct

ura

l Co

llab

ora

tio

n

●

Restructuring Univ. IT Research ResourcesExtensive On-Line Publications of Results

Management Body / Education & Training

Deployment of NAREGI Middleware

Virtual LabsLive Collaborations

Towards a Cyber-Science Infrastructure for R & D

UPKI: National Research PKI Infrastructure

★

★

★★★

★★

☆

SuperSINET and Beyond: Lambda-based Academic Networking Backbone

Cyber-Science Infrastructure （ CSI)

Hokkaido-U

Tohoku-U

Tokyo-UＮＩＩ

Nagoya-U

Kyoto-U

Osaka-U

Kyushu-U

（ Titech, Waseda-U, KEK, etc.)

NAREGIOutput

GeNii (Global Environment forNetworked Intellectual Information)

NII-REO (Repository of ElectronicJournals and Online Publications

Standards: Setting & Adoption Critical to Grid Adoption &

Longevity• Project lifetimes limited• Most production centers, commercial entities, and users wo

n’t commit their time & money unless longevity guaranteed– A few brave ones… (high-risk, high-return)– Success legacies overtold

• Responsible Action for major research grid infrastructural projects (e.g. NAREGI) to guarantee longevity beyond project lifetime

• Commercial Adoption by IT vendors: ISPs, ISVs, ASPs, etc. key for such longevity– But how do we achieve that?

Approach: “Bait and Catch”--- Typical Japanese National Project Strategy ---

• IT vendors involved (seriously) from Day 1• Public entity project benefactors should not solely expend its grants to sus

tain itself– Most money going to IT vendors to have them commit

• OTOH, vendors are later pressured by the government not to make direct profit

• (Exclusive IPR acquisition or) standardization used as bait for commercial adoption– Thus, even a research grid infrastructural project MUST FOCUS on standards, above al

l other priorities

• Then, standards & supporting technologies are DIRECTLY transferred to IT vendors– Productization and support => longevity => adoption !

A Picture is Worth a 1000 Words…

• Picture deleted due to copyright reasons.• It had a snapshot of an European Toyota car T

V commercial where a prop car was being used to bait a young guy to approach a lake where a gigantic octopus would suddenly appear and instantly whisk him away with his huge tentacles.

• Thus the joke “catch-and-bait”, and if you are reading this slide you have to roll over laughing here :-)

• Work Package Structure (~100 FTEs): – Universities and National Labs: technology leadership– Vendors (Fujitsu, NEC, Hitachi, etc.): professional develop

ment, commercial support

• WP-1: Resource Management:– Matsuoka(Titech), Nakada(AIST/Titech)

• WP-2: Programming Middleware:– Sekiguchi(AIST), Ishikawa(U-Tokyo), Tanaka(AIST)

• WP-3: Application Grid Tools:– Usami (new FY2005, NII), Kawata(Utsunomiya-u)

• WP-4: Data Management (new FY 2005, Beta):– Matsuda (Osaka-U)

• WP-5: Networking & Security– Shimojo(Osaka-u), Oie( Kyushu Tech.)

• WP-6: Grid-enabling Nanoscience Appls– Aoyagi(Kyushu-u)

R&D in Grid Software and Networking Area (Work Packages)

FY2003 FY2004 FY2005 FY2006 FY2007

UNICORE/Globus/Condor-based R&D Framework

Globus4/OGSA/WSRF-based R&D Framework

Utilization of NAREGI NII-IMS Testbed Utilization of NAREGI-Wide Area Testbed

PrototypingNAREGI Middleware

Components

Development andIntegration of

αrelease

Development and Integration of

βrelease

Evaluation on NAREGI Wide-area

Testbed Development ofOGSA-based MW

beta 1, 2,

Verification& Evaluation

Of Ver. 1

Apply ComponentTechnologies toNano Apps and

Evaluation

Evaluation of αrelease in NII-IMS

Testbed

Evaluation of βReleaseBy IMS and other

Collaborating Institutes

Development and deployment of β1, 2, 3,…

αRelease(internal)

OGSA(subset)/Unicore+GT2/3/3.9

β1 Release(public)

OGSA/WSRF/GT4GGF17

Version1.0

Release

Mid

po

int

Eva

lua

tion

NAREGI Middleware Roadmap

List of NAREGI “Standards”(beta 1 and beyond)

• GGF Standards and Pseudo-standard Activities set/employed by NAREGI

GGF “OGSA CIM profile” GGF AuthZ GGF DAIS GGF GFS (Grid Filesystems) GGF Grid CP (GGF CAOPs) GGF GridFTP GGF GridRPC API (as Ninf-G2/G4)GGF JSDL GGF OGSA-BES GGF OGSA-Byte-IO GGF OGSA-DAI GGF OGSA-EMS GGF OGSA-RSS GGF RUS GGF SRM (planned for beta 2) GGF UR GGF WS-I RUS GGF ACS GGF CDDLM

• Other Industry Standards Employed by NAREGI

ANSI/ISO SQL DMTF CIM IETF OCSP/XKMS MPI 2.0 OASIS SAML2.0 OASIS WS-Agreement OASIS WS-BPEL OASIS WSRF2.0 OASIS XACML

• De Facto Standards / Commonly Used Software Platforms Employed by NAREGI

GangliaGFarm 1.1Globus 4 GRAMGlobus 4 GSI Globus 4 WSRF (Also Fujitsu WSRF for C binding)IMPI (as GridMPI)Linux (RH8/9 etc.), Solaris (8/9/10), AIX, …MyProxy OpenMPI Tomcat (and associated WS/XML standards) Unicore WF (as NAREGI WFML)VOMS Necessary for Longevity

and Vendor Buy-InMetric of WP Evaluation

Implement “Specs” early even if nascent if seemingly viable

NAREGI Software Stack (Beta Ver. 2006)

Computing Resources and Virtual Organizations

NII IMS Research Organizations

Major University Computing Centers

（（WSRF (GT4+Fujitsu WP1) + GT4 and other services)WSRF (GT4+Fujitsu WP1) + GT4 and other services)

SuperSINET

Grid-Enabled Nano-Applications (WP6)

Grid PSEGrid 　Programming (WP2)

-Grid RPC -Grid MPI

Grid Visualization

Grid VM (WP1)

Packag

ing

DistributedInformation Service

(CIM)

Grid Workflow (WFML (Unicore+ WF))

Super Scheduler

Grid Security and High-Performance Grid Networking (WP5)

Data (W

P4)

WP3

WP1WP1

The NAREGI SSS Architecture (2007/3)

Grid Resource

Grid-Middleware

PETABUS (Peta Application services Bus)

NAREGI-SSS

WESBUS (Workflow Execution Services Bus)

CESBUS (Coallocation Execution Services Bus; a.k.a. BES+ Bus)

BESBUS (Basic Execution Services Bus)

GlobusWS-GRAM I/F

(with reservation)

GridVM

UniGridSAtomic Services(with reservation)

UniGridS-SCGRAM-SC AGG-SC with RS(Aggregate SCs)

CSGEPS

FTS-SC

JMBPEL Interpreter

Service

ApplicationSpecific Service



NAREGI SSS Structure (2006/06)

WFML2BPEL

SS

NAREGI JM(SS) 　 Java I/F module

NAREGI-WP3 WorkFlowTool, PSE, GVS

JM-Client

SubmitStatusDeleteCancel

EPS

CSG

IS

Cancel StatusSubmit

BPEL2WFST

CreateActivity(FromBPEL)GetActivityStatusRequestActivityStateChanges

CES

OGSA-DAI

CIM

DB PostgreSQL

JSDLJSDL

JSDLJSDL

MakeReservationCancelReservation

globus-url-copy

uber-ftp

CES

CES

S

SR

R

BPEL (include JSDL)BPEL (include JSDL)

Invoke EPS

Invoke SC

JSDLJSDLJSDL

JSDLJSDLJSDL

NAREGI-WFML

NAREGI-WFML

JSDL

Co-allocation FileTransfer

CreateActivity(FromJSDL)GetActivityStatusRequestActivityStateChanges

MakeReservationCancelReservation

JSDLJSDLJSDL

JSDLJSDLJSDL

JSDLJSDLJSDL

JSDLJSDLJSDL

JSDLJSDL

JSDLJSDL

JSDLJSDL

Delete

JSDLJSDL

NAREGI JM (BPEL Engine)

S

JSDLJSDL

JSDLJSDLJSDL

JSDLJSDLJSDL

JSDLJSDL

AbbreviationSS: Super SchedulerJSDL: Job Submission Description DocumentJM: Job ManagerEPS: Execution Planning ServiceCSG: Candidate Set GeneratorRS: Reservation ServiceIS: Information ServiceSC: Service ContainerAGG-SC: Aggregate SCGVM-SC: GridVM SCFTS-SC: File Transfer Service SCBES: Basic Execution Service I/FCES: Co-allocation Execution Service I/F (BES+)CIM: Common Information ModelGNIS: Grid Network Information Service

SelectResourceFromJSDL

GenerateCandidateSet

GenerateSQL QueryFrom JSDL

AGG-SC/RS

SC

CES

PBS, LoadLeveler

S

GRAM4 specific

SC(GVM-SC)

SC

CES

PBS, LoadLeveler

S

GRAM4 specific

SC(GVM-SC)

R R

WS-GRAM

GridVMWS-GRAM

GridVM

FTS-SC

Fork/Exec

is-query

Fork/Exec

globusrun-wsglobusrun-ws

Fork/ExecFork/Exec

GFarmserver

GNIS

GetGroups-OfNodes

NAREGI SSS “Standards” (1/3)Service Name Description Standards

NAREGI-JM •is an OGSA-EMS Job Manager implementation.•accepts NAREGI-WFML (Work-Flow Markup Languange), w

hich is an intermediate workflow language for workflow-to-workflow language conversion

•generates a BPEL document for each NAREGI-WFML (workflow-job) as a OGSA-EMS/RSS service ohrchestration script.

@ OGSA-WG [OGSA-EMS] $ OGSA WFM-RG# OASIS WS-BPEL

NAREGI-BPEL •is an BPEL script execution service implementation.

•accepts a BPEL document.•interpret and execute a BPEL document.

# OASIS WS-BPEL

NAREGI-EPS •is an OGSA-RSS Execution Planning Service implementation.

•accepts a NAREGI Complex Job Instance document or a (simple) Job Instance document, which express a job execution instance for a coupling job. Each Job Instance document contains a JSDL document as a resource requirement description.

•provides a co-allocation-enabled resource selection service as a whole.

•is a aggregation point of the following underlying selection services: single CSG (Candidate-Set Generator) service and multiple "Narrowing down candidates" services such as Grid Storage Service, Network Neighborhood Information Serivce and Reservation Cache Service.

@ OGSA-RSS-WG@ JSDL-WG $ GSM-WG (Grid Storage Management WG)$ GHPN-RG (Grid High Performance Networking RG; network services)$ GSA-RG (Grid Scheduling Architecture RG)$ CDDLM-WG

@: GGF Standard we use, $ GGF Standard we will use, # Industry Standard % De-facto Standard


NAREGI-CSG - is an OGSA-RSS Candidate Set Generator service implementation. - accepts a NAREGI Complex Job Instance document or a (simple) Job Instance document, which express a job execution instance for a coupling job.

- extracts JSDL documents from a Job Instance document, generates resource-database query expressions from these JSDL documents, and reflects query results (resource candidates) into the Job Instance document.

- Virtual Organization aware resource query by VOMS

@ OGSA-RSS-WG@ JSDL-WG@ DAIS-WG@ OGSA-WG [Resource Management] % MyProxy, VOMS # DMTF CIM, ANSI/ISO SQL $ RUS-WG

NAREGI-AGGwith RS

- is a virtual Service Container for a coupling job at coallocation execution services level, and aggregates "real" Service Containers at basic exeuction services level.

- accepts a NAREGI Complex Job Instance document or a (simple) Job Instance document, including final candidates.

- provides logical service operations: MakeReservations() and CommitReservations() to co-allocate and co-schedule a coupling job, and Cancel() and Destroy() to control the coupling job execution.

- contains Reservation (Cache) Service to efficiently select available resources over "time-space" during final candidates. A Reservation Cache Service tracks each resource usage (including reservation) on Aggregation Service Container as possible.

@ OGSA-RSS-WG $ GRAAP-WG [WS-Agreement]$ GSA-RG



NAREGI-FTS - is a file operation service, support basic file operations such as 3rd-party-file-transfer and check-file-existence

- accepts single NAREGI Job Instance document, including DataStaging job without Application in JSDL.

@ GridFTP-WG % GFarm (a Grid File System) $ WS-Naming-WG, $ GFS-WG (Grid File System WG)

NAREGI-SC(GRAM-SC)(UniGridS-SC)

- is a tentative wrapper service to hide grid-middleware-specific service interfaces (in particular for OGSA-BES to become widely used and standardize reservation operation in OGSA-BES).

- accepts single NAREGI Job Instance document.

@ OGSA-BES@ JSDL-WG % Globus [GRAM4]% UniGridS [Atomic Services]% Proxy Certificate, MyProxy $ GRAAP-WG [WS-Agreement]

(infrastructure)

- native https/soap and wsrf processor [WS/RF 1.2 r/rp/rl/bf, WS/N 1.3 bn/topic] in C

- native WSRF-aware WSDL compiler (WSDL to C compiler) - native BPEL interpretor (BPEL4WS) in C

@ OASIS WS/RF@ OASIS WS/N# OASIS WS-BPEL


NAREGI Info Service (beta) Architecture

Client(Resource Broker etc.)

ClientLibrary

Java-API

RDB

Light-weightCIMOMService

AggregatorService OS

Processor

File System

CIM Providers

●　●

ResourceUsageService

RUS::insertURs

ChargeableService

(GridVM etc.)

Job Queue

Cell Domain Information Service

Node B

Node A

Node C

ACL

GridVM


Information Service Node

… Hierarchical filtered aggregationParallel Query …

PerformanceGanglia

DataService

ViewerUserAdmin.

・ CIMOM Service classifies info according to CIM based schema. ・ The info is aggregated and accumulated in RDBs hierarchically.・ Client library utilizes OGSA-DAI client toolkit.・ Accounting info is accessed through RUS.

Client(publisher)

NAREGI IS: Standards Employed in the Architecture

GT4.0.1DistributedInformation Service

Client(OGSA- RSS etc.)

　 Clientlibrary

Java-API

RDB Light-weightCIMOMService

AggregatorService

OS

Processor

File System

CIM Providers

●　●

WS-IRUS

RUS::insertURs

GridVM(Chargeable

Service)

Job Queue


Node B

Node A

Node C

ACL

GridVM


Information Service Node Information Service Node

… Hierarchical filtered aggregation... Distributed Query …

PerformanceGanglia

OGSA-DAI WSRF2.1

ViewerUserAdmin.

OGSA-DAIClient toolkit

CIM Schema2.10

/w extension

GGF/UR

GGF/UR

APP

CIM spec.CIM/XML

Tomcat 5.0.28

Client(OGSA- BES etc.)

APP

NAREGI Dist. Info Standards• GGF Standards we help set within a WG

– “OGSA CIM profile”• GGF and related Standards we employ

– OGSA-DAI– WS-I RUS– GGF UR– OGSA-EMS

• Other industry standards we employ– DMTF-CIM

• Other de-facto “standards” we employ– Globus 4– Tomcat (and associated WS/XML standards)

IS Future Plans• GLUE→CIM translation

• OGSA Information producer / consumer i/f• Access control with XACML policy• Service separability

having a set of sites all communicate some minimal set of data (a simple subset of GLUE schema) getting the information from GLite/R-GMA consumer i/f, Globus/WS-MDS Aggregator translating it to CIM/XML format, archiving it in RDBs and retrieving it through OGSA-DAI Data Service.

NAREGI GridVM (beta) Architecture

Virtual execution environment on each site•Virtualization of heterogeneous resources•Resource and job management services with unified I/F Super

SchedulerSuper

SchedulerInformation

ServiceInformation

Service

AIX/LoadLeveler

GridVM SchedulerGridVM Scheduler

GRAM4GRAM4 WSRF I/FWSRF I/F

Local SchedulerLocal Scheduler

GridVM EngineGridVM Engine

Linux/PBSPro





GridMPI

sitePolicy

Advance reservation, Monitoring, Control

Accounting

ResourceInfo.

SandboxJob Execution site

Policy

NAREGI GridVM: Standards Employed in the ArchitectureSuper

SchedulerSuper

SchedulerInformation

ServiceInformation

Service









GridMPI

sitePolicy

xacml-like access control policy

CIM-based resource info. provider

GT4 GRAM-integration andWSRF-based extension services

UR/RUS-based job accounting

Job submission based on JSDLand NAREGI extensions

sitePolicy

GT4 GRAM-GridVM Integration

GridVMscheduler

RFT File Transfer

Local scheduler

GridVMEngine

SS

globusrun RSL+JSDL’

Delegate

Transferrequest

GRAMservices

DelegationScheduler

Event Generator

GRAMAdapter

GridVMJobFactory

GridVMJob

SUDO

Extension Service

Basic job management＋ Authentication, Authorization

Site

Integrated as an extension module to GT4 GRAM Aim to make the both functionalities available

NAREGI GridVM Standards• GGF Standards we help set within a WG

– OGSA BES (made a presentation at BOF of GGF13) • GGF and related Standards we employ

– JSDL– WS-I RUS– GGF UR– WSRF2.0

• Other industry standards we employ– DMTF-CIM– XACML

• Other de-facto “standards” we employ– Globus Toolkit version 4






SuperSINET



-Grid RPC -Grid MPI

Grid Visualization

Grid VM (WP1)

Packag

ing


(CIM)


Super Scheduler


Data (W

P4)

WP3

WP1WP1

WPWP22

Ninf-G: A Reference Implementation of the GGF GridRPC API

Large scale computing across supercomputers on the Grid

user

① Call remote procedures

② Notify results

Utilization of remotesupercomputers

Call remote libraries

Internet

• What is GridRPC?Programming model using RPCs on a GridProvide easy and simple programming interfaceThe GridRPC API is published as a proposed recommendation (GFD-R.P 52)

• What is Ninf-G?A reference implementation of the standard GridRPC APIBuilt on the Globus ToolkitNow in NMI Release 8 (first non-US software in NMI)

• Easy three steps to make your program Grid aware– Write IDL file that specifies interface of your library– Compile it with an IDL compiler called ng_gen– Modify your client program to use GridRPC API

Architecture of Ninf-G

Client

WS/Pre-WS GRAMNAREGI-SS

Invoke Executable

Connect back

IDL file NumericalLibrary

IDL Compiler

Remote LibraryExecutable

Generate Interface Request

Interface Reply

Server side

Client side

MDS2/4 Interface InformationLDIF Fileretrieve

Globus-IO

GridRPC Standards and contributions to GGF

• GGF Standards we help set within a WG– Grid RPC WG (WG ではなく、スペック自身への ref が必要 )

• GGF and related Standards we employ– GridRPC API– GridFTP– WSRF 2.0– JSDL

• Other de-facto “standards” we employ– Globus Toolkit Versions 4– UNICORE

• Ninf-G is a reference implementation of the GridRPC API• Standardization of the GridRPC API

– GGF GridRPC API WG (spinned off from APM RG)• Co-chair: Hidemoto Nakada (AIST)• Secretary: Yoshio Tanaka (AIST)

– GridRPC API is published as a proposed recommendation (GFD-R.P 52)• A GridRPC Model and API for End-User Applications

– Co-Authors: Hidemoto Nakada (AIST), Satoshi Matsuoka (TITECH)

GridMPI• MPI applications run on the Grid environment• Metropolitan area, high-bandwidth environment: 10 Gpbs, 500 miles

(smaller than 10ms one-way latency)– Parallel Computation

• Larger than metropolitan area– MPI-IO

Wide-areaNetwork

Single (monolithic) MPI applicationover the Grid environment

computing resourcesite A

computing resourcesite A

computing resourcesite B

computing resourcesite B

GridMPI Software Architecture and Standards

• MPI 2.0 (test suite compliant)• IMPI (Interoperable MPI)

– The original IMPI is defined only for the MPI-1.2 feature– Extension for MPI-2

• Porting the extended IMPI protocol to Open MPI• Planning to submit the protocol to NIST

MPI API

TC

P/IP

PM

v2

MX

O2G

Vendor M

PI

P2P Interface

Request LayerRequest Interface

IMP

I

LACT Layer(Collectives)

IMP

I

ssh

rsh

SC

ore

Globus

Vendor M

PI

RPIM Interface






SuperSINET



-Grid RPC -Grid MPI

Grid Visualization

Grid VM (WP1)

Packag

ing


(CIM)


Super Scheduler


Data (W

P4)WP1WP1

WP3

Provisioning• Deployment• Configuration

App. Contents Service (ACS)

Information Services

ServiceContainer

DataContainer

Accounting Services

Execution Planning Services

Candidate Set Generator (Work -Resource mapping)

Job Manager

Reservation

RSS

① JSDL Expansion (WP1)

③NAREGI-PSE(WP3)

② Resource 　Information 　　 Model (WP1)

④Security (WP5)

Architectural Positioning of NAREGI‐PSE （ WP3 ）

NAREGI-PSE Architecture

Application Pool

Deployment Service

NAREGI WFT

Client Register

GetContent

compile deploy un-deploy

Register-UI EPR

Deploy-UI

EPR

GetContent EPR

Deploy

Register deploy info.

Submit

AA File

Contents (WF)

NAREGI-PSE

Register application info.

NAREGI

IS

NAREGI

SS

Request to deploy

EPR

ApplicationInformation

ApplicationRepository

ACS

NAREGI‐PSE （ WP3 ） Standards

• GGF Standards we help set within a WG– ACS （ Application 　 Contents 　 Service) WG

• GGF and related Standards we employ– CDDLM– GridFTP– WSRF 2.0– JSDL– SRM (planned for beta 2)

• Other industry standards we employ– BPEL


34

Web server(apache)

Workflow Servlet

tomcatWokflow Description

By NAREGI-WFML

Server

NAREGISuperscheduler server

NAREGI WP3 Workflow Tool

BPEL<invoke name=EPS-jobA> ↓ 　　 JSDL -A<invoke name=BES-jobA> ↓ 　　 JSDL -A…………………..

NAREGIJM I/F module

BPEL+JSDL

http(s)

Data iconProgram icon

Appli-A

Appli-B

JSDL

JSDL

applet

Data Access

Management (WP4)

Global fileinformation

Application Information

PSE

/gfarm/..

GridFTP(StdoutStderr)

IS

35

• GGF Standards we help set within a WG– Workflow Management Research Group WFM_RG 　（ currently observing ）

• GGF and related Standards we employ– GridFTP– JSDL

• Other industry standards we employ– BPEL+JSDL

• NAREGI WFML has roots in Unicore WF but is heavily extended, and is translated into BPEL + JSDL

• Other de-facto “standards” we employ– Apache+Tomcat– Unicore

NAREGI WFT Standards

GVS Framework

Demo System

CoupledSimulation

CoupledSimulation Computation

ResultsDB

(DataGrid, etc.)

Adaptor

Coupled Sim.Visualization

Service

ParticularVisualization

Service

Coupled Sim.Visualization

Service

Post-proc.Visualization

Service

DBVisualization

Service

MolecularVisualization

Service

CFDVisualization

Service

GridMPI

InformationService

NAREGI Middleware

GVS Client(Reference

Implementation)

Particular ClientConformed to

the Standard I/F

WorkflowTool

AdaptorAdaptorAdaptorAdaptorAdaptorAdaptor

Coupled Sim.Parallel

Visualizer

Coupled Sim.Parallel

Visualizer

Post-proc.Parallel

Visualizer

DBParallel

Visualizer

MolecularVisualization

Software

CFDVisualization

Software

ParticularVisualization

Software

Integrative use of variousvisualization software

through the standard I/F

Integrated WSRF-based remote visualization environment

Control

Control

Images

Images

Integration of other existing visualization software

(PyMOL, etc.)

GVS Architecture

Grid/Internet

Web/Grid ServiceMachine

Tomcat/Globus

VisualizationService

GVS ServerAdapter

jpg

Server Machine

GVS Server

IOIOOIOIIOIO

para

jpg

IOIOOIOIIOIO

para

User Machine

Web/Grid Client

VisualizationAPI

GVS ClientIOIOOIOIIOIO

para

GVS Standards• GGF Standards we help set in WG/RG

– SAGA W/RG & OGSA-Byte-IO WG• GGF and related Standards we employ

– GridMPI– WSRF 2.0– JSDL







SuperSINET



-Grid RPC -Grid MPI

Grid Visualization

Grid VM (WP1)

Packag

ing


(CIM)


Super Scheduler


Data (W

P4)WP1WP1

WP3

WP4

Data 1 Data 2 Data n

Grid-wide File System

MetadataManagement

Data Access Management

Data ResourceManagement

Job 1

Meta-data

Meta-data

Data 1

Grid Workflow

Data 2 Data n

NAREGI Data Grid beta1 Architecture (WP4)

Job 2 Job n

Meta-data

Job 1

Grid-wide Data Sharing Service

Job 2

Job n

Data Grid Components

Import data into workflow

Place & register data on the Grid

Assign metadata to data

Store data into distributed file nodes


Gfarm 1.2 PL4(Grid FS)

Data Access Management

NAREGI WP4: Standards Employed in the Architecture

Job 1

Data SpecificMetadata DB

Data 1 Data n

Job 1 Job nImport data into workflow

Job 2

Computational Nodes

FilesystemNodes

Job n

Data ResourceInformation DB

OGSA-DAI WSRF2.0

OGSA-DAI WSRF2.0

Globus Toolkit 4.0.1

Tomcat5.0.28

PostgreSQL 8.0

PostgreSQL 8.0

Workflow (NAREGI WFML =>BPEL+JSDL)

Super Scheduler (SS) (OGSA-RSS)

Data Staging

Place data on the Grid

Data Resource Management

Metadata Construction

OGSA-RSSFTS SC

GridFTP

GGF-SRM (beta2)

NAREGI WP4 Standards• GGF Standards we help set within a WG

– Grid FileSystems WG (discussion about functionality and usecase scenario)

• GGF and related Standards we employ– OGSA-DAI– OGSA-RSS– GridFTP– WSRF 2.0– JSDL– SRM (planned for beta 2)

• Other industry standards we employ– BPEL


Roadmaps, Future Plans• Extension of Data Sharing Service based on Grid

FileSystem– VO support based on VOMS and/or XACML VO group p

ermissions– Shared StorageResource Reservation for Work-Resour

ce Mapping (OGSA-RSS)• Data Transfer Service

– Stream-like Data Transfer for huge amount of data (OGSA ByteIO?)

• Virtualization and/or Integration of Metadata– For Data Exchange among different storage/file-syste

ms (SRM, SRB, …) – Logical Namespace Service (OGSA RNS, WS-Name)






SuperSINET



-Grid RPC -Grid MPI

Grid Visualization

Grid VM (WP1)

Packag

ing


(CIM)


Super Scheduler


Data (W

P4)WP1WP1

WP3

WP5

UserUser

Web Server

(PSE,WFT,GVS)

Policy Enforcement Point

CRL

Grid JobSubmission

OCSP/XKMS

SS

CA Service

VO and Certificate Management Service

User Authentication

Proxy

Certificate

XACML,ACL

GRAM (Grid VM)

Policy information description

CSR

NAREGI-1.0 Security Architecture

Credential Renewal

NAREGIIS

VO Context

VO Context

System

Administrator

System

Administrator

System

Administrator

System

Administrator

Discussed in GGF

SAML2.0

SAML2.0

GT4.2 or laterSecurity Attribute

(GGF AuthZ)

Accredited by APGridPMA

NAREGI-CANAREGI-CA

GCA

GPKI

CP/CPS

GRID CP(GGF CAOPs)

Audit Criteria

Subset of WebTrustPrograms for CA

GPMA

NAREGI-PMANAREGI-PMA

NAREGI-beta1 Security Architecture

NAREGICA

Client EnvironmentClient Environment

Portal WFT

PSE

GVM SS

clie

nt

ProxyCertificatewith VO


Super Scheduler



log-inlog-in

Request/GetRequest/Get CertificateCertificate

VOMSVOMS

CertificateManagement Server

CertificateManagement Server

ProxyCertificate

withVO

ProxyCertificate

withVO

UserCertificate

PrivateKey

Put ProxyCertificaPut ProxyCertificatete

with VOwith VO

Get VOMS Get VOMS AttributeAttribute

MyProxyMyProxyProxy

Certificatewith VO


voms-myproxy-initvoms-myproxy-init

InformationService

Resources InfoResources Infoincl. VOincl. VO

ResourceResource

GridVMlocal Info.local Info.incl. VOincl. VO

CA Service

ssh + ssh + voms-myproxy-initvoms-myproxy-init

ProxyCertificateProxyCertificatewith VOwith VO

queryquery(requirements(requirements+VO info)+VO info)

resourcesresourcesin the VOin the VO

globusrun-wsglobusrun-ws　　　　 GridVM serviceGridVM service

ss(incl. GSI)(incl. GSI)

Resource Info.Resource Info.(Incl. VO info)(Incl. VO info)

Signed Signed Job Job

DescriptionDescription

VO 、 Certificate

Management Service

VO InfoVO Info 、、 Execution Info,Execution Info,Resource InfoResource Info

CP/CPS

GRID CP(GGF CAO

Ps)

Audit Criteria

Subset of WebTrust

Programs for CA

WP5 SecurityRoadmaps, Future PlansTOPIC Standardizatio

n StatusStandardization

To GGF From GGF Comment

Attribute Propagation 　

－ ◎

Definition of VO － ◎

NAREGI Security Architecture

－ ◎

ID federation ◎ SAML2.0× Grid

－ ◎

Security Attribute(OGSA-AuthZ WG)

○ ◎ ◎

Policy Description － ○

NAREGI WP5 Network Architecture

Grid Network Service Components Grid Network Service Components

GridVMGridVM

ProviderService

ProviderService

ClientClient Super Scheduler Distributed Information ServiceDAI

CIM-XML(network topologyand measurement)

Workflow

GridVMGridVM

CIMProvider

CIMProvider

Grid Network Resource / Policy Manager Grid Network Resource / Policy Manager

Grid Network Resource DBGrid Network Resource DB

Network Monitoring ElementsNetwork Monitoring Elements

Activemeasurement

Activemeasurement

Passivemeasurement

Passivemeasurement

Real time flowmeasurement

Real time flowmeasurement

Network Control Elements

Network Control Elements

Path / BandwidthPath / Bandwidth

RTT/delay/utilizationRTT/delay/utilizationflow packet countsflow packet counts

NetworkNetworkControlControl

GridNetworkService

GridNetworkService

WSDLSuperSINETSuperSINET

Operators

GUIGUI

Visualization of Network Topology Visualization of Network Topology and Measurementand Measurement

NAREGI WP5 Network Standards

• GGF Standards we help set within a WG– Grid High-Performance Networking (We

presented network measurement, control and measurement architecture as NAREGI activities)

– Network Measurement (in Future)

• GGF and related Standards we employ– Grid network service use-case scenario between

NAREGI Grid middleware and grid network services (discussion currently and contribution in Future)

– IETF Network Standards such as OCSP

• Other de-facto “standards” we employ– Apache + Tomcat

NAREGI WP5 Sec Standards• GGF Standards we help set within a WG

– CAOPs,AuthZ(in Future)• GGF and related Standards we employ

– GRID CP[CAOPs]– OCSP (in Future)– Security Attibutes[AuthZ] (in Future)– Grid Policy Management Authority– Grid Certificate Authorities

• Other industry standards we employ– WebTrust Programs for CA(referred to define the grid audit cri

teria)• Other de-facto “standards” we employ

– GT4.2 or later(in Future)– MyProxy, VOMS– SAML 2.0(in Future)






SuperSINET



-Grid RPC -Grid MPI

Grid Visualization

Grid VM (WP1)

Packag

ing


(CIM)


Super Scheduler


Data (W

P4)WP1WP1

WP3

WP6

InformationService

NAREGI Application Mediator (WP6) for Coupled Applications

Job 1

WorkflowNAREGI WFT

co-allocated jobs

Super Scheduler

GridVM GridVM GridVM

Simulation ASimulation ASimulation A

Mediator

Simulation ASimulation ASimulation B

MediatorA

MediatorA

MediatorA

MediatorB

MediatorB

Sim.B

Sim.B

Sim.A

Sim.A

Sim.A

Support data exchange between coupled simulation

ＭＰＩ

OGSA-DAI WSRF2.0

SQL

JNI

・ Semantic transform- ation libraries for different simulations

Data transfer management

・ Synchronized 　　　　　 file transfer

Mediator Components

・ Multiple protocol GridFTP/MPI

Job n

ＭＰＩ

Data transformationmanagement

・ Coupled accelerator

ＭＰＩ

GridFTP

MediatorMediator

Globus Toolkit 4.0.1

GridFTP

APIAPI

Data1

Data2

Data3

*SBC: Storage-based communication

SBC* -XML・ Global Job ID・ Allocated nodes・ Transfer Protocol etc.

NAREGI WP6 Standards

• GGF and related Standards we employ– GridFTP– OGSA-DAI– WSRF 2.0 　

• Other industry standards we employ– MPI Standard 1.1– SQL

• Other de-facto “standards” we employ– JNI 1.2 (JavaTM Native Interface)– Globus Toolkit 4.0.1

Lessons Learned• Relieving Delivery pressure

– Standards unavailable, process slow => “screw with it” attitude

– => Long-term planning, staged delivery plans• Time/Money/Personnel Investment

– => Commitment from leadership & companies– => made part of yearly budget/plan

• Valuation/Accreditation– May conflict with academic valuations– => Students not involved, only “established” academia and

commercial pros– => made primary metric for project success

NAREGI and Standards: Summary

• Interoperability and longevity is the key for grid adoption

• NAREGI believes, as a major national grid project, that committing to standards is the best bet for interoperation, industry adoption, and resulting longevity– Will be working next 5 years to set and adopt

more standards• Will be working with other parties, within

and outside GGF to achieve common standards

Backups

NAREGI Software Stack (beta 1 2006)





SuperSINET



-Grid RPC -Grid MPI

Grid Visualization

Grid VM (WP1)

Packag

ing


(CIM)


Super Scheduler


Data (W

P4)

WP3

WP1WP1

Grid Interoperations---Ideal World: Ubiquitous VO & user management

for international e-Science

Europe: EGEE, UK e-Science, …

US: TeraGrid, OSG,

Japan: NII CyberScience (w/NAREGI), … Other Asian Efforts (GFK, China Grid, etc.)…

Gri

d R

eg

ion

al In

frast

ruct

ura

l Eff

ort

sC

olla

bora

tive t

alk

s on

PM

A,

etc

.

HEPGridVO

NEES-EDGridVO

AstroIVO

Standardization,commonality in software platforms will realize this

All differentsoftware stacksbut users don’t see them

http://www.ggf.org/index.php

The Reality: Convergence/Divergence of Project

Forces(original slide by Stephen Pickles, edited by Satoshi Matsuoka)

NGS(UK)

TeraGrid(US)

OSG(US)

EGEE(EU)

LCG(EU)GridPP(UK)

OMII(UK)

GT4 WSRF (OGSA?)

gLite / GT2

Own WSRF & OGSA

Globus(US)

GGF

DEISA(EU) common staff & procedures

common users

NAREGI (JP)

interoperable infrastructure talks

WSRF & OGSA: GT234/Unicore, GT4/Fujitsu WSRF

NMI(US)

IBMUnicore

CSI (JP)

UniGrids(EU)

WS-I+ & OGSA?

AIST-GTRC



Condor(US)

NAREGI is/has/will…• Is THE National Research Grid in Japan

– Part of CSI and future Petascale initiatives– METI “Business Grid” counterpart 2003-2005

• Has extensive commitment to WS/GGF-OGSA– Entirely WS/Service Oriented Architecture– Set industry standards e.g. 1st impl. of OGSA-EMS

• Will work with EU/US/AP counterparts to realize a “global research grid”– Various talks have started, incl. SC05 interoperability meeting

• Will deliver first OS public beta in May 2006– To be distributed @ GGF17/GridWorld in Tokyo

NAREGI is not/doesn’t/won’t…

• Is NOT an academic research project– All professional developers from Fujitsu, NEC, Hitachi, (No

students)– IMPORTANT for Vendor buy-in and tech transfer

• Will NOT develop all software by itself– Will rely on external components in some cases– Must be WS and other industry standards compliant– IMPORTANT for Vendor buy-in and tech transfer

• Will NOT hinder industry adoption at all costs– Intricate open source copyright and IP policies– We want companies to save/make money using NAREGI

MW– IMPORTANT for Vendor buy-in and tech transfer

Nano-Science : coupled simluations on the Grid as the sole future for true scalability

The only way to achieve true scalability!

--- Coupled Simulations as “Killer Apps” for Grid ---

10 -6 10-9 m

Material physics 的 (Infinite system) ・ Fluid dynamics ・ Statistical physics ・ Condensed matter theory　　…

Molecular Science ・ Quantum chemistry ・ Molecular Orbital method ・ Molecular Dynamics　　 …

Multi-PhysicsMulti-Resolution

Limit of Computing Capability

Limit of Idealization

Coordinates decoupled resources on Grid;

　　 Meta-computing, High throughput computing,

　 Multi-Physics simulationw/ components and data from different groups

within VO composed in real-time

Old HPC environment:・ decoupled resources,・ isolated users,・ special software, ...

・ no coupling!

DistributedServers

LifeCycle of Grid Apps and Infrastructure

MetaComputing

Workflows and Coupled Apps / User

Many VO Users

Application Contents Service

HL WorkflowNAREGI WFML (Unicore+)

Grid Info Service (CIM)

GridRPC/Grid MPI UserApps

UserApps

UserApps

VO Application Developers&Mgrs

GridVM+GRAM4

　 SuperScheduler(OGSA-RSS)


Meta-data

Meta-data

Meta-data

Grid-wide Data Management Service (GridFS, Metadata, Staging, etc.)

Place & register data on the Grid

Assign metadata to data Implementation of

OGSA-EMSArchitecture

GridVM+GRAM4

GridVM+GRAM4

Highlights of NAREGI Beta (May 2006, GGF17/GridWorld)

• Professionally developed and tested• “Full” OGSA-EMS incarnation

– Full C-based WSRF engine (Java -> Globus 4)– OGSA-EMS/RSS WSRF components– Full WS-Agreement brokering and co-allocation– GGF JSDL1.0-based job submission, authorization, etc.– Support for more OSes (AIX, Solaris, etc.) and BQs

• Sophisticated VO support for identity/security/monitoring/accounting (extensions of VOMS/MyProxy, WS-* adoption)

• WS- Application Deployment Support via GGF-ACS• Comprehensive Data management w/Grid-wide FS• Complex workflow (NAREGI-WFML) for various coupled simulatio

ns• Overall stability/speed/functional improvements• To be interoperable with EGEE, TeraGrid, etc. (beta2)

adopting and setting standards the naregi strategy satoshi matsuoka professor, global scientific...

Documents

academic project

aoyagi kyushuu participation

vendors productization

adoption critical

dave snelling slide

tf national aaa infr

commercial entities

mil us