reusuable grid portals

8/14/2019 Reusuable Grid Portals

1/38

OGCEGCEConsortium

Reusable Grid Portal

Components

Mary Thomas


2/38

OGCEGCEConsortium

Outline

The Portal Challenge

The Solution:

The Open Grid Computing Environments

Consortium (OGCE)

The OGCE Portal Architecture

Application Portal Examples

NMI OGCE Plans


3/38

OGCEGCEConsortium

The Portal Challenge


4/38

OGCEGCEConsortium

Po

rtal

UserInterfa

ce

Grid Resource

Broker Service

Grid and

Web

Protocols

Information

and

Data Services

Database

Service Database

HPC

or

Compute Cluster

Grid InformationServices, SRB

Portal

Client

Stub

Portal

Client

Stub

Portal

Client

Stub

JDBC,Local, or

Remote

Connection

Three-Tiered Architecture

Three-tiered architecture is accepted standard foraccessing Grid and other services


5/38

OGCEGCEConsortium

Portal are Gateways to Grid

Early Grid Portals were great success: Applications oriented,

Provided scientists with some useful tools

Encapsulated some complexities.

But they were:

All stovepipe solutions.

Very little reusable code and infrastructure

Nothing could be shared between portals

Many ignored emerging architecture of the Grid

A framework based on distributed ubiquitous services


6/38

OGCEGCEConsortium

Impact of Distributed Grid and Web

Services

Resource layer

1000s of PCs ->massive supercomputers

Grid Portals

Launch, configure

And controlApplication Interface

Workflow service

App Instance

App Instance

App Instance

Security

Security

Data Management

Service

Data Management

Service

Accounting

Service

Accounting

Service

Logging

Logging

Event/Mesg

Service

Event/Mesg

Service

Policy

PolicyAdministration

& Monitoring

Administration

& MonitoringGrid Orchestration

Grid Orchestration

Registries and

Name binding

Registries and

Name binding

Reservations

And Scheduling

Reservations

And Scheduling

Open Grid Service Architecture Layer

Open Grid Service Infrastructure (web service component model)

Online instruments

(slide by Dennis Gannon)


7/38OGCEGCE

Consortium

Advantage of Portlet Architecture

Portlet/component approach very compatible withWeb/Grid services model

Each Grid service can be associated with a uniqueportlet Very easy to add new services

Many different Groups can contribute portlets which can beplugged into a portal. Currently Indiana, Argonne, Michigan, NCSA,Texas + GridSphere

portal group following this model. Most likely others as well.

Jetspeed is Apache standard. Basis for IBM WebSphere portal.Also Oracle, Sun and others.

Now a Java Standard (waiting for JSR 168) User can select and configure portlets he/she wishes

to use Selection becomes part of persistent context


8/38OGCEGCE

Consortium

The Next Generation Grid Portal

Building on Standard Technologies Portlet Design (JSR-168) IBM, Oracle, Sun, BEA, Apache Grid standards: Java CoG, Web/Grid Services Web server: JetSpeed (open source)

User configurable, Service Oriented Philosophy: The Portal is a gateway to distributed Grid and Web

Services

With common API, portlets can be exchanged, interoperate

Portal Server

MyProxy

Server

Metadata

DirectoryService(s)

Directory

& index

Services

Application

Factory

ServicesMessaging

and group

collaboration

Event and

logging

Services


9/38OGCEGCE

Consortium

User Requirements

Have Not Changed

Users & Clients are: Portal clients/users Portal developers

Security Single sign-on, authentication to all grid resources

Certificate based / uses MyProxy VO management

File Management Grid FTP File staging

Metadata management Remote Job Management

Job submission, scheduling, and tracking. Log and output mgmt. Workflow management tools


10/38OGCEGCE

Consortium

User Requirements

Information Services Grid resource and status information directories

Data and metadata catalogs

Personal and VO metadata and service index/directories

Database Interfaces

Application Interfaces

Configure and launch workflow from domain specific interface

Access to Collaboration Tools

VO collaboration. Synchronous and asynchronous Access Grid venues

Developers need common API


11/38OGCEGCE

Consortium

The Solution:

The Open Grid Computing

Environment (OGCE) Consortium


12/38

OGCEGCEConsortium

Open Grid Computing Environments

(OGCE) Consortium

Motivation: Emerging technologies impacting portal developers:

Grid/Web services; ubiquitous database use, XMl

Lessons learned from GCE-RG and other projects

(GridPort/HotPage, Alliance Portal, DOE Components,NEESGrid, Gateway, GRaDS, others)

Multiple projects repeating similar tasks, work,

development;

OGCE established in Fall 2003 to foster

collaborations and sharable components with

portal developers worldwide.


13/38

OGCEGCEConsortium

OGCE

Funded by: NSF NMI: software releases will be part of NMI

and synchronized with GRIDS releases

Also: NCSA Alliance, NPACI, DOE, NASA,others

Release Schedule:

First Release: 11-15-03

Next Release: April, 2004

Website: http://www.ogce.org


14/38

OGCEGCEConsortium

OGCE Tasks/Goals

Development of interoperable Grid portalarchitecture Define standards for common APIs between portlet

service and remote services

Establishment of a Grid Portal Collaboratory Repository of portlet and portal service components

An online forum for developers of Grid Portals The building of reusable portal components:

Can be integrated in a common portal container system

Contribute to standards organizations: NSF NMI program software releases GGF if appropriate or applicable


15/38

OGCEGCEConsortium

The OGCE Portal Architecture


16/38

OGCEGCEConsortium

OGCE Initial Architecture

Porta

l

LocalPortlets

Teamlets

Proxy

Portlets

JetspeedInternal

Services

Java

COG

API

JavaCoG

Kit

Grid Services

Grid

Protocols

GRAM,MDS-LDADMyProxy

Service

API

CHEF

Services

Remote

In

terfaces

CoG

Stubs

HTTP

Grid Services

Other ServicesSOAP

Initial architecture aggregates multiple services into a single

portal using portlet containers

OGCE I iti l A h


17/38

OGCEGCEConsortium

OGCE Initial Arch:

Features & Capabilities

OGCE approach encompasses multiple portaldevelopment systems:

Preserving development investments

Central portal system (Jetspeed) acts as a central

portal container: Portal responsible for managing the lifecycle of pluggable portal

components, or portlets.

Grid/computational portlets must provide services on

the behalf of the user: Not just a collection of links/static pages

User view (from the left) is of a single portal composed

of portlets.

S t A ti f


18/38

OGCEGCEConsortium

Supports Aggregation of

Diverse Tasks

Portlets provide a uniform management interface tothe portal (on the left) connect to heterogeneous set ofAPIs on the right. First horizontal thread

Portlets interact with web interfaces written in other

languages such as Perl, Python, PHP, ColdFusion, etc. Architecture for bridged support is depicted in the secondhorizontal thread

Examples: Locally running Myproxy Grid portlet uses Java CoG kit to interact

with remote Grid services

Job Scheduler portlet uses GridPort Toolkit to submit job viaCommunity Scheduling Frameworks (CSF) service

Teamlets (sharable portlets) share collaboration and Grid services,such as calendars, message boards, file services.


19/38

OGCEGCEConsortium

OGCE - Revised Architecture

Revised portal architecture provides a unified interface for portal services. Supports aggregation of services in backend

Portal

Portlets

and

Teamlets

JetspeedInternal Services

Service

API

GridProtocolsGrid

Service

Stubs

Remote

Content

Services

Remote

Content

ServersHTTP

Grid

ServicesJava

CoG

Kit

Local

Portal

Services


20/38

OGCEGCEConsortium

Features

All services will employ a single (generic)service API that allows them to be deployedand accessed in a uniform fashion.

Developer then chooses (during portal

configuration) the implementations ofparticular services. Services may be local, may access remote

sites via Grid protocols, or may access remotecontent, such as web interfaces written in Perl.

OGCE Twist: Entire portal will auto-install and include all required

software components


21/38

OGCEGCEConsortium

Key Technologies????

Core Portal Framework: CHEF/JetSpeed (U. Mich)

Core Grid Technologies:

JavaCoG (provides Globus interface)

GridPort Toolkit (aggregates JavaCoG, and

other grid services)


22/38

OGCEGCEConsortium

Portal/Application Gallery


23/38

OGCEGCEConsortium

TeraGrid Portal

Access Point for TeraGrid

Services Version 0: Collecting Initial

Services Public Information about

Resources

Private Information for thedevelopers.

Version 1: User centered portal HotPage/GridPort style access to

user accounts, credentials, jobsubmission & management.

Version 2: Portals for ScienceCollaborations (Q3 2004) Shared spaces, whiteboards, AG

access, group authorization,shared application services


24/38

OGCEGCEConsortium

TeraGrid Integrated Architecture

Diagram demonstrates how existing software projects (such as GridPort) can

be adapted to support NMI Portals software system

Portal

Portlets

and

Teamlets

JetspeedInternal Services

GridService

Stubs

Remote

Content

Services

Remote

Content

ServersHTTP

GridServices

Java

CoG Kit

Local

Portal

Services

Service

API

GridPortToo

lkit

Web

Services


25/38


26/38

OGCEGCEConsortium

DOE Fusion Grid Portal

Portal Development onthe DOE SciDAC Grid

Initially, Fusion Grid

Multi-site collaborative

project: TACC, Indiana, SDSC,

General Atomics

Developing interoperable

portal and Web services

Portal architecture based

on NMI Portal software


27/38

OGCEGCEConsortium

LEAD Portal


28/38

OGCEGCEConsortium

CHEF - NEESGrid

NEES Tools Data Viewers (Live/Stored)

Telepresense

Video as Data

Electronic Notebook Data/Metadata Repository

Grid Technologies

Grid sign on using myproxy

Grid computational portal

GridFTP

..Many more


29/38

OGCEGCEConsortium

OGCE Plans


30/38

OGCEGCEConsortium

OGCE Portal System

System

Integration

OGSAGlobus

COG

CHEF

Toolkit

Gridport

Alliance

Portal

Workflow

Testing

NMI

Dist

Slide courtesy of Chuck Severance


31/38

OGCEGCEConsortium

NMI Features List (April 04 Release)

Content Management: Resources: Users and groups can upload and share documents andURLs.

News: Several news feeds are available, including XMLHack and BBCNews.

Grid Context: Store arbitrary web objects (movies, web pages, audio

files) in a customizable, annotated directory tree. Collaborative:

Chat: Engage other members of your group in online discussion.

Discussion: Participate in discussions with other members of yourgroup

Calendar Program:schedule appointments and attach associated

documents and URLs. Calendars can be used to schedule both userand group activities.

Anabas Impromptu: system for real time shared display, audio, andchats.


32/38

OGCEGCEConsortium

NMI Features List (April 04 Release)

GT 3.0/Interactive: Proxy Manager and Job Submit: Obtain and manage one or more Grid credentialsthat can be used to access Grid resources through your browser.

GridFTP: Use Grid credential to browse remote directories and to upload anddownload files between your desktop and a remote host.

Application Manager Application Factory Manager

Remote Job Management GridAnt for workflow Job submission portal Community Scheduling Framework (CSF): submit jobs to scheduler

Information Services: LDAP Browser: Navigate the LDAP server of your choice.

GridPort Information Repository Portlets (GPIR): Portlets are available for browsingMachine Summaries, System Status, and Total Grid Capabilities.

Application Portlets: Specialized portlets for projects

Key Feature


33/38

OGCEGCEConsortium

Key Feature

Download a Portal & Install

Prerequisite Software Java 2 Standard Edition version

1.4.x or higher. Apache Ant version 1.5.x or higher.

Preparing the Installation Optionallly, create a "portal"

account on your web server.

Download the portal bundle Put the download bundle into thehome directory of "portal" account.

Untar or unzip the portal bundle. Globus: installation process will

create the directories .globus and.globus/certificates in the

Software automatically installedincludes: Globus/Java CoG CHEF

Install the Portal Move into the portal/chef-1.0.7

directory. Edit file "build.properites" Type "ant deploy" to build in and

install the portal. Start the Tomcat web server.

Point your browser at http://:10081/ogce and createan account.

OS/Machines Tested: MAC OSX PC/Windows XP Linux

Cygwin


34/38

OGCEGCEConsortium

Current Status

SC03 Demonstration portal and software downloadavailable Beta1

Updated Release available now Bug fixes

Now supporting GT3.x with J2SDK 1.4.2 Working on NMI April 2004 release

Including new features to support science portal applications Open GCE Runtime Engine (OGRE) workflow services and portlets Sample template portlets for simple science applications

Based on aggregation of existing projects OGSA/OGSI/GT3.x compatible


35/38

OGCEGCEConsortium

Future Directions

Moving towards new portlet standard JSR 168 Should allow compatibility with

GridSphere Academic education and business portals Oncourse, OneStart, CHEF

(of course), and uPortal

Major commercial products offerings from Sun, IBM, Oracle, etc. Incorporate new features:

Grid Services (CSF, OGSA-DAI portlets, Job Sequencer) Access Grid, Polycom and other AV Portlets

Collaborations and Contributions

DOE SciDAC projects (such as CMCS) Expand online repository

Develop process for third party contributions


36/38

OGCEGCEConsortium

Future Directions

Moving towards new portlet standard JSR 168 Should allow compatibility with

GridSphere Academic education and business portals Oncourse, OneStart, CHEF

(of course), and uPortal

Major commercial products offerings from Sun, IBM, Oracle, etc. Incorporate new features:

Grid Services (CSF, OGSA-DAI portlets, Job Sequencer) Access Grid, Polycom and other AV Portlets

Collaborations and Contributions

DOE SciDAC projects (such as CMCS) Expand online repository

Develop process for third party contributions


37/38

OGCEGCEConsortium

Participants

Indiana University

University of Michigan

University of Texas

at Austin

Argonne National Labs

JavaCoG

National Center for

Supercomputing Applications

the globus

alliance


38/38

reusuable grid portals

Documents