Common Component Architecture for High-Performance Computing

CCA Working Group

info: Rob Armstrong, rob@sandia.gov

http://www.acl.lanl.gov/cca-forum

What’s the problem?

HP computing has as much need for component arch’s as common computing does

Complex and hard to program algorithms, data structures.

Time-to-market is the consideration (e.g. SSI, ASCI curves).

Currently little reuse of code. attempts: PetSc, POET, POOMA most HP applications are written from the ground up

for a single purpose.

10 -5 s 1 s10 -1 s10 -2 s10 -3 s10 -4 s

D O E apps

C C A /E S I/...

G lobus /Leg ion

C O R B A /D C O M /...

Latency R equirem ents/C apabilities

C om m erc ia lD is tribu ted O b jec t

S ys tem s

D O ED is tribu ted O b jec t

S ys tem s

D O E A pp lica tionsIn te rac tive apps

N one A va ilab leN one P lanned

D O E S im u la tionsP ara lle lA pp lica tions

W eb S earch E ng ine

HP Component Architecture meets a critical DOE need that is not and will not

be addressed by Silicon Valley.

The Common Component Architecture Requirements

Simple to adopt to understand to use

Support a composition mechanism that does not impede high-performance component interactions.

Language interoperability Fortran, C, C++, Java interface introspection

Meant to live with and rely on other component frameworks to provide services ...

and other complicated, hard-to-implement stuff.

The problem with existing Component Architectures Overall:

hi-latency connection is OK for loosely-coupled app’s: network OK for component composition

not OK for high-performance app’s.

Specifics JavaBeans: composition only by events/listener. COM: no real interoperability, not vendor neutral. CORBA2: no component model. CORBA3 components: OK but complicated, only for loosely-

coupled app’s.

Seek to supplement existing CA’s with an HP add-on.

Who are we? Researchers in the HP components field that

are dedicated to forming an open standard for HP components.

addressing the concerns of HPC.

Originated from DOE’s DOE2K program has its place in ASCI: Software Integration Curve. participation from universities.

CCA Active Participants *

Dennis Gannon (Indiana U) Randall Bramley (Cal Tech) Andy Cleary, Scott Kohn, Brent Smolinski (Livermore) Kate Keahey, Pete Beckman, Bill Humphrey, Pat Fasel,

(Los Alamos) Rob Armstrong, Robert Clay, Carl Melius, Noel Nachtigal

(Sandia) Al Geist, Jim Kohl (Oak Ridge) Lois Curfman McInnes, Barry Smith, Paul Hovland Brian

Toonen (Argonne) Steve Parker (U Utah) John Wu (Lawrence Berkeley) Jarek Nieplocha (Pacific Northwest Labs)

* Contact Rob Armstrong (rob@sandia.gov) if interested in joining the CCA mailing list.

Scientific IDL provides OO interoperability to C, C++, Java, Python, Fortran 77/90

Follows a Java-like interface model that will “glue” languages together with a minimum performance hit

expected overhead: ~2-3 virtual function calls

IDL interoperability library supports interface introspection

Necessary for run-time discovery of objects by frameworks

Allows interfaces to be “recognized” by the framework without having specific advanced knowledge.

No plan for world domination... do not try to

organize complexity top-down.

Provide a medium of exchange

believe in Adam Smith’s invisible hand ...

What have we come up with?

Scientific IDL spec. provides language

interoperability only - not part of the component arch. per se.

introspection. Metaphor from visual

programming (e.g. AVS) instead of data-flow ports

are linked with interfaces Provides/Uses design

pattern 0.5 CCA specification:

Flow Executive

Com ponent

Link

Com ponent

Link

Com ponent

P

P

P

P

Port

http://z.ca.sandia.gov/~cca-forum/port-spec

Port model hooks up an interface from one component to another

Target Com ponent

Input Port

Int foo(float)float bar(int)

F lub(float)dub(int)

Source Com ponent

Port *o= new Port(“foobar”);

int x;x = o->foo(3.15)

float y;y = bar(3);

O utput PortInt foo(float)float bar(int)

Ports preserve the high-performance of direct connections plus versatility of distributed object systems

Allows for directly connected interfaces: the next component is one function call away.

Adapters will create network-distributed objects out of the same components without altering them.

C om ponent 1

U sesP ort

C om ponent 2

P rov ides-P ort

C om ponent 1 C om ponent 2

P rovides/U ses P ort

Generalized ports and the provides/uses design pattern for coupling components

Component 1 Component 2

CCAServices2

CCAServices

4

registerUsesPort("A")1

Port

Port

Port

addProvidesPort( ,"A")

= getPort("A")Port

3

Where are we going?

•Solving the MxM and MxN problem

•SC99 reference implementation of CCA with Equation Solver Interface ESI component.

•experiments with the architecture.

BoneYard

An IDL compiler automatically generates “glue” code for supported languages

IDL Compiler

IDL InterfaceDescription

StubGlue Code

SkeletonGlue Code

CallerLanguage

LibraryLanguage

Language MappingsCompiler Details

IDL Database

called bylibrary user

wraps libraryimplementation

IDL

(e.g. code written in C)currently C and f77-C++, Java soon

Ports can be used to characterize potential links to other components Each CCA Component has

A list of “input” Ports where• An input gPort defines an interface to

services provided by the component. A list of “output” Ports where

• An output gPort is the interface to a set of services required by the component.

Other “self-identification” features • allow component containers to probe

the component properties. This may be rolled into its own gPort.

Client OutputPort

Server InputPort

CCA is a component architecture that does not dictate frameworks or runtime

We are trying to create components that are usable under a variety of frameworks;

We are not making a new framework implementation.

provide a means for discovering interfaces, similar to IUnknown.

specifically exclude how the components are linked, that is the job of a framework.

provide language-independent means for creating components from existing and future languages.

Where are we? Details of the CCA specification ... Overall: http://www.acl.lanl.gov/cca-forum draft Port spec in RFC stage:

http://z.ca.sandia.gov/~cca-forum/port-spec

draft IDL spec in RFC stage: http://www.llnl.gov/CASC/babel

CCA Demo applet & plug-in: http://z.ca.sandia.gov/~cca-forum/cca-demo/

Multiple test codes already exist. Working on repository/naming issues.

(1) Add pea to pod component

Pea

Pod

(2) Create input and output port objects via introspection

Port

Port

(3) Connect components

User-created

portPort

Programmer can create a new Port by tying methods to a Port

Conclusions

Philosophy: Not trying to “own the world.” Trying to provide a “coin-of-the-realm” -- a medium of exchange -- that promotes an HPC software economy.

Overall: http://www.acl.lanl.gov/cca-forum Draft Port spec in RFC stage:

http://z.ca.sandia.gov/~cca-forum/gport-spec

Draft IDL spec in RFC stage: http://www.llnl.gov/CASC/babel

CCA Demo applet & plug-in: http://z.ca.sandia.gov/~cca-forum/cca-demo

SPMD view ...

Data

Calc CalcSteering/

Viz

2

Data

Calc CalcSteering/

Viz

2

Data

Calc CalcSteering/

Viz

2

Different processes

What is a software framework and components

Component Concepts Autonomous interchangeable

parts Links between components that

create an application Builders Framework provides context or

environment to components

Frame

ComponentComponentComponentComponents

Characteristics of Component Arch’s

Composition Phase

Run Phase

fram e

eqn solver

load balancer

stencil m ethod

fram e

eqn solver

load balancer

stencil m ethod

Language interoperability is a critical first step towards software interoperability

Software re-use is often hampered by language barriers DOE labs use many languages (f77, f90, C, C++, Java, Python) can be difficult for some languages to call others (f77 to C++)

We are developing IDL technology for interoperability interface definition language (IDL) describes calling interface tools automatically generate code to “glue” languages

A. Cleary, S. Kohn, S. Smith, B. Smolinski, Language Interoperability Mechanisms for High-Performance Scientific Applications, SIAM Interoperability Conference, 1998.

C

C++

f77

f90

Python

Java

IDL

CCA Ports employ a Uses/Provides design pattern for connections

C C A S erv ices

U sesP ort

4 Port

C C A S erv ices

P rov idesP ort

Port

Port1

2

3

FrameWork

Port

One component uses an interface provided by another.At the containing framework’s option:

• Allows direct connections.•Allows connections through remote proxies

all with the same components

Where we are going ...

Port and IDL spec’s need to be solidified time and testing are now the big contributors to the

spec. Publication: Web, HPDC in Aug, San Diego,

demo SC99.

CCA as a standards body soliciting, accepting/rejecting proposals of

interfaces and implementations for inclusion into the core CCA.

Command-line applet that implements the specification

link together pretend components to do string manipulations Source code available

Component - Encapsulated software object that provides a certain functionality or service and can be used in conjunction with other components to build applications

A component consists of an API (abstract interface) and one or more component implementations common behavior among these components

CCA uses a peer component model - No particular component assumes it is “in charge” of the others

By definition components are software modules that are composable...

ESI is a multi-lab effort writing an equation- solver component specification

DOE initiative, building on previous efforts.

SNL Ben Allan, Robert Clay, Lee Taylor, Alan Williams

LLNL Colin Aro, Andy Cleary, Rob Falgout, Juliana Hsu

LANL Mike DeLong, Robert Ferrell, Wayne Joubert

ANL Barry Smith, Lois Curfman McInnes

ORNL Noel Nachtigal, Jack Dongarra

LBL John Wu

CSU Chico Kyran Mish

U.Indiana Randy Bramley

technical forum: if-forum@z.ca.sandia.gov

(http://z.ca.sandia.gov/esi)

Equation Solver Interface (ESI) group is defining interfaces for solvers

CCA CoreObject

Operator PreconditionerVector

MatrixSolver

Map

Matrix IF1 Matrix IF2 Matrix IFn

Matrix Im plem entation 1

Matrix IF ...

Matrix Im plem entation 2

The goal of CCA group is enable applications constructed entirely of components

Discretization

Algebraic Solvers

Parallel I/O

Grids

Data Reduction

Physics Modules

Optimization

Derivative Computation

Collaboration

Diagnostics

Steering

Visualization

Adaptive Solution

gPorts connect component interfaces in a data-flow-like way

Needs to work for tightly coupled and dist. objects tightly coupled for performance

• allow overhead approx. a virtual function call transparently marshal args and return values

• transparency over various networks and environments

gPorts is intended to provide a more efficient and scalable component connection mechanism

Input/Output gPorts can call each other explicitly and directly.

Recent (7 Nov ‘98) CORBA3.0 spec hints at similar “port” concept.

Where we are now ...

draft gPort spec in pre-RFC stage:http://z.ca.sandia.gov/~cca-forum/gport-spec

draft IDL spec in pre-RFC stage: http://www.llnl.gov/CASC/babel

Multiple test codes already exist. experiments with the Common Component

Architecture.

Working on repository/naming issues. Most likely to be developed from LDAP or JNDI.

Where we are going ...

gPort and IDL spec’s need to be solidified a formal RFC must be accomplished for both IDL

and gPorts. will be tested in the working group’s own frameworks

and refined from real-world experiences. the work will be published and advertised in

appropriate places.

Repository/Naming Issues Addressed Make appropriate hooks into the spec. to import a

“standard” registry/repository/naming scheme.

CCA is our only hope Here is the list of pan-lab HP software integration

groups:• ESI - Equation Solver Interface specific only to this one field

• CCA

The outside world is componentizing and integrating software, we cannot remain in the one-lab-one-code mode forever.

DOE has special needs with regard to HP software that the outside world is not addressing.

Conclusion

http://www.acl.lanl.gov/cca-forum

(1) Add pea to pod component

Pea

Pod

(2) Create input and output port objects via introspection

Port

Port

(3) Connect components

User-created

portPort

Programmer can create a new Port by tying methods to a port

CCA Plans: A Big Picture

Vector

component Linear solver

component

Non-linear solver

component

Multi-component interoperability

Rob:

Paul’s from Barry

Rob:

Paul’s from Barry

IDL specification for the SMG structured multigrid preconditioner from hypre

package hypre { class stencil { static stencil NewStencil(in int dim, in int size); int SetStencilElement(in int index, in array<int> offset); }; class grid { static grid NewGrid(in mpi_com com, in int dimension); int SetGridExtents(inout array<int> lower, array<int> upper); }; class vector { static vector NewVector(in mpi_com, in grid g, in stencil s); int SetVectorBoxValues(/* long argument list omitted */); }; class matrix { /* matrix member functions omitted */ }; class smg_solver { int Setup(inout matrix A, inout vector b, inout vector x); int Solve(inout matrix A, inout vector b, inout vector x); };);

Common Component Architecture Working Group

http://www.acl.lanl.gov/cca-forum

Rob Armstrong

rob@sandia.gov

What we are doing

The problem Many incompatible frameworks. Much replication of s/w effort. Brave new world of network grids presents many

opportunities & challenges to interoperability.

The solution A component arch. enabling interoperability

between modules for scientific computing. Not defining a new framework, but a standard that

will work within many frameworks.

Frame

ComponentComponentComponentComponents

Who we are

Participants design and implement components/frameworks for their respective institutions.

know component arch’s and HP computing meeting DOE lab needs.

Venue: Meet every 3 months.

The goal of CCA group is enable applications constructed entirely of components

Discretization

Algebraic Solvers

Parallel I/O

Grids

Data Reduction

Physics Modules

Optimization

Derivative Computation

Collaboration

Diagnostics

Steering

Visualization

Adaptive Solution

Where we are now ...

draft Port spec:http://z.ca.sandia.gov/~cca-forum/port-spec

draft IDL spec: http://www.llnl.gov/CASC/babel

Multiple test codes already exist. Command-line demo:

http://z.ca.sandia.gov/~cca-forum/cca-demo

Working on repository/naming issues. Will use the existing Ports mechanism to access

these and other services.

Scientific IDL provides OO interoperability to C, C++, Java, Python, Fortran 77/90

Follows a Java-like interface model that will “glue” languages together with a minimum performance hit

expected overhead: ~2-3 virtual function calls

IDL interoperability library supports interface reflection Necessary for run-time discovery of objects by frameworks Allows interfaces to be “recognized” by the framework

without having specific advanced knowledge of active components

CCA Ports employ a Uses/Provides design pattern for connections

C C A S erv ices

U sesP ort

4 Port

C C A S erv ices

P rov idesP ort

Port

Port1

2

3

FrameWork

Port

One component uses an interface provided by another.At the containing framework’s option:

• Allows direct connections.•Allows connections through remote proxies

all with the same components

Where we are going ...

Port and IDL spec’s need to be solidified time and testing are now the big contributors to the

spec. Publication: Web, HPDC in Aug, San Diego,

demo SC99.

CCA as a standards body soliciting, accepting/rejecting proposals of

interfaces and implementations for inclusion into the core CCA.

CCA Active Participants *

Rob Armstrong, Robert Clay, Carl Melius, Bill Mason (Sandia)

Dennis Gannon (Indiana U) Randall Bramley (Cal Tech) Andy Cleary, Scott Kohn, Brent Smolinski

(Livermore) Kate Keahey, Pete Beckman, Bill Humphrey, Pat

Fasel, (Los Alamos) Al Geist, Noel Nachtigal (Oak Ridge) Satish Balay, Lois Curfman McInnes, Barry Smith,

Paul Hovland, Lori Freitag (Argonne) Steve Parker (U Utah) Brent Milne (Lawrence Berkeley) Jarek Nieplocha (Pacific Northwest Labs)

* Contact Rob Armstrong (rob@sandia.gov) if interested in joining the CCA mailing list.

gPort model hooks up an interface from one component to another

Targe t C om ponen t

Inpu t gP ort

In t foo (floa t)floa t ba r(in t)

F lub (floa t)dub (in t)

S ource C om ponen t

G port *o= new G port(“foobar”);

in t x;x = o -> foo (3 .15 )

floa t y ;y = ba r(3 );

O u tpu t gP ortIn t foo (floa t)floa t ba r(in t)

User applications can now invoke hypre routines from both C and Fortran 77

integer b, xinteger Ainteger smg_solver

b = hypre_vector_NewVector(com, grid, stencil)...x = hypre_vector_NewVector(com, grid, stencil)...A = hypre_matrix_NewMatrix(com, grid, stencil)...

smg_solver = hypre_smg_solver_new()call hypre_smg_solver_SetMaxIter(smg_solver, 10)call hypre_smg_solver_Solve(smg_solver, A, b, x)call hypre_smg_solver_Finalize(smg_solver)

hypre_vector b, x;hypre_matrix A;hypre_smg_solver smg_solver;

b = hypre_vector_NewVector(com, grid, stencil);...x = hypre_vector_NewVector(com, grid, stencil);...A = hypre_matrix_NewMatrix(com, grid, stencil);...

smg_solver = hypre_smg_solver_new();hypre_smg_solver_SetMaxIter(smg_solver, 10);hypre_smg_solver_Solve(smg_solver, &A, &b, &x);hypre_smg_solver_Finalize(smg_solver);

C Test Code Fortran 77 Test Code

The end user is completely unaware that IDL tools were used