alice workshop, aug 21, 20081 recent experiences in grid- enabling legacy software codes dana petcu...
Post on 15-Jan-2016
215 views
TRANSCRIPT
![Page 1: Alice Workshop, Aug 21, 20081 Recent experiences in Grid- enabling legacy software codes Dana Petcu petcu@info.uvt.ro petcu Prof](https://reader036.vdocuments.site/reader036/viewer/2022070412/56649d265503460f949fd6d3/html5/thumbnails/1.jpg)
Alice Workshop, Aug 21, 2008
1
Recent experiences in Grid-enabling legacy software codes
Dana Petcu
[email protected]://web.info.uvt.ro/~petcu
Prof. & Dir. of Computer Science Dep., West Univ. of Timisoara, www.uvt.roRes. & Dir. of research Institute e-Austria Timisoara, Romania, www.ieat.ro
![Page 2: Alice Workshop, Aug 21, 20081 Recent experiences in Grid- enabling legacy software codes Dana Petcu petcu@info.uvt.ro petcu Prof](https://reader036.vdocuments.site/reader036/viewer/2022070412/56649d265503460f949fd6d3/html5/thumbnails/2.jpg)
Alice Workshop, Aug 21, 2008
2
Content
Context Romanian sites & Grid initiatives UVT & IeAT Grid-related activities Grid – where to?
Cases studies of Grid-enabling existing codes For Computational Grids (CFDs, DLs etc) For Service Grids (EOs, CASs etc)
Conclusions
![Page 3: Alice Workshop, Aug 21, 20081 Recent experiences in Grid- enabling legacy software codes Dana Petcu petcu@info.uvt.ro petcu Prof](https://reader036.vdocuments.site/reader036/viewer/2022070412/56649d265503460f949fd6d3/html5/thumbnails/3.jpg)
Alice Workshop, Aug 21, 2008
3
RO sites in EGEE-II / Feb 08
![Page 4: Alice Workshop, Aug 21, 20081 Recent experiences in Grid- enabling legacy software codes Dana Petcu petcu@info.uvt.ro petcu Prof](https://reader036.vdocuments.site/reader036/viewer/2022070412/56649d265503460f949fd6d3/html5/thumbnails/4.jpg)
Alice Workshop, Aug 21, 2008
4
RO sites in EGEE-III / 20 Aug 08
Locations (SEE-Grid monitoring):
![Page 5: Alice Workshop, Aug 21, 20081 Recent experiences in Grid- enabling legacy software codes Dana Petcu petcu@info.uvt.ro petcu Prof](https://reader036.vdocuments.site/reader036/viewer/2022070412/56649d265503460f949fd6d3/html5/thumbnails/5.jpg)
Alice Workshop, Aug 21, 2008
5
UVT in SEE-Grid-Sci / 20 Aug 08
![Page 6: Alice Workshop, Aug 21, 20081 Recent experiences in Grid- enabling legacy software codes Dana Petcu petcu@info.uvt.ro petcu Prof](https://reader036.vdocuments.site/reader036/viewer/2022070412/56649d265503460f949fd6d3/html5/thumbnails/6.jpg)
Alice Workshop, Aug 21, 2008
6
Current infrastructureUVT-CS Department’s cluster IeAT’s cluster
(28 CPUs, gLite 3) (30 CPUs, GT 4)
![Page 7: Alice Workshop, Aug 21, 20081 Recent experiences in Grid- enabling legacy software codes Dana Petcu petcu@info.uvt.ro petcu Prof](https://reader036.vdocuments.site/reader036/viewer/2022070412/56649d265503460f949fd6d3/html5/thumbnails/7.jpg)
Alice Workshop, Aug 21, 2008
7
Recent history – UVT & IeAT team 2002
UVT’s cluster of 8 IBM PCs with Myrinet connection (2Gps) included in ASC facilities (NSF’s Astrophysics Simulation Collaboration)
2003 Test platform between UVT and RISC-Austria
2004 First prototypes of Grid-enabled codes (CFD + CAS) in
CompGrid project 2005
Started national collaborations MedioGrid (imag.proces.) http://mediogrid.utcluj.ro GridMOSI (optimization) http://www.gridmosi.ro
![Page 8: Alice Workshop, Aug 21, 20081 Recent experiences in Grid- enabling legacy software codes Dana Petcu petcu@info.uvt.ro petcu Prof](https://reader036.vdocuments.site/reader036/viewer/2022070412/56649d265503460f949fd6d3/html5/thumbnails/8.jpg)
Alice Workshop, Aug 21, 2008
8
Recent history – UVT & IeAT team 2006:
Involved in FP6 SCIEnce – SymGrid (symb. Comp) http://www.symbolic-computation.org
Involved in national SINRED prj. Digital lib. & Grids Started Grid training and dissemination activities (ISPDC06 symp., GridCAD ws, GridTDT training, PhD programme)
2007: Participation at FP6 EGEE-II and SEE-Grid-II Accepted in RoGrid consortium Participation at national prj. PEGAF (workflow &Grid services)
2008: Participation at FP7 EGEE-3 and SEE-Grid-Sci Leading the ESA’s project: GiSHEO http://gisheo.info.uvt.ro
![Page 9: Alice Workshop, Aug 21, 20081 Recent experiences in Grid- enabling legacy software codes Dana Petcu petcu@info.uvt.ro petcu Prof](https://reader036.vdocuments.site/reader036/viewer/2022070412/56649d265503460f949fd6d3/html5/thumbnails/9.jpg)
Alice Workshop, Aug 21, 2008
9
Application prototypes in the last year gLite oriented applications
DEMO/G – Distributed Evolutionary Multiobjective Optimization on Grid
GOC – portal for gLite jobs
GT4 oriented applications SymGrid-Services – Web and Grid services for
symbolic computations components of MedioGrid platform – Web and
Grid services for satellite image processing
![Page 10: Alice Workshop, Aug 21, 20081 Recent experiences in Grid- enabling legacy software codes Dana Petcu petcu@info.uvt.ro petcu Prof](https://reader036.vdocuments.site/reader036/viewer/2022070412/56649d265503460f949fd6d3/html5/thumbnails/10.jpg)
Alice Workshop, Aug 21, 2008
10
Back to the international context (1)
European Commission programs: FP6 funds:
Infrastructure (e.g. EGEE-II) IST – Grid Unit F2
FP7 funds: E-Infrastructure (e.g. EGEE-3) ICT – Service Oriented
Architectures (Grid Unit -> Service-Oriented Archs. Unit)
The business pressure companies promoting Web
services & SOA E.g. Aug’08 GridToday newsletter -> On-demand Entreprise Daily news EU projects promoting SOA in
FP7-ICT coming from enterprise-driven consortiums like NESSI
![Page 11: Alice Workshop, Aug 21, 20081 Recent experiences in Grid- enabling legacy software codes Dana Petcu petcu@info.uvt.ro petcu Prof](https://reader036.vdocuments.site/reader036/viewer/2022070412/56649d265503460f949fd6d3/html5/thumbnails/11.jpg)
Alice Workshop, Aug 21, 2008
11
Back to the international context (2) NSF’s Cyberinfrastructure Vision – March 07
Grid infrastructure = world-wide Cyber-utility populated by self-* cooperating services which interact as in a complex gigantic software ecosystem
European NGG3 expert group – Jan 2006 Grid services + Knowledge technologies = SOKU
Services are knowledge-assisted + Architecture comprises services + Utility: immediately useable service
Different Grids: Computational Grids Service Grids Instrument Grids …
New-related buzzwords: cloud computing, on demand computing (ODC) Service oriented computing (SOC) Service oriented infrastructures (SOI) Software as a Service (SaaS) …
![Page 12: Alice Workshop, Aug 21, 20081 Recent experiences in Grid- enabling legacy software codes Dana Petcu petcu@info.uvt.ro petcu Prof](https://reader036.vdocuments.site/reader036/viewer/2022070412/56649d265503460f949fd6d3/html5/thumbnails/12.jpg)
Alice Workshop, Aug 21, 2008
12
Legacy codes & Grids
Legacy system = a mission-critical software system developed sometime in the past that has been around and has changed for a long time without undergoing systematic remedial actions
Modernization techniques are applicable to legacy systems:1. Redevelopment, from scratch, using a new hardware platform and modern
architectures tools and database; 2. Migration that moves the system to a new platform, while retaining the
original system data and functionalities3. Wrapping, which surrounds existing data, programs, applications and
interfaces with new interfaces
Grid-related solutions Migration towards Grids in order to increase their ability to solve bigger
problems by (“gridification” for Computational Grids) Wrapping as Grid services is a cost-effective and quality solution for evolving
them in order to meet new requirements (“wrapping” for Service Grids)
![Page 13: Alice Workshop, Aug 21, 20081 Recent experiences in Grid- enabling legacy software codes Dana Petcu petcu@info.uvt.ro petcu Prof](https://reader036.vdocuments.site/reader036/viewer/2022070412/56649d265503460f949fd6d3/html5/thumbnails/13.jpg)
Alice Workshop, Aug 21, 2008
13
Gridification for Computational Grids1. Identify parallelism degree
2. Manage the issue related to the resources heterogeneity
3. Deal with intra-cluster and inter-cluster communications
4. Create interfaces for easy-to-use
![Page 14: Alice Workshop, Aug 21, 20081 Recent experiences in Grid- enabling legacy software codes Dana Petcu petcu@info.uvt.ro petcu Prof](https://reader036.vdocuments.site/reader036/viewer/2022070412/56649d265503460f949fd6d3/html5/thumbnails/14.jpg)
Alice Workshop, Aug 21, 2008
14
Example for 1. Parallelism degree Embarrassing gridification in analogy with
embarassing parallelism – no communication Digital libraries [in our case for SINRED project, developments on
RoGrid infrastructure based on gLite] Multimedia conversions for large collections of
documents (e.g. PDF -> audio) – considerable requirements in what concerns response time & storage
Applying OCR techniques on large collections of documents in order to create indexes (keywords, references etc), extract images etc
Used available free convertors
![Page 15: Alice Workshop, Aug 21, 20081 Recent experiences in Grid- enabling legacy software codes Dana Petcu petcu@info.uvt.ro petcu Prof](https://reader036.vdocuments.site/reader036/viewer/2022070412/56649d265503460f949fd6d3/html5/thumbnails/15.jpg)
Alice Workshop, Aug 21, 2008
15
Example for 2. Manage heterogeneity Stage 1 – 2003 parallel version of STHAMAS 3D
commercial CFD simulator for crystal growing
three-dimensional finite-volume code
based on a block-structured finite-volume Navier-Stokes solver Stage 2 – 2004 (CompGrid project)
adapt the parallel code to use heterogeneous computing resources improved efficiency through avoiding load balancing problems
![Page 16: Alice Workshop, Aug 21, 20081 Recent experiences in Grid- enabling legacy software codes Dana Petcu petcu@info.uvt.ro petcu Prof](https://reader036.vdocuments.site/reader036/viewer/2022070412/56649d265503460f949fd6d3/html5/thumbnails/16.jpg)
Alice Workshop, Aug 21, 2008
16
Example for 3. Intra/Inter-cluster comm.DEMO/G (in GridMOSI project)Distributed Evolutionary Multiobjective
Optimization on Grid= implementation of evolutionary algorithms
for multiobjective optimization (EMOA) based on a layered structure First layer: sequential EMOA executed on
one processor Second layer: parallel EMOA based on a
colony of inter-communicating populations
Third layer: multiple colonies which evolves separately
Used for multiobjective multidisciplinary optimization (case study: airfoil design) Evaluation of objective functions is based
on numerical simulations; coupling between EMOA and other codes
e.g. CFD
![Page 17: Alice Workshop, Aug 21, 20081 Recent experiences in Grid- enabling legacy software codes Dana Petcu petcu@info.uvt.ro petcu Prof](https://reader036.vdocuments.site/reader036/viewer/2022070412/56649d265503460f949fd6d3/html5/thumbnails/17.jpg)
Alice Workshop, Aug 21, 2008
17
Example for 4. Create interfaces easy-to-use
GOC Grid Operations Center for gLite 3 Modular Approach
Authentication Application
interface Demo/G,
STHAMAS, OCR etc
Grid Engine
Application Interface / DEMO/G
![Page 18: Alice Workshop, Aug 21, 20081 Recent experiences in Grid- enabling legacy software codes Dana Petcu petcu@info.uvt.ro petcu Prof](https://reader036.vdocuments.site/reader036/viewer/2022070412/56649d265503460f949fd6d3/html5/thumbnails/18.jpg)
Alice Workshop, Aug 21, 2008
18
Wrapping codes for Service Grids Non-interactive legacy systems1. Example 1: Service for CFD
simulation
Interactive legacy systems*1. Example 2: Service for image
processing2. Example 3: Services for
symbolic computations3. Example 4: Service for an
expert tool
*systems where any transaction is completed by means of a dialogue between the user and the system
![Page 19: Alice Workshop, Aug 21, 20081 Recent experiences in Grid- enabling legacy software codes Dana Petcu petcu@info.uvt.ro petcu Prof](https://reader036.vdocuments.site/reader036/viewer/2022070412/56649d265503460f949fd6d3/html5/thumbnails/19.jpg)
Alice Workshop, Aug 21, 2008
19
Example 1: service for CFD simulation Legacy code: GERRIS
open source free software library for the solution of the PDEs describing fluid flow
Inputs: a file describing the simulation
parameters; the number of processes used by
Gerris; the name(s) of the resulting file(s)
to be transferred to the client side.
Similar experiences with other codes E.g. for optimization DEME - library
for sequential, parallel & distributed metaheuristic algorithms
INPUT – SIMULATION PARAMETERS:
# The classical lid-driven cavity test case.
1 0 GfsSimulation GfsBox GfsGEdge {} {
Time { end = 300 } Refine 6
SourceDiffusion {} U 1e-3
SourceDiffusion {} V 1e-3
EventStop { istep = 10 } U 1e-4 DU
OutputScalarNorm { istep = 10 } du { v = DU }
OutputPPM { start = end } { v = Velocity }
{ convert -colors 256 ppm:- velocity.eps } }
GfsBox {
top = Boundary { BcDirichlet U 1 BcDirichlet V 0 }
bottom = Boundary { BcDirichlet U 0 BcDirichlet V 0 }
right = Boundary { BcDirichlet U 0 BcDirichlet V 0 }
left = Boundary { BcDirichlet U 0 BcDirichlet V 0 } }
OUTPUT:
![Page 20: Alice Workshop, Aug 21, 20081 Recent experiences in Grid- enabling legacy software codes Dana Petcu petcu@info.uvt.ro petcu Prof](https://reader036.vdocuments.site/reader036/viewer/2022070412/56649d265503460f949fd6d3/html5/thumbnails/20.jpg)
Alice Workshop, Aug 21, 2008
20
Example 1: service for CFD simulationThe service exposes: Simple client interface:
setType(type) Set the execution type: use gerris2D, sequential (type=1) or in parallel (type=2), or gerris3D, sequential (type=3) or in parallel (type=4); by default type=1
exec(simFileName) Launch the simulation described in the simFile
getExample(type) Get an example of simulation file
getHelp() Get a user manual for problem description
getStatus() Get the status (Active, Done or Error) of the simulation; by default the status is Done
![Page 21: Alice Workshop, Aug 21, 20081 Recent experiences in Grid- enabling legacy software codes Dana Petcu petcu@info.uvt.ro petcu Prof](https://reader036.vdocuments.site/reader036/viewer/2022070412/56649d265503460f949fd6d3/html5/thumbnails/21.jpg)
Alice Workshop, Aug 21, 2008
21
Example 2: Service for image processing A Grid service for remote image processing based on GIMP
GIMP accept command lines! despite its classical interactive mode
A distributed DB of images is available to a virtual organization (VO) as a shared facility.
A user apply a sequence of image processing standard procedures to extract a particular information from specific images
Usage in MedioGrid platform
Follows: ESA project GiSHEO! Combining ESA’s G-POD with MedioGrid platform services for educational purposes
The service exposes:
createServInstance <FactoryRef> <EPRFile>
startGIMPServer <EPRFile>
<DatabaseRef>
clientGIMPAction <EPRFile>
<GIMPcommand>
clientGetFile <EPRFile>
<FileName> <Destination>
Simple client interface
![Page 22: Alice Workshop, Aug 21, 20081 Recent experiences in Grid- enabling legacy software codes Dana Petcu petcu@info.uvt.ro petcu Prof](https://reader036.vdocuments.site/reader036/viewer/2022070412/56649d265503460f949fd6d3/html5/thumbnails/22.jpg)
Alice Workshop, Aug 21, 2008
22
Example 3: Services for Symbolic Computations Computer Algebra Systems
(CAS) are wrapped Create an instance for each
client Gradually interpret
commands Examples CAS: GAP, Kant,
MuPad, Maple, Yacas, Singular, Pari, CoCoA, Macaulay etc
Allows interoperability between CASs
Only one part of SymGrid- (EU SCIEnce project) – another part: access Grid/Web/gLite services within CAS
![Page 23: Alice Workshop, Aug 21, 20081 Recent experiences in Grid- enabling legacy software codes Dana Petcu petcu@info.uvt.ro petcu Prof](https://reader036.vdocuments.site/reader036/viewer/2022070412/56649d265503460f949fd6d3/html5/thumbnails/23.jpg)
Alice Workshop, Aug 21, 2008
23
Example 4: an expert tool for ODEs EpODE (1997) @ UVT EpODE (2008) @PEGAF prj.
![Page 24: Alice Workshop, Aug 21, 20081 Recent experiences in Grid- enabling legacy software codes Dana Petcu petcu@info.uvt.ro petcu Prof](https://reader036.vdocuments.site/reader036/viewer/2022070412/56649d265503460f949fd6d3/html5/thumbnails/24.jpg)
Alice Workshop, Aug 21, 2008
24
Conclusions
The process of Grid-enabling legacy codes depends at least the followings: Grid type (e.g. Computational/Service etc) Code type (e.g. Interactive/Non-interactive, GUI oriented/or
not etc) Despite there are several existing tools for Grid-
enabling codes, those are useful for particular conditions, not for the general case
An automatized process for Grid-enabling any legacy code is far for being possible at this moment