wgiss22 annapolis,11 sept.2006, cnes/j.p.antikidis wide area grid concepts (wgiss-22, annapolis,...
TRANSCRIPT
WGISS22 Annapolis,11 sept.2006, CNES/J.P.Antikidis
Wide Area Grid Concepts(WGISS-22, Annapolis, sept.2006 )[email protected] National d'Etudes Spatiales,Programme & Strategy directorate,"Space Information Systesms"
Introduction Implementation concepts Way ahead
Introduction Implementation concepts Way ahead
WGISS22 Annapolis,11 sept.2006, CNES/J.P.Antikidis
50 major international systems ! Quote: WGISS user chair report,
Kiev, Sept 2005 !!
how to deal with the complexity wall ?How to deal wih the Babel syndrome ?
A pressing need: data & service large scale integration
WGISS22 Annapolis,11 sept.2006, CNES/J.P.Antikidis
WGISS also addresses the needs of data and information users by aiding the development of simpler and wider access to the resources and systems they require. WGISS places great emphasis on the use of demonstration projects involving user groups to solve the critical interoperability issues associated with the achievement of global services. WGISS has developed and will capitalize on a number of tools, standards, and services to assist access to, preservation of, and efficient use of Earth observation data resources.
John Faudeen, WGISS chairman, 2005
WGISS directions
WGISS22 Annapolis,11 sept.2006, CNES/J.P.Antikidis
WAG+Virtualization is the key ?
WGISS22 Annapolis,11 sept.2006, CNES/J.P.Antikidis
GRID
Classical GRID based
To day EO large scale objectivesWorld-wide follow-on ozoneWorld-wide follow-on ozone
World-wide follow-on of sea state (MWorld-wide follow-on of sea state (MEERCATOR)RCATOR)
World-wide follow-on of vegetation indexWorld-wide follow-on of vegetation index
World-wide follow-on of crop and food resourcesWorld-wide follow-on of crop and food resources
GMES programmeGMES programme
GEO endeavourGEO endeavour
........
Can we afford on the long run to pileCan we afford on the long run to pile--up investments as we do todayup investments as we do today
WAG-ObjectivesWAG stand for Wide Area Grid is in fact nothing new with respect to WAG stand for Wide Area Grid is in fact nothing new with respect to GRID technology apart from the fact it deals with a particular type of GRID technology apart from the fact it deals with a particular type of GRID system implementation.GRID system implementation.
In fact GRID concept as derived from Ian foster ideas is basically a set of In fact GRID concept as derived from Ian foster ideas is basically a set of hardware machines being connected together with basically no pre-hardware machines being connected together with basically no pre-specified implementation topology.specified implementation topology.
The major difference in between a “distributed system” and a GRID lies in The major difference in between a “distributed system” and a GRID lies in the fact the hardware organisation and the job handling is not anymore the fact the hardware organisation and the job handling is not anymore carried out by the application software itself but by a suitable MiddleWare carried out by the application software itself but by a suitable MiddleWare supposed in theory to hide the actual hardware arrangement specificity.supposed in theory to hide the actual hardware arrangement specificity.
=>> WAG experiments should enable testing a Widely shared =>> WAG experiments should enable testing a Widely shared GRID implementation carrying several EO Applications in a GRID implementation carrying several EO Applications in a concurent way concurent way
May prefigure GMES or GEO up-coming May prefigure GMES or GEO up-coming genericgeneric technical technical architecturesarchitectures
WAG solutions
Intra-GRID (WAG1)Intra-GRID (WAG1)
Sharing of individual and widely dispersed Sharing of individual and widely dispersed installations “seen” as a unique and general installations “seen” as a unique and general purpose GRIDpurpose GRID
Inter-GRID(WAG2)Inter-GRID(WAG2)
Connection of widely dispersed GRIDS either:Connection of widely dispersed GRIDS either:
- for the sake of a given application- for the sake of a given application
- for the sake of creating a unified entity (GRID - for the sake of creating a unified entity (GRID or Supr-GRIDor Supr-GRID
The MiddleWare question
WAG1: Just use Ian Foster vision by making the WAG1: Just use Ian Foster vision by making the distributed resources invisible while operated by distributed resources invisible while operated by a common MiddleWare as a unique Grid a common MiddleWare as a unique Grid resource resource
------------------------------------------------------------------------------------------------------------------------
WAG2: either enabling ad’oc solutions for Grids WAG2: either enabling ad’oc solutions for Grids connections or creating a “Super” MiddleWare connections or creating a “Super” MiddleWare able to catch and operate heterogeneous and able to catch and operate heterogeneous and widely distributed GRID systemswidely distributed GRID systems
WAG1: Intra Grid
WAG (in its original description from WAG (in its original description from WGISS20) works by melting together WGISS20) works by melting together existing hardware resources from as existing hardware resources from as many place as needed in a UNIQUE GRID many place as needed in a UNIQUE GRID (because by essence a local GRID is (because by essence a local GRID is nothing different than a highly nothing different than a highly dispersed GRID).dispersed GRID).
Pros:Pros: Zero Middleware development and Zero Middleware development and full compatibility right from scratch, full compatibility right from scratch, straightforward implementation straightforward implementation
Cons:Cons: Need to agree on the rule of the Need to agree on the rule of the game: patch all harware resources in a game: patch all harware resources in a network and operate it in a common network and operate it in a common middleware.middleware.
WAG2: Inter-Grids
Inter-Grid : means ways to make different and Inter-Grid : means ways to make different and isolated GRID systems becoming unified either isolated GRID systems becoming unified either by by
- creating some added middleware "gluing" all - creating some added middleware "gluing" all GRIDS togetherGRIDS together
- generating exchanges rules for all connected - generating exchanges rules for all connected GRID to speak together (kind of equivalent to GRID to speak together (kind of equivalent to Web service composition but with GRIds)Web service composition but with GRIds)
Pros: allow heterogeneous GRIDS to speak Pros: allow heterogeneous GRIDS to speak togethertogether
Cons: Not simple by essence since close to be Cons: Not simple by essence since close to be as complicated as making speaking machines as complicated as making speaking machines together (may mean a middleware for making together (may mean a middleware for making heterogeneous middleware compatible ?)heterogeneous middleware compatible ?)
WGISS & WAGRole of WGISS forwarding recommendation on Role of WGISS forwarding recommendation on “upcoming solutions” able to simplify Remote sensing “upcoming solutions” able to simplify Remote sensing data utilization and societal regognition.data utilization and societal regognition.
A system is not the “pilling up” of needed products or A system is not the “pilling up” of needed products or services but better a suitable balance in between generic services but better a suitable balance in between generic technological solutions and the thematic.technological solutions and the thematic.
Soon or later we will have to switch to “virtual Soon or later we will have to switch to “virtual cooperative solutions” and better capitalize on the cooperative solutions” and better capitalize on the enormous simplification brought by GRID conceptsenormous simplification brought by GRID concepts
CEOS-WGISS have ever shown the way to go with:CEOS-WGISS have ever shown the way to go with:
CEOS format, CEOS AVHRR 1km maps, CEOS CIP and CEOS format, CEOS AVHRR 1km maps, CEOS CIP and time has come for WGISS to introduce CEOS-WAG as time has come for WGISS to introduce CEOS-WAG as the next needed evolutionthe next needed evolution
What GEOSS or GMES need ?
Thematic solutions identified on a case by case Thematic solutions identified on a case by case basis (pollution, risk, ozone etc..)=> free mandate basis (pollution, risk, ozone etc..)=> free mandate given to agencies to deliver a service (present state given to agencies to deliver a service (present state of the art, very expensive since lot of technical of the art, very expensive since lot of technical duplication, low portability)duplication, low portability)
““Making thematic beeing virtualized” and leave Making thematic beeing virtualized” and leave technical implementation being WAG based (Full technical implementation being WAG based (Full use a Grid properties, MiddleWare unification, full use a Grid properties, MiddleWare unification, full portability, easy updates and evolutions, LOW portability, easy updates and evolutions, LOW COST and full distribution of development effort)COST and full distribution of development effort)
WGISS22 Annapolis,11 sept.2006, CNES/J.P.Antikidis
Vision: Technological evolutions
Blueprint plans (1850) CATIA (1985) (virtual mockup)
From Real to « Virtual »
Formal programming (1948) WINDOWS (1990)
Electronic circuitry (1914) SPICE simulators (1985)
MINITEL Physical links(1970) INTERNET (1990)
Processing systems (1960) eInfrastructures
WGISS22 Annapolis,11 sept.2006, CNES/J.P.Antikidis
The "Silos" infrastructure:Complex, little flexible, and Expensive
Not Scaleable
Single Points of Failure
Underutilized Hardware
Dedicated Staff
Applications Tied to Specific Hardware
• IT Staff Focused on Maintenance, Not Competitive Advantage
• Overall Resource Utilization: 10%-30%; No Resource Sharing
B o ttle n e c k e d A p p lic a tio n s
U n ix S M P
A p p lica tio n B
L in u x C lu s te r
A p p lica tio n C
M a in fra m e
A p p lica tio n D
W in d o w s S e rv e r
A p p lica tio n A
WGISS22 Annapolis,11 sept.2006, CNES/J.P.Antikidis
Scenario: Evolution of networks
Communication Networks
Computer Networks
eInfrastructures (networks +
services)
Tomorrow:Virtual GridsVirtual NetworksOutsourcing of Processes
Today:Physical ComputersPhysical Networksand Security issues!
Who will take the advantage in this new avenue of Business?
GRID
?
WGISS22 Annapolis,11 sept.2006, CNES/J.P.Antikidis
The Shift to Service-Oriented Computing
Source: Credit Suisse First Boston and DataSynapse
SERVICE-ORIENTED ARCHITECTURE
Services Policies Virtual Resources
Users Physical Resources
SERVICE-ORIENTED INFRASTRUCTURE
Applications
DemandDemand SupplySupply
Provisioning
Charge-Backs
Provisioning
Metering
VirtualResource
Market
VirtualResource
Market
Utility Grid
Compute& Service Grids
DataGridS
tora
ge
Pro
ce
ss
ing
Ne
two
rks
Functionality
SLAs
Accountability
WGISS22 Annapolis,11 sept.2006, CNES/J.P.Antikidis
The WAG Approach: A New Consumption
and Fulfillment Model for IT • Creates a Creates a virtual application operating virtual application operating
environment environment
• Virtualizes application services Virtualizes application services executionexecution
• Dynamically fulfills requests Dynamically fulfills requests over a virtual pool of system resourcesover a virtual pool of system resources
• Offers an adaptive, self-managed Offers an adaptive, self-managed operating environment that guarantees operating environment that guarantees high availabilityhigh availability
• The business gets what they need, The business gets what they need, when they need it, when they need it, on demandon demand
WAG
App BApp A App C
Application Layer
Systems Layer
WGISS22 Annapolis,11 sept.2006, CNES/J.P.Antikidis
Induced conceptual changes
No more knowledge of information or/and data-bank physical location (an archive is just known to be accessible trough its naming)No more need to describe the physical connections in between different locations (since fully supported by the underlying GRID MiddleWare) making in theory any world-wide distributed systems easy to implement.No more physical handling of information sets. This could trigger completly new merchandising concepts (for ex. come close to Telecomm recharge)No need to know the mapping of user ’software enabling virtual service to be easily created and offered to any-one on an autonomous fashion (virtual stores/services)
WGISS22 Annapolis,11 sept.2006, CNES/J.P.Antikidis
Integrated Generic Service layer will
happen because:
Higher needs and systems costs are not compatibleNeed to fight against the complexity barrier (rely on Middleware/UpperWare system management)New business are appearing making dedicated structure not competitive anymoreAre we sure we need on the long term specific resources for the only sake of Earth Observation ? WAG or/and WEB2 for systems development +Google technologies evolution for user access may become common place and incredibly cheap
WE NEED WGISS HELP FOR IMPLEMENTATION