grid perspectives in earth system modelling · using the hpc-nwp model lm of dwd manchester csar...

GRID Perspectives in

Earth System Modelling

Reinhard G. Budich

Max-Planck-Institute for Meteorology, Hamburg, [email protected] || www.mpimet.mpg.de

GRID Perspectives in Earth System Modelling

IntroductionEarth System SciencesEarth System Modelling

GRID related issues

StatusProjects

Europe: Unicore application by DWDClimateprediction.*

US: Earth System GRID

European InitiativesGMES: GRID type solutions neededC3 Grid

Future Perspectives

Earth System Sciences

Earth System SciencesProblem:Time scales range from a few seconds (chemical compounds) to multi century (ocean interiror)Space scales from local (O km’s; where you live) to global (O 1000’s km’s, where we live)State of the complete system is never known

Earth System SciencesObservation

Theory

Experiment


Long glacial periods;

short interglacials

Periodicity at ca.100 000 years

Atmos compositionand climate areclosely coupled

Set points at ca.200 & 290 ppm

Observations

Observational Data BaseNumber of SensorsSpace-born: IncreasingEarth-born, Ocean: Rather decreasing

Data volume per SensorDramatically increasing

Demand to link SensorsDramatically increasing

Reanalysis data setsVery high volume data sets (10s of TB)

Earth System SciencesTheory

Earth System SciencesHigh degree of complexity in all processes

High number of processes

High number of interfaces

Earth System SciencesExperiments


No Experiments!!

Earth System ModellingVery old idea

=> Numerical Weather Prediction

Earth System ModellingModel subsystems

Atmosphere

Atmospheric Chemistry

Ocean Land Surfaces

Sea Ice Ocean Biogeochemistry Regional Climate


AtmosphereAtmospheric Chemistry

Ocean

Land SurfaceSea Ice

Ocean Biogeochemistry

Regional Climate

Coupler

Earth System ModellingFirst “Complete” ESMs are now configuredGeneral Circulation Models as test casesModels of reduced complexity state of the art

Reduce uncertainty: Use Ensembles,Multi Model -,Multi Site - Configurations

Data Storage @ DKRZ

Earth System ModellingFirst “Complete” ESMs are now configuredGeneral Circulation Models as test casesModels of reduced complexity state of the art

Reduce uncertainty: Use Ensembles,Multi Model -,Multi Site - Configurations

Data volumes increase above all measures: many Petabyte archives by mid decade

Earth System SciencesGRID related issues:Observations and there derivatives, i.e. data processing: -> Go GRID <-Theory/Modelling:

Coupled models are -> not apt for GRID <-:Very high demand on low latencies, bandwidthDifficult to realize even within a single machineVery communication intense

Complete experiments and there evaluation:-> GRID based solution environments seem to be promising

GRID Perspectives in ESM: ProjectsMeteo-GridClimatePrediction.*Earth System GridGMESC3Grid

Idea: Your individual weather forecast using the HPC-NWP model LM of DWDManchester CSAR SGILugano CSCS NECOffenbach DWD IBM and CrayParis IDRIS IBM

Unicore based

Meteo-GRID

Idea: Your individual weather forecast using the HPC-NWP model LM of DWDManchester CSAR SGILugano CSCS NECOffenbach DWD IBM and CrayParis IDRIS IBM

Unicore basedProblems: Who pays? How?

Meteo-GRID

1.) ClimatePrediction.net2.) ClimatePrediction.eu

1.) Running, 2.) Planned1.) A low resolution atmosphere model + slab ocean on ten thousands of PCs2.) 4 AOGCMS in different resolutions on ten thousands of PCs, + Cluster computingBoth:Comparable to seti@homeVary parameters to decrease uncertaintyGRID approaches to distribution, data analysis and visualisation

Earth System Grid The Earth System Grid (ESG) integrates supercomputers with large-scale data and analysis servers located at numerous national labs and research centers to create a powerful environment for next generation climate research.

ESG Collaborators

Argonne National LaboratoryLawrence Berkeley National LaboratoryLawrence Livermore National LaboratoryNational Center for Atmospheric ResearchOak Ridge National LaboratoryUniversity of Southern California/Information Sciences Institute

Funded by the U.S. Department of Energy (DOE)

Earth System Grid

EU-Project GMESGlobal Monitoring for Environment and Security

Partners: EC, EEA, ESA, Eumetsat and othersIntegration of

Environmental DataEarth System Models, incl. human dimensionAssimilation

... to provide relevant information to policy makers and the publicProblems:

Distribution of information and dataIntegration of data into even higher-volume data setsShort resources in emergency cases, fast resource sharing Short term establishment of virtual organizations

C3 GridCollaborative Climate Community Data and Processing GridAccess to distributed data archivesEfficient distributed data processingInter-institutional data exchangeMetadirectoriesEffective accessIntegrated analysis toolsReplicatesSchedulingResource Brokers

Daten-Transfer Dienst

Archiv-Daten

Metadaten

Job

Daten

VerteilteDaten-Archive


Nutzer

Archiv-Anbindung

Pre-Processing

Nutzer-Module

VerteilteProcessing Ressourcen


BestehendeRessourcender Institute

LokaleRessourcen-

Schnittstellenbei den Instituten

VerteilteGrid-

Infrastruktur

Nutzer-Schnittstelle

Informations-Dienst

Daten-Management -Dienst Job-Management -Dienst

API GUI

Daten-Transfer Dienst

Archiv-Daten

Metadaten

Job

Daten



Nutzer

Archiv-Anbindung

Pre-Processing

Nutzer-Module



BestehendeRessourcender Institute

LokaleRessourcen-

Schnittstellenbei den Instituten

VerteilteGrid-

Infrastruktur

Nutzer-Schnittstelle

Informations-Dienst

Daten-Management -Dienst Job-Management -Dienst

API GUI

C3 Grid

C3 GridCollaborative Climate Community Data and Processing GridAccess to distributed data archivesEfficient distributed data processingInter-institutional data exchangeMetadirectoriesEffective accessIntegrated analysis toolsReplicatesSchedulingResource Brokers

Teil der deutschen D-Grid-Initiative

Future PerspectivesBringing it all together

AtmosphereModels

OceanModels

Land SurfaceModels

TerrestrialBiosphere

Models

Solid Earth

Models

Car

bon

Cyc

le a

ndA

tmos

pher

ic C

hem

istr

y

Wat

er C

ycle

The Earth SystemUnifying the Models

The PredictiveEarth System

Megaflops Gigaflops Teraflops Petaflops

Natural HazardPrediction

HydrologyProcessModels

Climate / WeatherModels

Future Perspectives

Future PerspectivesFor ESM:Issues:Data QoS on the network“Sociology”

Non-Issues:“Meta-computing”

Ensemble runs (trivially parallel) might become an issue

Education is key

Future PerspectivesGermany is lagging behindSee US and UK initiatives resulting in a considerable competency gap for Germany

D-GRID initiative deserves full support, especially by the industryStandardization and “Sociology” are key

!!! Danke !!!??? Fragen ???

[email protected]

grid perspectives in earth system modelling · using the hpc-nwp model lm of dwd manchester csar...

Documents