ESMF,WRF and ESMF,WRF and ROMSROMS

PurposesPurposes Not a tutorial Not a tutorial Educational and conceptualEducational and conceptual Relation to our workRelation to our work

What do they represent What do they represent for?for?

ROMS: Regional Oceanographic ModelingROMS: Regional Oceanographic ModelingSystemSystem

WRF: Weather Research Forecasting WRF: Weather Research Forecasting

ESMF: Earth System Modeling ESMF: Earth System Modeling FrameworkFramework

ROMSROMS Classical software structure for Classical software structure for

numerical model numerical model No clear hierarchical levelNo clear hierarchical level Focus on physics Focus on physics

Curvilinear coordinateCurvilinear coordinate Nested and composed gridNested and composed grid

ROMS ROMS Some level of modularitySome level of modularity

Fortran 90/95Fortran 90/95 C – preprocessing macro managingC – preprocessing macro managing

Supports MPI and OpenMPSupports MPI and OpenMP Portable Portable Widely used – 30 countriesWidely used – 30 countries

WRF GeneralWRF General

WRFWRF A framework for regional weather A framework for regional weather

research and forecastingresearch and forecasting Emphasize on Emphasize on

physicsphysics software architecture software architecture standardstandard

WRF software WRF software ArchitectureArchitecture

Three hierarchical levelThree hierarchical level ProvideProvide dynamic and physics dynamic and physics

corescores Applications have options to add Applications have options to add

their own components including their own components including corescores

WRF software design schematic

Driver

ConfigInquiry I/O API

ConfigModule

WRF Tile-callableSubroutines

SolveMediation Layer

Model Layer

Driver Layer

DM comm

Thre

ads

OMP

External Packages

PackageIndependent

PackageDependent

Data formats,Parallel I/O

Mes

sage

Pass

ing

(Adapted from WRF software design document athttp://www.mmm.ucar.edu/wrf/users/WRF_arch_03.doc)

WRF FeaturesWRF Features Supports MPI and OpenMPSupports MPI and OpenMP PortablePortable Fortran 90Fortran 90 Widely Used by international Widely Used by international

research communityresearch community

Two level decomposition

Logical domain 12 patches

Level 1

Distributed memoryMPI etc.

Level 2Sha red m

emory 4 tiles

We are interested in

Now ESMF Now ESMF

Why ESMF?Why ESMF? Standard interface to couple Earth Standard interface to couple Earth

system modelssystem models InteroperabilityInteroperability

code validation, tuningcode validation, tuning migration from research to operationsmigration from research to operations

Productivity Productivity model components easily to buildmodel components easily to build maintenancemaintenance

ESMFESMF Superstructure: couple component Superstructure: couple component

(required)(required)How models are coupled!How models are coupled!

Substructure: tools, data component,Substructure: tools, data component,Data regridding etc.Data regridding etc.

No physics/dynamics inside, pure No physics/dynamics inside, pure frameworkframework

ESMF StructureESMF Structure Sandwiched architectureSandwiched architecture

ESMF Superstructure

User Code

ESMF Infrastructure

couple

commonutility

Now Examples of ESMF and Now Examples of ESMF and WRFWRF

Land Information System

LIS-GCE coupling (standard LIS-GCE coupling (standard way)way)

ESMF Coupler

LIS GCE

ESMF Utility

LIS-WRF couplingLIS-WRF couplingWRF-surface driver

LIS

ESMF couplerESMF becomes a

subcomponent

Impact of ESMF on Impact of ESMF on Coupled PerformanceCoupled Performance

0

2

4

6

8

10

12

14

16

18

0 20 40 60 80

Num ber of Processors

Tota

l ela

psed

tim

e (h

rs)

Default WRF

LIS-WRF

Weather Research and Forecasting Model (WRF)

Now about IOsNow about IOs

ROMS IO ROMS IO NetCDF onlyNetCDF only Stick to NetCDF toolsStick to NetCDF tools No motivation to use other data No motivation to use other data

formatformat Need parallel IONeed parallel IO

WRF IOWRF IO• Standard IO APIs• NetCDF NetCDF • GRIBGRIB• HDF5HDF5• Currently supports parallel IO Currently supports parallel IO

through HDF5through HDF5

WRF I/O Schematic

Application

I/O API

FormatPackage

CommPackage

Data Medium

Package-independent

Package-specific

Installation-specific

Wall Clock Time Used with Different Output File Size Case 1: Conus

IBM WinterHawkII (256 Processors)

0

10

20

30

40

50

60

70

80

0 5 10 15 20 25

Output File Size(GB)

Wal

l Clo

ck T

ime(

Min

ute)

Parallel HDF5NetCDF

ESMF IOESMF IO Only support NetCDF IO through Only support NetCDF IO through

WRF APIsWRF APIs Very preliminary Very preliminary No goal to improve IO in the near No goal to improve IO in the near

futurefuture

How’s that related to us?How’s that related to us? All these model developers express All these model developers express

interests about NetCDF4 project, interests about NetCDF4 project, especially parallel IOespecially parallel IO

NetCDF4 is the key product for NetCDF4 is the key product for HDF5 to be utilized HDF5 to be utilized

Simple Parallel IO?Simple Parallel IO? Fortran 90 supports?Fortran 90 supports?

ESMF web page:ESMF web page: http://www.esmf.ucar.edu/http://www.esmf.ucar.edu/

WRF web page:WRF web page: http://www.wrf-model.orghttp://www.wrf-model.org//

Applications about ESMF and WRF:Applications about ESMF and WRF: http://esto.nasa.gov/conferences/estc2004/presenhttp://esto.nasa.gov/conferences/estc2004/presen

tation/A4/a4p2.pdftation/A4/a4p2.pdf http://esto.nasa.gov/conferences/estc2005/papershttp://esto.nasa.gov/conferences/estc2005/papers

/a7p3.pdf/a7p3.pdf

ROMS web page: ROMS web page: http://marine.rutgers.edu/po/http://marine.rutgers.edu/po/