outline
DESCRIPTION
End-to-End Workflows for Coupled Climate and Hydrological Modeling Kathy Saint (SGI) and Sylvia Murphy (NOAA/CIRES) iEMSs, Ottawa Canada July 5-8, 2010. Outline. Project Objective Motivation System Description Components Frameworks System Driver Logical Workflow Data Flow Challenges - PowerPoint PPT PresentationTRANSCRIPT
End-to-End Workflows for Coupled Climate and Hydrological Modeling
Kathy Saint (SGI) and Sylvia Murphy (NOAA/CIRES)
iEMSs, Ottawa CanadaJuly 5-8, 2010
Outline
• Project Objective• Motivation• System Description
– Components– Frameworks– System Driver
• Logical Workflow• Data Flow• Challenges• Future Directions
Project Objective
The development of an end-to-end workflow that executes, in a loosely coupled mode, a distributed modeling system comprised of an atmospheric climate model using ESMF and a hydrological model using OpenMI
Motivation
• Hydrological impact studies can be improved when forced with data from climate models [Zeng et al., 2003; Yong et al., 2009]
• A technology gap exists:– Many hydrological models run on personal computers– Most climate models run on high performance supercomputers
• The leveraging of ESMF and OpenMI can mitigate the communication difficulties between these modeling types– ESMF contains web services interfaces that can be used to
communicate across a distributed network– Both ESMF and OpenMI are widely used within their respective
communities
System Description
Data Files
SWAT
OpenMI
CAM OpenMI Wrapper
Driver
High Performance Computer
ESMF Web Services
ESMF CAMComponent
Personal Computer• SWAT (hydrology model) runs on PC• CAM (climate model) runs on HPC• Wrappers for both SWAT and CAM
provide OpenMI interface to each model• Driver (OpenMI Configuration Editor)
uses OpenMI interface to timestep through models via wrappers
• Access to CAM across the network provided by ESMF Web Services
• CAM output data written to NetCDF files and streamed to CAM wrapper via ESMF Web Services
• Resulting output files archived to science gateway
Components: SWAT
• The hydrological model chosen for this project is the Soil Water Assessment Tool (SWAT)
• It is a river basin scale model developed to quantify the impact of land management practices in large, complex watersheds
• It was chosen for this project because it is widely used, is open source, and runs on a Windows platform
Components: CAM
• The atmospheric model chosen for this system is the Community Atmospheric Model (CAM4), part of the Community Climate System Model (CCSM4)
• It was chosen because:• Has ESMF Component
Interfaces• Our group has an ongoing
collaboration with CCSM• It is Open Source
Frameworks: Earth System Modeling Framework
• Is a high-performance, flexible software infrastructure that increases the ease of use, performance portability, interoperability, and reuse of Earth science applications
• Provides an architecture for composing complex, coupled modeling systems and includes array-based, multi-dimensional data structures
• Has utilities for developing individual models including utilities to make models self-describing
• Web services are being developed that allow any networked ESMF component to be available as a web service. (Target merge into ESMF distribution in Sept. 2010.)
Frameworks: OpenMI
• The OpenMI Software Development Kit (SDK) is a software library that provides a standardized interface that focuses on time dependent data transfer
• Primarily designed to work with systems that run simultaneously, but in a single-threaded environment [Gregerson et al., 2007]
• The primary data structure in OpenMI is the ExchangeItem, which comes in the form of an InputExchangeItem and an OutputExchangeItem (single point, single timestep)
The system driver
• Controls the application flow• Implemented using OpenMI’s Configuration Editor• Convenient tool for the testing of the OpenMI
implementations and model interactions• The OpenMI Configuration Editor will eventually be
replaced to handle more complex workflows
Logical Workflow
Driver SWAT/OpenMI CAM/OpenMI Wrapper ESMF Web Services ESMF Component
Initialize
Initialize
Prepare
Prepare
GetValues
Finish
Finish
Dispose
Dispose
NewClient
Initialize
Run
Finalize
GetDataValues
EndClient
GetValues
ESMF_GridCompInitialize
ESMF_GridCompRun
ESMF_GridCompFinalize
Logical Workflow
Driver SWAT/OpenMI CAM/OpenMI Wrapper ESMF Web Services ESMF Component
Initialize
Initialize
Prepare
Prepare
GetValues
Finish
Finish
Dispose
Dispose
NewClient
Initialize
Run
Finalize
GetDataValues
EndClient
GetValues
ESMF_GridCompInitialize
ESMF_GridCompRun
ESMF_GridCompFinalize
Logical Workflow
Driver SWAT/OpenMI CAM/OpenMI Wrapper ESMF Web Services ESMF Component
Initialize
Initialize
Prepare
Prepare
GetValues
Finish
Finish
Dispose
Dispose
NewClient
Initialize
Run
Finalize
GetDataValues
EndClient
GetValues
ESMF_GridCompInitialize
ESMF_GridCompRun
ESMF_GridCompFinalize
Logical Workflow
Driver SWAT/OpenMI CAM/OpenMI Wrapper ESMF Web Services ESMF Component
Initialize
Initialize
Prepare
Prepare
GetValues
Finish
Finish
Dispose
Dispose
NewClient
Initialize
Run
Finalize
GetDataValues
EndClient
GetValues
ESMF_GridCompInitialize
ESMF_GridCompRun
ESMF_GridCompFinalize
Logical Workflow
Driver SWAT/OpenMI CAM/OpenMI Wrapper ESMF Web Services ESMF Component
Initialize
Initialize
Prepare
Prepare
GetValues
Finish
Finish
Dispose
Dispose
NewClient
Initialize
Run
Finalize
GetDataValues
EndClient
GetValues
ESMF_GridCompInitialize
ESMF_GridCompRun
ESMF_GridCompFinalize
Current Data Flow
ESMF Component/CAM
NetCDF File
CAM/OpenMIWrapper
OutputExchange Item
Import
Convert
SWAT/OpenMI
InputExchange Item
GetValues
Personal ComputerHigh PerformanceComputer
GetDataValues
The data exchange is initiated by the OpenMI GetValues call, and the data, retrieved from a NetCDF file as multi-dimensional array, is indexed and streamed to the CAM Wrapper as a single point, single timestep value
Target Data Flow
ESMF Component/CAM
ESMF StateCAM/OpenMI
Wrapper
OutputExchange Item
Import
Convert
SWAT/OpenMI
InputExchange Item
GetValues
Personal ComputerHigh PerformanceComputer
GetDataValues
The goal is to bypass the NetCDF file, and instead, retrieve the data directly from an ESMF State object
Model Configurations
• SWAT– Hydrology science information
provided by Jon Goodall of University of S. Carolina
– Lake Fork Watershed (TX) – Watershed Area: 486.830 km2– Model run: 2 years, 1977 – 1978– Timestep = 1 day– Weather Stations:
• wea62 (33.03 N, 95.92 W)• wea43 (33.25 N, 95.78 W)
• CAM– Global Atmospheric Model– Model run: 1 day– Timestep: 1800 sec – Dynamical Core: finite volume– Horizontal Grid: 10x15– Export data variables:
• surface air temperature• precipitation• wind speed• relative humidity• solar radiation
Challenges
• Bridging the Gaps – dealing with the complexities of interoperating between 2 diverse models– Single threading vs. Multi-threading (MPI)– Desktop platforms vs. High Performance Computing (HPC)
platforms– Windows vs. Linux/Unix– Multiple programming languages throughout the system
• CAM and SWAT Models in Fortran• OpenMI in C#• ESMF in Fortran and C++• Web Services in Java, C++, and Fortran• NetCDF interface in J#
Remaining Tasks
• Replace the OpenMI Configuration Editor for more complex workflows
• Modify CAM to stream output instead of writing to a file• Automatically archive data from both CAM and SWAT to a
science gateway• Time stepping through CAM instead of pre-running during
initialization• 2-way coupling (long-term)
Questions
Earth System Curator: http://earthsystemcurator.org ESMF: http://earthsystemmodeling.org OpenMI: http://www.openmi.org CAM: http://www.ccsm.ucar.edu/models/ccsm4.0/cam SWAT: http://swatmodel.tamu.edu
[email protected]@noaa.gov