arctic climate and weather forecast model visualization scenario

Content developed by the GEO Architecture Implementation Pilot Licensed under a Creative Commons Attribution 3.0 License

Arctic Climate and Weather Forecast Model Visualization Scenario

Engineering Report GEOSS Architecture Implementation Pilot

Version 1.0

GEO Architecture Implementation Pilot, Phase 3 Version: 1.0 Arctic Climate and Weather Forecast Model Visualization Scenario Engineering Report

Date: 28Dec10

Revision History

Version Date Editor and Content providers

Comments

0.1 12/07/2010 Yuqi Bai Section 1.1, 1.2, 3.1, 3.2, 3.3, 4

0.2 12/08/2010 Xuanang Cheng Section 5.1 2)3), 5.2

0.3 12/09/2010 Doug Nebert Section 5.3, 5.4, 6 , Revised Draft

0.4 12/13/2010 Yuqi Bai Figure 3a, Revised Draft

0.5 12/21/2010 Ben Domenico Edit Section 5.1

1.0 12/28/2010 Doug Nebert Final edits and accept comments/changes

Document Contact Information

If you have questions or comments regarding this document, you can contact: Name Organization Contact Information

Douglas D. Nebert USGS/ FGDC [email protected]

Liping Di CSISS, GMU [email protected]

Xuanang (Roman) Cheng CSISS, GMU [email protected]

Genong (Eugene) Yu CSISS, GMU [email protected]

Yuqi Bai CSISS, GMU [email protected]

Ben Domenico UCAR UNIDATA [email protected]

Tyler Erickson MTU MTRI [email protected]


Date: 28Dec10

Table of Contents

Contents

1. Introduction 4 1.1 Scope of this document 4 1.2 GEOSS AIP 4

2. Community SBA Objectives 4

3. Scenario: Climate Change and ArcSDI 4 3.1 Actors 4 3.2 Context and pre-conditions 5 3.3 Scenario Events 5 3.4 Post-Conditions 6 3.5 Special Requirements 7

4. System Model of the Scenario 7

5. Implementation 7 5.1 Deployed Components 7 5.2 Use of the GCI 11 5.3 Demonstrations 11 5.4 Future plans for deployment 12

6. Discussions and Lessons Learned 12

7. References 12


Date: 28Dec10

Arctic Climate and Weather Forecast Model Visualization Scenario

1. Introduction 1.1 Scope of this document This AIP-3 ER will describe the user scenario entitled "Arctic Climate and Weather Forecast Model Visualization Scenario." This scenario demonstrated publishing, discovery, and visualization of weather and climate data through GEOSS, highlighted value of geospatial standards common to Spatial Data Infrastructures – as layers of interest to a conceptual Arctic SDI, and supported end-user desktop visualization of complex climate and weather models for decision support in the Arctic region. The architecture, workflow of the developed system is presented; the standards-based GEOSS components and services are highlighted, along with our lessons learned. 1.2 GEOSS AIP The GEOSS Architecture Implementation Pilot (AIP) task develops process and infrastructure components for the GCI and the broader GEOSS architecture as a means of coordinating cross-disciplinary interoperability deployment. The AIP Task provides phased delivery of components to GEOSS operations, with each phase consisting of: architecture refinement based on user interactions; component deployment and interoperability testing; and SBA-focused demonstrations.

This Engineering Report (ER) is a key result of the third phase of AIP. AIP-3 was conducted from January 2010 to December 2010. A separate ER describes the overall process and results of AIP-3 and thereby provides a context for this Community SBA ER.1

2. Community SBA Objectives The objectives of this scenario are 1) to demonstrate the availability of climate and weather forecast data through GEOSS, 2) to visualize climate and weather phenomena in formats and terms more meaningful to non-expert users, and 3) to provide a viewing scenario amenable to use in the Arctic or polar environment (suitable projection via “globe” view of KML) along with other geographic data for context. The primary audience would be planners and logistics individuals interested in arctic climate and weather data.

3. Scenario: Climate Change and ArcSDI 3.1 Actors

The main involved actors in this Scenario are: • Analyst: The end user of the GEOSS system, who consults the GEO Web Portal about the availability

of mapped weather and climate forecast data for his area of interest from the present through 2050. • Real-Time and Forecast Weather Data Service Provider:

o Unidata Motherlode • Climate Change Model Data Map Provider:

o Intergovernmental Panel on Climate Change (IPCC) • Weather Station Climatology Data Provider:

o CSISS/GMU from NOAA climatology CD

1 A listing of all AIP-3 Engineering Reports: http://www.ogcnetwork.net/AIP3ERs


Date: 28Dec10

3.2 Context and pre-conditions These resources were registered in the GEOSS Component and Service Registry, and are discoverable through GEO Portal.

1. Current Weather Forecast WMS and WCS interface (UCAR)

2. KML to wrap IPCC-data.gov/maps long-term climate model WMS services (GMU)

3. Station Climatology KML from NOAA NCDC data (GMU)

3.3 Scenario Events Table 1 – Steps in the Community Scenario

Step Description Trans. Tech Use Case

0 Analyst in the arctic region consults the GEO Web Portal about the availability of mapped weather and climate forecast data for his area of interest from the present through 2050.

General search with a geographic footprint over near-shore waters in northern Europe, and content keywords of 'climate weather forecast.' Their interest is in identifying surface temperature changes that may affect permafrost conditions, with impact on surface infrastructure maintenance.

4. Search for Resources

1 In the right pane, a list of search results includes the "Unidata Motherlode Real-Time Weather Data Services."

5. Present Services and Alerts


Date: 28Dec10

2 The analyst clicks on the brief metadata, and requests and views a "Full Description" metadata record.

7. Exploit Data Visually and Analytically

3 The analyst returns to the prior URL "Summary Description" and sees a KML file link present.


4 The analyst clicks on the KML link and is prompted to save the file or launch an installed program, such as Google Earth - selects launch in GE.

6. Interact with Services

5 Analyst zooms to northern Europe in Google Earth, illustrating that a circumpolar view is possible.


6 The analyst browses the available model parameters and time periods in GE, opening the appropriate 'folders' in the legend control to select surface temperature for all time periods, with an interest in the month of July.


7 The analyst views an animated sequence of temperature with decadal frames, stops the animation on a period of interest (2050).


8 The analyst is now interested in knowing what weather station climatologies exist in northern Europe. Using geoportal.org, they search for "monthly climate"


9 Select the registered Component for the monthly climate KML file and also launch in GE.


10 Click on stations to reveal basic station information, click on embedded hyperlink to display the climate graph for the station to answer the question of what current "normal" temperatures should be for July.


11 Returning to a more immediate facilities management problem, the analyst is interested in knowing current weather forecasts for the same area of interest. Using geoportal.org, they search for "weather forecast KML" and locate the Motherlode GFS model reference.


12 The analyst launches the discovered data link in the portal as KML and loads to Google Earth.


13 Analyst selects Temperature (Surface) and Precipitation (Total) to see what the forecast conditions are, including the ability to identify and then animate a parameter over a selected range of time.


14 Analyst can save these changing feed references for quick and updated reload in the future.


3.4 Post-Conditions For each of the three sub-scenarios, the end-state is that the user is able to interact with a visualization of the data, largely achieved through applying an OGC KML wrapper around either a point location-based data series (station climatology database) or around model-driven OGC Web Map Services. This broadens the scope of accessibility beyond a limited number of specialized WMS clients and to the broader reach of KML viewers and globes. This means to visualize and discover data in a globe viewer is especially valuable in the polar regions where there is


Date: 28Dec10

systematic distortion of the landscape in more traditional cylindrical projections that conform at lower latitudes.

3.5 Special Requirements

4. System Model of the Scenario

Figure 1 System Model of the Scenario

5. Implementation 5.1 Deployed Components

1) Current Weather Forecast WMS and WCS interface (UCAR)

The University Corporation for Atmospheric Research (UCAR) Unidata program has implemented a THREDDS Data Server (TDS) called motherlode that provides near real-time access to global weather forecast model output from the National Centers for Environmental Prediction (NCEP) using OGC Web Map Service and Web Coverage Service protocols. These models provide forecast values for multiple atmospheric parameters at regular time steps at different levels in the atmosphere – a complex multi-dimensional solution that presents challenges for rendering and visualization. The NCEP GFS (Global Forecast System) model is run every six hours. A python program was written to develop an OGC KML wrapper to name and organize selected global weather forecast outputs into the ‘folder’ metaphor by parameter and time. This organizational approach enables KML animation of the time series data through repeated WMS calls for a global image for each time stop and parameter combination, on-the-fly. This solution allows casual users to view symbolized atmospheric phenomena globally (including coverage in the arctic region) in time series using KML clients and enabling display with additional data layers for context. Additionally, the model outputs are, of course, available as OGC WMS layers and WCS coverages for interaction by other clients. The model parameters (layers) were harvested into a CSW catalogue that


Date: 28Dec10

was registered in the GEOSS CSR for promotion through Clearinghouse. The multi-dimensional and multi-temporal issues present a continuing challenge in describing and discovering dynamic weather data. Two specific challenges were revealed by this exercise. First, the real-time nature of data on the motherlode TDS illustrates the need for a means of specifying “time relative to the present” rather than specific absolute times. Such a facility is needed both in the WMS/WCS protocols as well as in KML. The lack of such a capability has made it necessary to generate a set of KML scripts on a regular schedule rather than on an as-requested schedule. The second issue is the sheer volume and volatile nature of the data served by motherlode. Harvesting metadata for service in catalogs can generate huge XML files that are, in fact, out of date by the time the harvesting is complete. This points to a need to be able to harvest discovery metadata at a “collection” level rather than at the individual dataset level. But that means the client has to be able to “drill down” to be able to get to individual datasets. This is often referred to as the “granularity issue” and is solved in the case of the TDS by a hierarchical set of “catalogs of catalogs” that is, in effect, a remote file system.

2) KML to wrap the Intergovernmental Panel on Climate Change (IPCC) Third Assessment Report long-term climate model WMS services (GMU)

Long-term climate model data published with the IPCC Third Assessment Report were available through Web Map Coverage (WMS) service interface operated by the IPCC [2]. These WMS services were wrapped in KML files by the GMU team for visualization in Google Earth or other KML viewer that would create less distortion in Polar Regions.

IPCC Third Assessment Report provides long-term (1970-2099) predicted climate data parameters based on six different climate models. These climate data were available for multiple change scenarios and include up to 20 parameters per model, e.g. 200 hPa Mean Temperature, 850 hPa u-Wind, Mean Temperature. They are accessible via WMS requests by specifying key values including model, scenario, time period, and parameter. One sample WMS request is listed as follows, it requests for January 200 hPa Mean Temperature data in 1970-1999 time period based on CCSR/NIES model, SRES Scenario A2:

http://ipcc-data.org/maps/wms/tar?FORMAT=text%2Fhtml&VERSION=1.3.0&CRS=CRS%3A84&SERVICE=WMS&REQUEST=GetLegend&STYLES=&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&LAYERS=nies99%3Aa2%2Ftmp200&TIME=1984-1-15T0%3A0%3A0.0Z&SRS=EPSG%3A4326

Since the output of each WMS request is simply an image file for a specified geography (in this case, global), it could be wrapped in KML file for visualization in Google Earth or other KML viewer software as an image overlay. By specifying geographic parameters in the KML, the output of WMS could be re-projected in Google Earth onto a globe automatically.

The Capabilities file of the IPCC Third Assessment Report WMS server was extracted and analyzed by Java program developed by GMU team in order to generate all possible WMS requests and wrap them into nested KML files, using the folder metaphor to organize the presentation of the models, scenarios, and parameters in a logical hierarchy, with the final images ordered and named to support time series animation in KML. Furthermore, a java servlet program was also developed by GMU team to generate legend file for each WMS request so that they could be displayed together in Google Earth.

The KML file was then uploaded on one of GMU servers and registered with GEOSS Component and Service Registry (and GEOSS Clearinghouse) so that GEOSS community users could access it online. The KML file would be automatically launched on computers with Google Earth or other KML viewer software installed to recognize the KML file type. In this way, long-term animated climate data generated from IPCC Third Assessment Report could be visualized on-the-fly by user’s request.


Date: 28Dec10

Figure 2 IPCC Third Assessment Report Long-Term Climate Data Visualization In Google Earth

3) Station Climatology KML from NOAA NCDC data (GMU)

A CD-ROM describing the 1961-1990 global standard climate normals was acquired from NOAA NCDC as a source for station climatology data (reference needed) [1]. This CD-ROM contained a collection of DOS files which described the 1961-1990 global standard climate normals for over 4000 stations worldwide computed by more than 135 countries and territories.

The GMU team processed the content of the CD by program and loaded the climate data into a MySQL database. A PHP program was also deployed to read data from the database and generate statistical climate graphs of each station by request. A Java program was then developed to read and generate KML file containing information about all weather stations in Arctic region. The KML file was designed to show basic information and climate graph (using PHP program) for each station.

The KML file was then uploaded on one of GMU servers and registered with GEOSS Component and Service Register System so that GEOSS community users could access it online. The KML file would be loaded automatically on computers with Google Earth or other KML viewer software installed. In this way, historical weather station climatology in Arctic region can be visualized on-the-fly by user’s request.


Date: 28Dec10

Figure 3a System Workflow for Generating Station Climatology KML from NOAA NCDC CD at CSISS/GMU

Figure 3b WMO Weather Stations of Canada Showing in Google Earth

Parse

NOAA Data CD

Java Program

Database Insertion SQL Commands

Ingested into

Station Climatology KML

Generate KML

Generate

PHP

Query Station Climatology Record Database

Generate Analysis graph

Google Earth

Load into

References


Date: 28Dec10

Figure 3c Station Climatology KML showing in Google Earth (Lynn Lake, MB, Canada)

5.2 Use of the GCI The following GCI components are directly utilized in this scenario: Component and Service Registry (CSR) for community registration of the KML resources, the GEOSS Clearinghouse as search engine, and the GEO Portal to provide the user interface for search. A deficiency in the GEO Portal implementation was noted in the search and presentation of suitable metadata. In all cases, a familiar resource format (KML) was being sought for presentation and access. Although there are multiple URLs in the underlying metadata, the portal presentation of the basic record only showed the information link. It was discovered that if resources were registered as services in the CSR, that this actionable link would not be presented to the user except through the Full Metadata view. To improve usability, the recognition that a resource has a URL that could be acted upon – drawn on a map or globe, available to launch a desktop application – should be embedded in the viewer result as links, buttons, or similar options. Otherwise, most users would not know what to do with the one or more links provided. It is recommended that the Portal provide the means to detect WMS, KML, and similar resource types, and offer to render them or pass them along to helper applications on the desktop.

5.3 Demonstrations The scenario was described in a six-minute video linked from the following page as “Biodiversity: Arctic SDI”: http://www.ogcnetwork.net/pub/ogcnetwork/GEOSS/AIP3/pages/Demo.html


Date: 28Dec10

5.4 Future plans for deployment All three components and the data services that lie behind them (IPCC WMS and UNIDATA Motherlode THREDDS services) are intended to stay online indefinitely. These resources have been registered as Components and Services in the GEOSS for general public access. There are no restrictions on the use of these visualizations. In addition to the visualization services, actual data services are available for IPCC data – as model data downloads – and for the general climate models hosted by UNIDATA, available as netCDF via OGC WCS.

6. Discussions and Lessons Learned The first issue is a need in access service interfaces to specify time relative to the present so that a client can make requests for the "latest" or "last ten" instances of a given type of data. In the case of forecast model output, it is often useful to request data that applies in the future, e.g., the forecast temperature for the next 48 hours. The lack of such a capability makes it difficult or impossible to get the required data from a server which is making available hundreds of new data products each second (weather radar for example). This gap also makes it cumbersome to create automated scripts that request the latest real-time maps. This came to light in our efforts to create KML scripts that request maps of the latest weather data from WMS servers for display in Google Earth. A second difficulty is sometimes referred to as the "granularity issue." In cases where there are large collections of datasets on a server, it has been known for sometime that all the OGC "GetCapabilities" requests generate long lists of available layers, coverages, observations, and features. The lists can be so long as to be of little practical use. At one time, it was thought that using catalogs would be the solution, but our experience has been that this just shifts the problem into the domain of the catalog system which has to harvest the long lists and then serves them up as responses to search queries. A facility partitioning these long lists into groups, or files, or folders, or collections, or categories is needed. In addition, a mechanism is needed for drilling down from a collection level entry to the actual dataset level where the data may be retrieved by the client. Unfortunately, the concept of "collections" was left out of the most recent proposed version of the Observations and Measurements specs. At some point, this needs to be revisited. From a deployment point of view, the visualization analysis of near real-time weather conditions and forecasts presented some computational challenges to the selected solution. The value of the model interface is in the diverse number of parameters and time steps available to the end user for multiple time runs of the models. However, one side effect is that rendered snapshots of the selected variables must be generated on demand for the full period of interest to support the animation function in Google Earth. Because these views are generated on-demand, the inability to deliver these rapidly often caused the user to have to run the loop several times in order to fetch all time slices. This side-effect is not a problem with the standards, per se, but of the underlying server software being able to respond quickly, to pre-cache, or otherwise anticipate the request for animation generation. This also indicates one possible upgrade on the Google Earth side to enable the success retrieval of each KML when presenting the series as an animation.

7. References [1] Global Climate Normals 1961 – 1990. http://ols.nndc.noaa.gov/plolstore/plsql/olstore.prodspecific?prodnum=C00058-CDR-A0001

[2] IPCC Data Distribution Center Visualisation. http://ipcc-data.org/maps/

[3] UNIDATA Motherlode Data Server http://motherlode.ucar.edu

arctic climate and weather forecast model visualization scenario

Documents