U.S. Department of the InteriorU.S. Geological Survey

July 2014

OPeNDAP Services – Present and Future at LP DAACOPeNDAP Services – Present and Future at LP DAACBrian DavisBrian Davis11, Rob Quenzer, Rob Quenzer11, Jason Werpy, Jason Werpy22 , David Meyer , David Meyer33, , Chris DoescherChris Doescher33, Chris Torbert, Chris Torbert33

11Stinger-Ghaffarian Technologies, contractor to U.S. Geological Survey (USGS) Earth Resources Observation and Science (EROS) Center, Sioux Falls, Stinger-Ghaffarian Technologies, contractor to U.S. Geological Survey (USGS) Earth Resources Observation and Science (EROS) Center, Sioux Falls, SD,SD,22Information Dynamics, contractor to the USGS EROS Center, Information Dynamics, contractor to the USGS EROS Center, 3 3 USGS EROSUSGS EROS

OPeNDAP Prototype EnvironmentOPeNDAP Prototype EnvironmentThe LP DAAC began offering OPeNDAP services to select science stakeholders in 2013. This has allowed us to evaluate OPeNDAP as a data access protocol and investigate what types of data and access methods LP DAAC users will require in the future. All LP DAAC MODIS tile data was made available and some of the LP DAAC’s MODIS tile data was aggregated in to full time series coverage data sets.

OPeNDAP access through other tools and OPeNDAP access through other tools and partnerspartnersOPeNDAP services allow other interfaces from outside of the LP DAAC to make LP DAAC data available. The USGS Center for Integrated Data Analytics (CIDA) group’s Geo Data Portal (GDP) can access OPeNDAP datasets from the LP DAAC and provide subsets across time. The Giovanni interface from the Goddard Earth Sciences (GES) Data and Information Services Center (DISC) is planning on using LP DAAC OPeNDAP data access to pull LP DAAC data into its system for viewing and analysis by its users. National Snow and Ice Data Center (NSDIC) is working on accessing LP DAAC data through OPeNDAP to develop an Earth Data Cube.

Coding/Scripting with OPeNDAP servicesCoding/Scripting with OPeNDAP servicesUsing R and Shiny, the LP DAAC has developed a prototype application to interactively extract Normalized Difference Vegetation Index (NDVI) values derived from a single tile of 500m resolution 16-day composite MODerate resolution Infrared Spectrometer (MODIS) data from both the Terra and Aqua platforms, and graphically plot the variations by date.

Shiny applications have two components: a user-interface definition and a server script. These components define the input controls and reactive functions which generate textual and graphical data descriptions. The R libraries incorporate the connectivity to OPeNDAP services which provide interactive data archive exploration.

Future Production Future Production EnvironmentEnvironmentThe production environment that is being planned to provide OPeNDAP services to the public will utilize a cluster of servers to maintain performance. The servers will be configured with access to the products on the LP DAAC Data Pool.

CIDA GDP Giovanni

Open-source Project for a Network Data Access Protocol (OPeNDAP) services allow Earth Science data users of the LP DAAC to Open-source Project for a Network Data Access Protocol (OPeNDAP) services allow Earth Science data users of the LP DAAC to directly access data in a flexible, open manner that lets them customize data selection and delivery. Users can focus more on what directly access data in a flexible, open manner that lets them customize data selection and delivery. Users can focus more on what they do with the data and less on how to find, download, arrange, manipulate, and alter it to fit their needs.they do with the data and less on how to find, download, arrange, manipulate, and alter it to fit their needs.

Dataset: MOD17A2.005.h10v04.ncmlGpp_1km[0][0], 0.0006, 0.0008, 0.0008, 0.0004, 0.0004, 0.0005, 0.0007, 0.0012, 0.0012, 0.0007, 0.0007, 0.0006, 0.0009, 0.0013, 0.0013, 0.0013, 0.0011, 0.001, 0.001, 0.001Gpp_1km[0][1], 0.0012, 0.0012, 0.001, 0.0008, 0.0005, 0.0004, 0.0009, 0.0009, 0.0009, 0.0005, 0.0005, 0.0006, 0.0009, 0.0013, 0.0011, 0.001, 0.001, 0.0006, 0.001, 0.001Gpp_1km[0][2], 0.0012, 0.0012, 0.0011, 0.0007, 0.0008, 0.0006, 0.0009, 0.001, 0.0006, 0.0005, 0.0007, 0.0007, 0.0008, 0.0008, 0.0007, 0.0008, 0.0007, 0.0006, 0.0007, 0.0009Gpp_1km[0][3], 0.0011, 0.0012, 0.0012, 0.0012, 0.0008, 0.0007, 0.0009, 0.0009, 0.0006, 0.0008, 0.0006, 0.0008, 0.0007, 0.0007, 0.0008, 0.0008, 0.001, 0.001, 0.0013, 0.001Gpp_1km[0][4], 0.0006, 0.0012, 0.0012, 0.0011, 0.0006, 0.0005, 0.0007, 0.0006, 0.0004, 0.0005, 0.0006, 0.0005, 0.0005, 0.0006, 0.0007, 0.0013, 0.001, 0.0012, 0.0013, 0.0014Gpp_1km[0][5], 0.0006, 0.0009, 0.0012, 0.0012, 0.0012, 0.0011, 0.0008, 0.0005, 0.0004, 0.0006, 0.0006, 0.0006, 0.0006, 0.001, 0.0012, 0.0012, 0.0012, 0.0012, 0.0012, 0.0014Gpp_1km[0][6], 0.0009, 0.0012, 0.0011, 0.0011, 0.0007, 0.0006, 0.0005, 0.0004, 0.0006, 0.0006, 0.0005, 0.0005, 0.001, 0.0012, 0.0007, 0.0012, 0.0013, 0.0014, 0.0014, 0.0014Gpp_1km[0][7], 0.0009, 0.0007, 0.0007, 0.0011, 0.0009, 0.0006, 0.0005, 0.0006, 0.001, 0.001, 0.0007, 0.0006, 0.001, 0.001, 0.0013, 0.0013, 0.0012, 0.0012, 0.0012, 0.0009Gpp_1km[0][8], 0.0007, 0.0007, 0.0006, 0.0013, 0.001, 0.0006, 0.0006, 0.0009, 0.0009, 0.0008, 0.0005, 0.0006, 0.001, 0.0008, 0.001, 0.0012, 0.0011, 0.0011, 0.0011, 0.0007Gpp_1km[0][9], 0.0008, 0.0007, 0.0007, 0.0009, 0.001, 0.0007, 0.0006, 0.0009, 0.001, 0.0007, 0.0007, 0.001, 0.0007, 0.0008, 0.0012, 0.001, 0.0007, 0.0007, 0.0009, 0.0016Gpp_1km[0][10], 0.0008, 0.0008, 0.0007, 0.0011, 0.0009, 0.0009, 0.0007, 0.001, 0.0008, 0.001, 0.001, 0.0007, 0.0009, 0.0008, 0.0007, 0.0007, 0.0011, 0.0015, 0.0019, 0.0018

…

LP DAAC Prototype OPeNDAP Environment

LP DAAC MODIS OPeNDAP Subset Example

OPeNDAP Aggregation Selection Form

Proposed LP DAAC OPeNDAP Cluster

Production LP DAAC OPeNDAP Use Cases

R is a free software environment for statistical computing and graphics. It compiles and runs on a wide variety of UNIX platforms, Windows, and MacOS.

Shiny is an open source R package which provides an elegant and powerful web framework for building web applications using R. Shiny transforms analytical scripts into interactive web applications without requiring HTML, CSS, or JavaScript knowledge.

Technologies like R and other scripting languages enable users of LP DAAC data to dynamically interact with datasets and increase the utilization of them for scientific analysis.

CLIENT SIDE DATA TIERMIDDLEWARE

User Interface(HTML, Javascript, Shiny)

RESTful Web Services(PHP, Python, R)

Hyrax Server

Ajax

LP DAACData Pool

OPeNDAP

Scripting Integration with OPeNDAP through a custom UI

Example R/Shiny Application