the next generation mcidas a look towards the future 21 st iips 8.2 san diego, ca - 11 january 2005...

The Next Generation McIDASA Look Towards the Future21st IIPS 8.2 San Diego, CA - 11 January 2005

Dave Santek

Tom Whittaker

Thomas Achtor

Space Science & Engineering Center (SSEC) , University ofWisconsin - Madison

Outline

• Basic functionality of McIDAS-X

• History of McIDAS and current user base

• The next generation: McIDAS-V– VisAD

– IDV (Integrated Data Viewer)

• Data servers: ADDE and OpenADDE

What is McIDAS ?(Man computer Interactive

Data Access System)

– UNIX, PC & Mac capable

• A synergistic tool that integrates numerous data types into one system

• First developed in the early 1970s

• Still in use world-wide at research, operational, educational, and commercial sites

• Collection of user programs and libraries for visualizing and analyzing geophysical data (focus on environmental satellites)

Key McIDAS Attributes

• Access to extensive geophysical database

• Core package plus user-written applications

• Diverse functionality through software (1 million + lines of code)

• Extensive visualization capabilities

• Satellite and NOAAPORT data ingest

McIDAS Functionality

• Digital Image Processing

• GIS Applications

• Weather and Climate Data Analysis and Applications

• Graphical Displays of Data & Information

• Gridded Data Processing and Analysis Tools

• Display & Process Control Utilities

1968WINDCO

generates wind vectors from ATS images

1972First Generation of

McIDAS Runs on Harris /5

with 96 KB of programmable

memory and 2-5 MB hard drives

1976GOES ingest

system added to McIDAS

1979Wichita, OK tornado disaster: Congress directs Operational

McIDAS system to be installed at the National Severe Storms Forecast

Center

1983People’s Republic of China funds port of

McIDAS software to IBM mainframe

1985McIDAS runs on PCs under DOS

operating system

1987McIDAS runs on PCs under OS/2

operating system

and McIDAS Users’ Group is

formed

1991McIDAS runs on

UNIX workstations

1994Satellite and

conventional data is served from UNIX

workstations, beginning the use of ADDE (Abstract Data

Distribution Environment)

1997McIDAS Users’ Group sunsets

support for mainframe

McIDAS1970 1980 1990 2000

McIDAS Users

• NOAA – NESDIS, AWC, SPC, TPC, etc.

• NASA – STS, LaRC, MSFC, JPL

• Unidata – 130 universities and colleges

• International – EUMETSAT, ECMWF, Spain, Greece, Mexico, Australia

• Industry – Boeing, Honeywell, Weathernews, Universal Weather, Meteorlogix, etc.

Why Change?

• McIDAS software (written in Fortran 77 and C) has a 30 year heritage resulting in limited extensibility potential

• New visualization concepts cannot be incorporated (4D)

• Forthcoming environmental satellite data cannot be utilized in the best way (GOES-R & NPOESS operational systems)

The road to the next generation Geostationary Sounders

(# of channels)

VAS (experimental)

GOES Sounder (operational)

GIFTS (experimental)

(12)

(18)

(~1600)

(~1600)

ABS/HES (operational)

time

The ABS/HES is the next step from experimental or low spectral resolution to operational high-spectral resolution, high temporal sounders.

HIRS (operational)

CrIS (operational)

IASI (operational)

AIRS (operational)

Aircraft data (experimental), IMG

Ground-based (exp)

Pioneering work (theory)

IRIS (experimental)

(~2400)

Simulated GOES-R winds (left) versus GOES current operational winds (right)

GOES R - IHOP simulation 1830z 12 June 02   GOES-8 winds 1655z 12 June 02 

Hyperspectral Altitude Resolved Water Vapor Winds

McIDAS-V: Looking to the Future

We seek to advance the functionality of McIDAS to meet the challenges of the future while retaining its current capabilities, including user-developed code.

We are investigating a means to transition existing McIDAS applications to operate in a new environment that does not have the limitations of the current McIDAS system.

IDV: The CandidateTransition System

• IDV (Integrated Data Viewer)– Based on SSEC’s VisAD package– Has a data model that is inherently capable

of working with data types, both now and in the future

– Developed for the education community by Unidata

– Designed to replace McIDAS-X and GEMPAK at universities

What is VisAD?

• Open-source, Java library for building interactive and collaborative visualization and analysis tools

• Features include:– Powerful mathematical data model that embraces

virtually any numerical data set

– General display model that supports 2- and 3-D displays, multiple data views, direct manipulation

– Adapters for multiple data formats (netCDF, HDF-5, FITS, HDF-EOS, McIDAS, Vis5D, etc.) and access to remote data servers through HTTP, FTP, DODS/OPeNDAP, and ADDE protocols

VisAD’s Data Model

• Designed to support virtually any numerical data

• Metadata can be integrated into each data object (units, errors, coordinates, etc.) – carried along through mathematical operations

• Supports mathematical operations as well as evaluation and resampling of data

• Supports spatial and temporal co-location of data

• Supports data sharing among different users, different data sources and different scientific disciplines

Gully erosion/deposition from VisAD-based

DEMViewer

What is the IDV?

• Unidata developed scientific analysis and visualization tool

• Freely available Java™ framework and reference application

• Provides 2- and 3-D displays of geo-scientific data (plus, of course, animations)

• Stand-alone or networked application

http://www.unidata.ucar.edu/idvhttp://www.unidata.ucar.edu/idv

IDV Features

• Integrated displays of a variety of data types

• Support for a variety data access methods

• Interactive probes• User defined formulas and

functions• Bundling of user preferences• Integrated HTML viewer• Easy configuration• Integrated documentation

Supported Data Formats

– Scientific• Geo-referenced

netCDF data (local or remote)

• McIDAS/ADDE – Societal

• GIS - ESRI shapefiles (polygons)

– Educational• HTML• Quicktime

NO2 concentrations over Berlin, Germany – data from netCDF, maps from shapefile

The IDV can integrate displays of a variety of geo-scientific and other data including:

Surface observations with Jet Stream winds

EOS AIRS Hyperspectral Data Analysis Applications

Current Efforts at SSEC

• A mechanism to transform McIDAS graphical displays into the geo-referenced VisAD / IDV display

• A unified user interface to access image data via ADDE and OPeNDAP

• The capability to run IDV processes in a background mode

• A mode to run heritage McIDAS applications

• OpenADDE: ADDE data servers available as Open Source

What is ADDE?

• Abstract Data Distribution Environment• Key data access method for McIDAS-V• Data transfer and reformatting

– All data returned in common format: not as stored

• Sectorizing by geographic coverage and spectral bands

• TCP/IP remote access through registered port 112• Compressed transmission• Variety of clients can access the servers• In use for 10 years

What servers are available?

• McIDAS format: Image, grid, point, text• netCDF as grid and point• HDF as point• MODIS Level 1b and products as images• AIRS Level 1b• AMSR-E Level 1b• Level III NEXRAD• AWIPS image• GRIB

What clients can be used?

• McIDAS-X• McIDAS-Lite• IDV• Matlab• IDL

What is OpenADDE?

• Open Source version of the ADDE servers and required McIDAS-X library routines

• Another method for data providers to serve data– McIDAS sites & non-McIDAS sites– Subsecting

• Encourage data providers to write servers for new data types

• Code provided under the GNU Lesser General Public License

What is OpenADDE?

• Initial release are only the image read servers:– McIDAS Area, MODIS & AIRS HDF,

NEXRAD, GINI, AWIPS (?)– McIDAS library routines

• Navigation and calibration modules, utilities

• 30% of the McIDAS-X library code

• Available at http://www.ssec.wisc.edu/openadde– Source code, installation and configuration

scripts, sample data

Where’s the data?

• Servers at NOAA, NASA, SSEC, Unidata• Realtime global GOES• Realtime global MODIS Level 1b• GOES product files• Polar composites of satellite data

The Next Generation McIDASA Look Towards the Future21st IIPS 8.2 San Diego, CA - 11 January 2005

Questions about McIDAS-V or OpenADDE?

Stop down to the University of

Wisconsin booth: 406