support for npp/npoess by the hdf group
DESCRIPTION
Support for NPP/NPOESS by The HDF Group. Mike Folk, Elena Pourmal, Peter Cao The HDF Group November 5, 2009. To develop and adapt HDF5 software and provide HDF5 support for NPP/NPOESS. Goal. Areas of immediate need. Ensure easy and intuitive data access and use by diverse communities - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/1.jpg)
www.hdfgroup.org
The HDF Group
Support for NPP/NPOESS by The HDF Group
Mike Folk, Elena Pourmal, Peter Cao
The HDF Group
November 5, 2009
November 3-5, 2009 HDF/HDF-EOS Workshop XIII 1
![Page 2: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/2.jpg)
www.hdfgroup.org
Goal
2
To develop and adapt HDF5 software and provide HDF5 support for NPP/NPOESS.
HDF/HDF-EOS Workshop XIIINovember 3-5, 2009
![Page 3: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/3.jpg)
www.hdfgroup.org
Areas of immediate need
• Ensure easy and intuitive data access and use by diverse communities
• Enable producers and consumers to view content, manage metadata, and convert data to other formats.
• Provide high-quality, rapid-response support of HDF5 for NPOESS users at all levels.
November 3-5, 2009 3HDF/HDF-EOS Workshop XIII
![Page 4: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/4.jpg)
www.hdfgroup.org
2009-2009 Priorities
• Data accessibility and usability• Developed library of high level APIs to support
NPP/NPOESS data management • Tool development
• Modify h5dump to enable viewing of region references and quality flags
• Modify HDFView to enhance viewing of references and quality flags
• System maintenance• User support
November 3-5, 2009 4HDF/HDF-EOS Workshop XIII
![Page 5: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/5.jpg)
www.hdfgroup.org
NPP/NPOESS Project Information
• Project Web site• http://www.hdfgroup.org/projects/npoess/
• Project description
• Released software
• Documentation
November 3-5, 2009 5HDF/HDF-EOS Workshop XIII
![Page 6: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/6.jpg)
www.hdfgroup.org
NPOESS Web site
November 3-5, 2009 6HDF/HDF-EOS Workshop XIII
![Page 7: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/7.jpg)
www.hdfgroup.org
NPOESS Software FTP Site
November 3-5, 2009 7HDF/HDF-EOS Workshop XIII
![Page 8: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/8.jpg)
www.hdfgroup.org
NPOESS Project Information
• Project Wiki site• http://wiki.hdfgroup.org/NPOESS-Project• Monthly progress reports, links to software
under development, presentations, etc.• Requires a password
• Contact [email protected]
November 3-5, 2009 8HDF/HDF-EOS Workshop XIII
![Page 9: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/9.jpg)
www.hdfgroup.org
HDF NPOESS Wiki
November 3-5, 2009 9HDF/HDF-EOS Workshop XIII
![Page 10: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/10.jpg)
www.hdfgroup.org
The HDF Group
Library of High-Level APIs to support NPOESS data
November 3-5, 2009 HDF/HDF-EOS Workshop XIII 10
![Page 11: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/11.jpg)
www.hdfgroup.org
NPOESS High-Level Library
• Library to support HDF5 region references and packed bits in support of NPOESS quality flags• C APIs• Documentation• Tests• Examples
• Source available from • ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/NPOESS/
source/• Documentation is available from
• http://www.hdfgroup.org/projects/npoess/
• Still a prototype!
November 3-5, 2009 HDF/HDF-EOS Workshop XIII 11
![Page 12: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/12.jpg)
www.hdfgroup.org
Documentation
November 3-5, 2009 HDF/HDF-EOS Workshop XIII 12
![Page 13: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/13.jpg)
www.hdfgroup.org
The HDF Group
h5dump New Features to Support Region
References and Quality Flags
November 3-5, 2009 HDF/HDF-EOS Workshop XIII 13
![Page 14: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/14.jpg)
www.hdfgroup.org
h5dump and Region Reference Data
• New flag –R to display data pointed to by region references
• Improved display of subsets• An element of a dataset is specified by its
indices
• Available in HDF5 1.8.4, November 2009
November 3-5, 2009 HDF/HDF-EOS Workshop XIII 14
![Page 15: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/15.jpg)
www.hdfgroup.org
h5dump and Quality Flags
• New flag –M to display packed bit-field data• Available only in the HDF5 NPOESS branch• Will NOT be in HDF5 1.8.4 release• Source and pre-built binaries can be found at
• ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/NPOESS/
November 3-5, 2009 HDF/HDF-EOS Workshop XIII 15
![Page 16: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/16.jpg)
www.hdfgroup.org
The HDF Group
HDFView New Features to Support
References and Quality Flags
November 3-5, 2009 HDF/HDF-EOS Workshop XIII 16
![Page 17: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/17.jpg)
www.hdfgroup.org
HDFView and references
• Aggregated granules: show the whole dataset pointed to by an object reference
• Granules: show data pointed to by a dataset region reference
November 3-5, 2009 17HDF/HDF-EOS Workshop XIII
![Page 18: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/18.jpg)
www.hdfgroup.org
Objects Pointed to by Object References
November 3-5, 2009 HDF/HDF-EOS Workshop XIII 18
![Page 19: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/19.jpg)
www.hdfgroup.org
Data Pointed to by Region References
November 3-5, 2009 HDF/HDF-EOS Workshop XIII 19
![Page 20: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/20.jpg)
www.hdfgroup.org
HDFView and quality flags
• By default, integers with quality flags are displayed as 8-bit decimal values
• Needed: a way to show values of specific quality flag bit fields
November 3-5, 2009 20HDF/HDF-EOS Workshop XIII
![Page 21: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/21.jpg)
www.hdfgroup.org
Default display: 8-bit decimal integers
November 3-5, 2009 HDF/HDF-EOS Workshop XIII 21
![Page 22: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/22.jpg)
www.hdfgroup.org
Displaying quality flags in 8-bit integers
• Use “Open As” on dataset with quality flags.
• Apply bitmask to choose individual quality flags
November 3-5, 2009 HDF/HDF-EOS Workshop XIII 22
![Page 23: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/23.jpg)
www.hdfgroup.org
Examples of bit masks
• Applied mask in red at top-right, e.g., “By bitmask {0, 1}”
November 3-5, 2009 HDF/HDF-EOS Workshop XIII 23
![Page 24: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/24.jpg)
www.hdfgroup.org
Quality Flags
• Data can be displayed in decimal, binary, or hexadecimal
November 3-5, 2009 HDF/HDF-EOS Workshop XIII 24
binary
decimal
hexadecimal
![Page 25: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/25.jpg)
www.hdfgroup.org
HDFView plug-in to Show Quality Flags
• Implemented using “TreeView” plug-in• Open HDFView• Select “User Options” • Select NPOESS plug-in• Restart HDFView
• Available with HDFView 2.5 patch3
November 3-5, 2009 HDF/HDF-EOS Workshop XIII 25
![Page 26: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/26.jpg)
www.hdfgroup.org
The HDF Group
Possible Future Work
November 3-5, 2009 HDF/HDF-EOS Workshop XIII 26
![Page 27: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/27.jpg)
www.hdfgroup.org
Possible future work
• Add new features
• Enhance current features
• System support
• User support
November 3-5, 2009 HDF/HDF-EOS Workshop XIII 27
![Page 28: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/28.jpg)
www.hdfgroup.org
Maintenance
• Maintenance NPOESS/NPP features • High-Level Library• H5dump, other tools?• HDFView• Testing, documentation, releases
November 3-5, 2009 HDF/HDF-EOS Workshop XIII 28
![Page 29: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/29.jpg)
www.hdfgroup.org
New features
• Support Variable-Length datasets
• Ease access to information about quality flags• Use definition of quality flags, which are in:
• The Common Data Format Control Book
• XML files as part of the CDFCB-X delivery
• Apply the structure of the XML to determine whether a field is a packed quality flag byte
• Tell us what you need!
November 3-5, 2009 HDF/HDF-EOS Workshop XIII 29
![Page 30: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/30.jpg)
www.hdfgroup.org
Enhance current features
• Extend reference display options• E.g., show properties of the referenced data
• Revise default TreeView and TableView to improve NPOESS plug-in support
• Improve quality of testing, documentation, and binary and source distribution
• Tell us what you need!
November 3-5, 2009 HDF/HDF-EOS Workshop XIII 30
![Page 31: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/31.jpg)
www.hdfgroup.org
NPOESS/NPP System Support
• Daily testing on systems critical to NPOESS/NPP• Linux 32 and 64-bit• Windows• AIX 5.3• Solaris 32 and 64-bit
• Regular testing of NPOESS high-Level library• Linux 32 and 64-bit• Windows• Solaris 32 and 64-bit
• Porting to requested compilers• Performance tuning for applications• Improved support for C++• Support for netCDF-4 access to NPOESS SDR and EDR
products
November 3-5, 2009 31HDF/HDF-EOS Workshop XIII
![Page 32: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/32.jpg)
www.hdfgroup.org
NPOESS/NPP User Support
• Priority support to NPOESS/NPP users• Send email to [email protected]• Add NPOESS to the subject field
• Example: Subject: [NPOESS] Where is the plug-in?
• Response in less than 4 hours• Continue gather requirements and enhance
HDF5 software as needed
• We want to hear from you!
November 3-5, 2009 32HDF/HDF-EOS Workshop XIII
![Page 33: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/33.jpg)
www.hdfgroup.org
Special Research Projects
• Improvements to HDF5 or HDF5 NPOESS products with the data management portion of the Raytheon Algorithm Development Library (ADL)
• Implement OpeNDAP server for HDF5 NPOESS SDR and EDR products
• Develop NPOESS specialized version of the HDF5 tool h5check
• Performance tuning of HDF-NPOESS product files
November 3-5, 2009 33HDF/HDF-EOS Workshop XIII
![Page 34: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/34.jpg)
www.hdfgroup.org
Questions?
November 3-5, 2009 HDF/HDF-EOS Workshop XIII 34
![Page 35: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/35.jpg)
www.hdfgroup.org
The HDF Group
Questions/comments?
November 3-5, 2009 HDF/HDF-EOS Workshop XIII 35
![Page 36: Support for NPP/NPOESS by The HDF Group](https://reader038.vdocuments.site/reader038/viewer/2022110402/56812a81550346895d8e12a2/html5/thumbnails/36.jpg)
www.hdfgroup.org
Acknowledgements
This work was supported by cooperative agreement number NA08AANEG0128from the National
Aeronautics and Space Administration (NASA).
Any opinions, findings, conclusions, or recommendations expressed in this material are
those of the author[s] and do not necessarily reflect the views of the National Aeronautics and Space
Administration.
November 3-5, 2009 HDF/HDF-EOS Workshop XIII 36