1

Overview of Hyper-spectral Infrared Activities in STAR

Chris BarnetJCSDA Infrared Sounding Working Group

Jan. 30, 2009

Chris.Barnet@noaa.gov

2

Outline of Talk

• Status of AIRS

• Status of IASI

• Plans for NPP CrIS– CrIS/ATMS Cal/Val team

• Working member of GEOS Reference Upper Air Network– Meeting to be held in Norman, OK in March

2009

3

Efforts are in support of the Initial Joint Polar System: An agreement between NOAA & EUMETSAT to exchange data and products.

NASA/Aqua1:30 pm orbit (May 4, 2002)

NPP & NPOESS1:30 pm orbit(6/2010, 2013, 2018)

EUMETSAT/METOP-AEUMETSAT/METOP-A9:30 am orbit (Oct. 19, 2006, 9:30 am orbit (Oct. 19, 2006, 2010, 2015)2010, 2015)

20 years of hyperspectral sounders are already funded for weather applications

4

Space Hyper-spectal Assets: A Comparison of AIRS, IASI & CrIS

AIRS IASI CrIS

Satellite height (km) 705 833 824

Equator Crossing Time (LST) 1:30 9:30 1:30

Fields or regard (FOR) per day 324000 324000 324000

Size of FOV (km, at nadir) 13.5 x ≈20 12.2 14

FOV per FOR 9 4 9

Channels per FOV 2378 8461 1305

Optical path difference (LW/MW/SW)

n/a 2.0/2.0/2.0 0.8/0.4/0.2

Spectral sampling (cm-1) /2400 0.5/OPD 0.5/OPD

Level 1 storage per day 35 GB 30 GB 30 GB

Minutes of data per granule 6 ≈ 3 6

AMSU scan lines/granule 45 22 or 23 45

# Granules/day 240 480 240

5

Status of AIRS

• Moving system to LINUx– Slow and painful due to lack of funding– Migration from SGI to LINUx should occur in spring 2009.

• AIRS Science team Version 6 Level 2 is in work– Biggest change is surface emissivity is more stable (less day/night variance)– Proper handling of loss of AMSU Chl.4 and 5– Scattering RTA will be installed and correction for aerosols made to level.2

products.• Recently found that AIRS BUFR has compressed radiances

– AIRS science team implemented radiance truncation in version 4 (Feb 2005) to reduce volume of level.1

– It was NOAA’s intent to not alter the level.1 radiances but due to a mis-communication with JPL the level.1 products were truncated.

• Very small impact to temperature and moisture products.• Impacts level.2 trace gas products.

– Version 4 was operational at NOAA on July 13, 2005• AIRS BUFR’s radiances have been truncated at NEDN/2• All stored level.1 (gridded products, sonde matchups) are truncated.

– This can be easily turned-off. We will be providing uncorrupted radiances in parallel via the LINUx system soon.

6

Status of IASI

• IASI level-1 system became operational at NOAA’s Environmental Satellite Processing Center (ESPC, a.k.a. OSDPD) on July 18, 2007

• IASI level-2 pre-operational system has been running continuously on our ESPC development machines since April 2008.

• IASI level-2 system became operational at ESPC on Aug. 14, 2008

• Preliminary validation is complete.

7

Planned upgrades to the IASI system

• METOP AMSU Chl.7 noise is increasing and has exceeded specification.– Plans are to remove AMSU Chl.7 from our processing.

• Planned Level.2 product upgrades– ILS sub-pixel homogeneity correction

• ILS shape and centroid is a function of the spatial radiometric structure within the 12 km FOV.

• Can either correct the radiance of include a spectrally correlated term in the error covariance matrix.

• In cloud cleared retrievals the impact of this effect is partially mitigated by the spatial averaging of the 4 FOV’s

– Updated regression coefficients– Improvements to trace gas algorithms.

8

IASI L1C NRT Granule ProductsAvailable via DDS in Near Real Time

Spectral Subset Data Type Spatial Subset Format

616 chls IASI Radiance Warmest FOV from every FOR BUFR NetCDF

616 chls IASI Radiance First FOV from every FOR BUFR NetCDF

616 chls IASI Radiance All 4 FOVs from every FOR BUFR NetCDF

616 chls IASI Reconstructed Radiance (1 band)

1 FOV from every FOR BUFR NetCDF

616 chls IASI Reconstructed Radiance (3 bands)

1 FOV from every FOR BUFR NetCDF

616 chls IASI Reconstructed Radiance (1 band)

4 FOVs from every FOR BUFR NetCDF

616 chls IASI Reconstructed Radiance (3 bands)

4 FOVs from every FOR BUFR NetCDF

8461 chls IASI Radiance 4 FOVs from every FOR NetCDF

8461 chls IASI Radiance 4 FOVs from 2 scans/granule NetCDF

FOV = Field of View; FOR = Field of Regard. Orange refers to internal files.

9

Product archiveAvailability usually within 6 hours

• Products available in near-real time via NOAA/ESPC Data Distribution Server (by subscription)

• Products available within ≈ 6 hours and archived at NOAA Comprehensive Large Array-data Stewardship System (CLASS)

www.class.ncdc.noaa.gov

10

IASI L2 NOAA Unique Products Granule Products (DDS)

Instrument Channel Data Type Description IASI FOV #

Format

IASI 616 CCR Cloud cleared radiance for each FOR

(uses all 4 FOV’s)

BUFR NetCDF

IASI n/a Geophysical T(p), q(p), O3(p), CO(p), CH4(p), SST/LST, surface emissivity, cloud fraction, cloud top height, convective products.

(uses all 4 FOV’s)

NetCDF

IASI (using AVHRR)

616 RAD Pick clearest IASI FOV for each FOR using AVHRR

1 (clearest)

BUFR NetCDF

AVHRR (on IASI FOVs)

5 RAD (clear and cloudy)

AVHRR channels spatially convolved to IASI FOV’s

1,2,3,4 BUFR NetCDF

IASI (using AVHRR)

616 CCR IASI CCR w/ AVHRR QA

(uses all 4 FOV’s)

BUFR NetCDF

AVHRR Products will be available in FY09

11

Activities in support of NPP CrIS

• Simulation system is up and running.– Uses GFS forecast and simple emissivity and cloud models to

generate ATMS and CrIS radiances 24/7.– Building NGST HDF format SDR files.

• AMSU-like spatial averaging of ATMS

– Building BUFR files for NWS applications.• Current choice of apodization is Hamming

– Less severe than Blackman– Reversible

– Selection of channel subsets for BUFR files is in progress.

• Porting IASI level.2 system for NPP CrIS/ATMS• Evaluation of NGST level.2 (EDR) product system. • STAR is lead of government CrIS/ATMS EDR calibration

and validation team.

12

Hierarchy of NPP calibration and validation activities

PL = Pre-launch EOC = Early Orbit Checkout (30-90 days) ICV = Intensive Cal/Val (stable SDR to L+24 m LTM = Long-term monitoring (to end of mission)

Activity Time-frame Value

Use of proxy datasets PL,EOC Exercise EDR and fix issues.

Use of forecast & analysis fields EOC Early assessment of performance

Compare early EDR’s to operational products from AIRS & IASI

EOC,ICV,LTM Early assessment of performance, diagnostic tools to find solutions.

Compare SDR’s w/ AIRS and IASI via SNO’s and double differences

ICV,LTM Separate SDR/EDR issues at detailed level.

Operational PCA EOC,ICV,LTM Identify and categorize interesting scenes. Monitor instrument health.

RTG-SST and Dome-C AWS LTM Long-term stability of ICT

Operational RAOB’s ICV,LTM Early assessment, long-term stability.

Dedicated RAOB’s ICV,LTM Definitive assessment.

Intensive Field Campaigns ICV,LTM Definitive assessment.

Scientific Campaigns of Opportunity Whenever Detailed look at specific issues.

13

Coordination with GRUAN

• Peter Thorne (UK/Hadley Center) is chair of GCOS working group on atmospheric reference observations (WG-ARO).

– Focus of this group is to define protocols for the reference network.

– Coordination meeting in March 2008 at ARM-SGP

• Holger Vomel became director of lead-center at Lindenberg in summer 2008.

• Chris Barnet is member of WG-ARO

– Hope to convince site leads to help us launch dedicated multiple sondes at NPP overpass times.

14

Backup Slides

15

Overview of CrIS/ATMS AVTP & AVMP Validation

• Objective– Incorporate lessons learned from AIRS and IASI Validation

• Concentrate on datasets that were proven valuable for global validation for AIRS (ECMWF, NCEP/GFS, RAOB’s, ARM-TWP/SGP/NSA, CAMEX, JAIVEX, etc)

• Work with experiments of opportunity for detailed characterization of products.– Discussions with key users to ensure our cal/val plan meets their needs.

• Have obtained POC’s from NOAA operational liasons.– Define the details of computing statistics from sparse in-situ measurements.

• Details on how to “roll-up” regional statistics need to be worked out and tested prior to launch.

– Characterize performance of EDRs in various ensembles of cases.• Test concepts pre-launch with simulated CrIS datasets and heritage instruments.

• Strategy– Build team of Subject Matter Experts (SME’s) from both customer and science

communities to leverage heritage knowledge and tools as well as assure understanding of Customer Mission Success.

– Leverage exisiting capabilities where-ever possible• operational real-time systems including CrIS/ATMS “NOAA” unique systems• AIRS and IASI processing and validation systems• routine AIRS and IASI instrument monitoring and characterization, • aircraft validation experience.

16

CrIS EDR Specifications(RED are KPP’s, Blue are P3I))

Parameter IORD-II (Dec. 10, 2001) NGST SY15-0007 (Oct. 18, 2007)

AVMP Clear, surface to 600 mb Greater of 20% or 0.2 g/kg 15% ocean, 15.8% land and ice

AVMP Clear, 600 to 300 mb Greater of 35% or 0.1 g/kg 17% ocean, 20% land and ice

AVMP Clear, 300 to 100 mb Greater of 35% or 0.1 g/kg 0.07 g/kg ocean, 0.1 g/kg land and ice

AVMP Cloudy, surface to 600 mb Greater of 20% of 0.2 g/kg 15.8%

AVMP Cloudy, 600 mb to 400 mb Greater of 40% or 0.1 g/kg 20%

AVMP Cloudy, 400 mb to 100 mb Greater of 40% or 0.1 g/kg 0.1 g/kg

AVTP Clear, surface to 300 mb 1.6 K/1-km layer 0.9 K/1-km ocean, 1.7 K/1-km land&ice

AVTP Clear, 300 to 30 mb 1.5 K/3-km layer 1.3 K/3-km ocean, 1.5 K/3-km land&ice

AVTP Clear, 30 mb to 1 mb 1.5 K/5-km layer 1.5 K/3-km

AVTP Clear, 1 mb to 0.01 mb 3.5 K/5-km layer 3.5 K/5-km

AVTP Cloudy , surface to 700 mb 2.5 K/1-km layer 2.0 K/1-km

AVTP Cloudy, 700 mb to 300 mb 1.5 K/1-km layer (clear=1.6) 1.5 K/1-km

AVTP Cloudy, 300 mb to 30 mb 1.5 K/3-km layer 1.5 K/3-km

AVTP Cloudy, 30 mb to 1 mb 1.5 K/5-km layer 1.5 K/5-km

AVTP Cloudy, 1 mb to 0.01 mb 3.5 K/5-km layer 3.5 K/5-km

CH4 (methane) column 1% precision, 5% accuracy n/a

CO (carbon monoxide) column 3% precision, 5% accuracy n/a

17

NOAA/NESDIS Cal/Val Team Members

Team Member Funding Source Activity

Chris Barnet IPO/Cal-Val Cal/Val coordination. Define performance metrics, EDR algorithm issues. Scientific field campaigns of opportunity (START,HIPPO,AEROSE,WAVES). Member of AIRS & IASI Science Team, GCOS/AOPC/WG-ARO.

Mitch Goldberg IPO/Cal-Val NGST EDR testing, PCA analysis of radiance, quick look regression products. Coordination w/ AOPC & GSICS.

Anthony Reale IPO/Instrument Systems NOAA PROduct Validation System (NPROVS) – CrIMSS/IASI/AIRS/ATOVS/RAOB matchups

Changyong Cao IPO/Instrument Systems Development of an integrated instrument cal/val system for NPP/NPOESS (e.g., SNO AIRS/IASI/CrIS). Coordination with GSICS, CEOS-Cal/Val working group chair (WGCV).

Mitch Goldberg NESDIS/PSDI NOAA Unique CrIS/ATMS Product System (NUCAPS)

Mitch Goldberg NESDIS/PSDI IASI Product System

Fuzhong Weng & Sid Boukabara

NESDIS/PSDI Microwave Integrated Retrieval System (MIRS) and activities related to ATMS calibration.

18

Non-NOAA, IPO funded, Cal/Val Team Members

Team Member Organization Responsibilities

Gail Bingham USU/SDL SDR cal/val lead, NGST code f/ cal/val team

Bill Blackwell MIT ATMS SDR/EDR issues. ATMS proxy datasets. NAST-M preparations and intensive campaign support.

John Derber, Paul Van Delst

JCSDA/NCEP Global characterization of ATMS & CrIS biases w.r.t. NCEP analysis ATMS & CrIS SDR/EDR issues.

Allan Larar NASA/LaRC NAST-I preparations and intensive campaign support.

Xu Liu NASA/LaRC EDR algorithm issues, IASI proxy dataset

Hank Revercomb,Dave Tobin

U.Wisc SDR issues (non-Gaussian noise), Scanning HIS preparation and intensive campaigns. ARM best estimate state analysis.

Larrabee Strow UMBC SDR issues, radiative transfer issues, pre-flight instrument cal/val issues, OSS validation.

Joel Susskind NASA/GSFC CrIS/ATMS proxy datasets derived from AIRS/AMSU/HSB.

19

Pre-launch testing using Proxy Data

• Three proxy data test datasets are available.1. Use GFS model + climatologies for trace gases, surface and cloud

properties with forward model– Can test data formats and throughput to NWP centers with 24/7 global stream.– Not robust enough to fully test all algorithm components.

2. Use Aqua AIRS/AMSU/HSB (prior to 1/15/2003).– Has advantage of having 9 IR FOV’s and 1:30 orbit.– Disadvantage is that model must be used to transform radiances subtle AIRS

issues will enter into CrIS radiances.3. Use of METOP-A IASI/AMSU/MHS.

• Direct conversion of IASI to CrIS is robust.• Disadvantage of 9:30 am orbit and only 4 IASI FOV’s

• Proxy datasets must be constructed by SME’s.– Local angle (and rotation) of CrIS/ATMS FOV’s are different than AIRS or

IASI.– ATMS polarization is different.

• Need to identify and resolve as many issues with NGST code as possible prior to launch

– Use of non-LTE channels– Emissivity hinge points– Methodology for deriving empirical bias corrections for CrIS and ATMS.

20

CrIS/ATMS will be simulated 24/7 for more than 1.5 years prior to launch..

• Build a reasonable atmospheric state

• Simulate the NPP orbit.• Simulate radiances using the

MIT ATMS and UMBC CrIS RTA.

• Package ATMS and CrIS radiances in NGST - SDR format (in work).

• Use simulated SDR’s to build BUFR and NETCDF products– Run retrieval system from these

products.

• At launch – flip a switch to send real data down the pipeline.

Used GFS model for T,q,clouds

Used climatologies for trace gases.

Used emissivity models for surface properties.

Simple, opaque cloud model.

21

PC Analysis can be used to characterize the instrument noise using Earth scenes.

• PC’s can be used to compute reduced noise radiance (reconstructed radiances).

• Subtracting observed radiance from reconstructed radiance gives an estimate of instrument noise derived from Earth scenes.– At upper right is IASI noise (red

curve) derived from blackbody measurements compared with noise derived from PC’s. PC’s generated from all 8461 channels shown in blue) and PC’s generated from the 3 individual bands (green) are very similar and very close to the black body derived noise.

– At lower right is the NEDT noise estimate for a single channel (2500 cm-1 on Sept. 10, 2007) shows the expected characteristics as a function of scene temperature (red lines are 1 sigma NEDT and green is 2 sigma NEDT).

22

Aerosol and Ocean Science Expeditions AEROSE

• Supported 4 low-cost “piggy-back” cruises.

• Acquired T,q, O3 sondes at overpass times for AIRS and IASI.

• M-AERI (SST, SSE) and AOD measured on-board.

23

NUCAPSNOAA Unique CrIS/ATMS Processing System

• Science code is the same for AIRS, IASI, and CrIS– File driven architecture (same code runs AIRS, IASI, and CrIS)

• All instrument specific information is read in from files.– Noise file specifies instrument noise characteristics.– RTA file specifies instrument specifications (channels, apodization, etc).

• Channel selection for retrieval steps is read in via namelist.– Design is modular – retrieval modules are programmed via namelist

commands.• Can quickly change retrieval configuration to identify source of problems.

– Full diagnostics. Each retrieval iteration and sub-step is compared to a “truth” state specified by the user (ECMWF, RAOB’s, GFS, etc.).

• This code was up and running within hours of “first-light” for both AIRS and IASI.– Can be used to quickly assess SDR’s and capabilities of CrIS/ATMS

for cloud clearing.– Can be used to quickly assess EDR capabilities w.r.t. AIRS and IASI

algorithm.

24

Trace Gas Products from AIRS & IASI

gas Range (cm-1) Precision d.o.f. Interfering Gases AIRS IASI

H2O 1200-1600 15% 4-6 CH4, HNO3 NASA DAAC CLASS

O3 1025-1050 10% 1.25 H2O,emissivity NASA DAAC CLASS

CO 2080-2200 15% 1 H2O,N2O NASA DAAC CLASS

CH4 1250-1370 1.5% 1 H2O,HNO3,N2O NASA DAAC CLASS

CO2680-795

2375-23950.5% 1 H2O,O3

T(p)NOAA

NESDISCLASS

Volcanic SO2

1340-1380 50% ?? < 1 H2O,HNO3 TBD TBD

HNO3860-920

1320-133050% ?? < 1 emissivity

H2O,CH4,N2ONOAA

NESDISCLASS

N2O 1250-13152180-22502520-2600

5% ?? < 1 H2OH2O,CO

NOAA NESDIS

CLASS

CFCl3 (F11) 830-860 20% - emissivity No plans No plans

CF2Cl (F12) 900-940 20% - emissivity No plans No plans

CCl4 790-805 50% - emissivity No plans No plans

25

Acronyms & Terminology

• AOPC (GCOS) = Atmospheric Observations Panel for Climate• ARM (DOE) Atmospheric Radiation Measurement• AVMP = Atmospheric Vertical Moisture Profile• AVTP = Atmospheric Temperature Moisture Profile• CEOS = Committee on Earth Observation Satellites• “Clear” means 50% cloudiness in field-of-regard• “Cloudy” means > 50% cloudiness in field-of-regard• EDR = Environmental Data Record (level-2)• GCOS = Global Climate Observing System (UNEP/WMO)• GRUAN = GCOS Reference Upper Air Network• GSICS = (WMO/NOAA) Global Space-based Inter-Calibration System• IP = Intermediate Product (e.g., CCR’s, microwave-only product, ozone)• KPP = Key Performance Parameters• NSA = (ARM) North Slope of Alaska• PEATE = Product Evaluation and Algorithm Test Element• P3I = Pre-Planned Product Improvement• SDR = Sensor Data Record (level-1)• SGP = (ARM) Southern Great Planes• TWP = (ARM) Tropical Western Pacific