Oct 27‐29, 2009 Boulder, CO Fourth FORMOSAT‐3/COSMIC Data Users Workshop
COSMIC Data Analysis and Archive Center (CDAAC) Current Status and Future Plans
Bill Schreiner B. Kuo, C. Rocken, S. Sokolovskiy, D. Hunt, B. Ho, X. Yue, T.‐K. Wee,
K. Hudnut, M. Sleziak‐Sallee, T. VanHove
UCAR/COSMIC Program Office
Oct 27‐29, 2009 Boulder, CO Fourth FORMOSAT‐3/COSMIC Data Users Workshop
Outline
• COSMIC System Overview • CDAAC Data Product Status and Highlights
– Neutral Atmosphere
– Ionosphere • Future Plans
– Planned CDAAC 3.0 Enhancements
– Development and Processing Schedule
• Summary
Oct 27‐29, 2009 Boulder, CO Fourth FORMOSAT‐3/COSMIC Data Users Workshop
CDAAC
NESDIS
GTS
NCEP
ECMWF
CWB
UKMO
Canada Met.
JMA
BUFR Files WMO standard 1 file / sounding
Getting COSMIC Results to Weather Centers
JCSDA
Meteo France
COSMIC OperaConal Processing
Science & Archive
TACC
AFWA
Input Data - COSMIC data - GPS ground data - GPS Bits - GFS Forecast - IGS/IGU ORB/CLK - Bernese Config files
Research Community
SFTP
Unidata’s LDM
WGET
RTS
Oct 27‐29, 2009 Boulder, CO Fourth FORMOSAT‐3/COSMIC Data Users Workshop
CDAAC Processing Flow
LEO data Level 0--level 1
POD (Orbits and clocks)
Excess Phase Full Spectrum Abel Inversion
1-D Var Moisture Correction
Fiducial data
Atmospheric processing
Excess Phase Processing Abel Inversion
Combination with other data
Ionospheric processing
Profiles
Real time Task Scheduling Software
atmPhs atmPrf
ionPrf ionPhs
bfrPrf
podTec scnLv1 tipLv1
wetPrf
Oct 27‐29, 2009 Boulder, CO Fourth FORMOSAT‐3/COSMIC Data Users Workshop
COSMIC System Summary
• COSMIC spacecraU (power and aWtude anomalies) and payloads (low SNRs) funcZoning with some degradaZon
• COSMIC RTS Download staZons ‐ Nominal – NOAA: Fairbanks, Tromso, Wallops (> 70 passes/day)
– NSF/NASA: McMurdo (10 passes/day)
• ~20% missed dumps to due SpacecraU/Payload/RTS problems • COSMIC 6‐staZon Bit Grabber (GPS Data Bits) network – Nominal
• CDAAC (2.0) – producing 1500–2500 profiles per day – > 2.0M/2.1M Neutral‐Atm/Ionospheric profiles to date
– Data Latency • > 90% observaZons available within 3 hr • Average BUFR file data latency ~90 min, single dump ~ 75 min
– CDAAC Reliability: > 99.5% – Providing data to 1,198 registered users from 54 countries
• OperaZons funded through April 2011
Oct 27‐29, 2009 Boulder, CO Fourth FORMOSAT‐3/COSMIC Data Users Workshop
COSMIC OccultaCon Count
> 2 Million Profiles in Real Time Launch to present
Oct 27‐29, 2009 Boulder, CO Fourth FORMOSAT‐3/COSMIC Data Users Workshop
Neutral Atmospheric Highlights
Oct 27‐29, 2009 Boulder, CO Fourth FORMOSAT‐3/COSMIC Data Users Workshop
Data Product Summary
• MulZ‐mission profiles > 2.5M (atmPrf: version 2007.3200) - COSMIC (4/06‐3/09: ~2M), CHAMP (5/01‐10/08: 393,000), GRACE (11/06‐3/09: 90,000), GPSMET
(5/95‐2/97: ~5,000: ~4,000 AS)
• COSMIC penetraZon to 1 km of surface - 85% high laZtudes, negligible N bias
- 65% low laZtudes, negaZve N bias wrt ECMWF (~2%) and Radiosondes (~1%)
• COSMIC Rising/SeWng mean N differences (~0.2‐0.3%) in stratosphere (> 30 km) wrt ECMWF
• COSMIC Quality Control: ~35% marked bad - ~25% due to poor L2 signal quality
• Data Processing Status - Implemented/evaluated L2C processing
- Implemented Phase Matching processing, sZll using FSI
- InvesZgated/improved filtering methods
- InvesZgated/modified truncaZon of RO signals
- SZll using dynamic transiZon height between GO/FSI
• DisseminaZng New Version of Data Products - Worked with weather centers to update BUFR files
- Real‐Zme: since Oct 12, 2009
- Post‐Processing (version 2009.2650): since Apr 2009
Oct 27‐29, 2009 Boulder, CO Fourth FORMOSAT‐3/COSMIC Data Users Workshop
• ECMWF Experiments: Old/New COSMIC data Nov ‐ Dec 08 and Jan 09
• The COSMIC processing changes have improved the GRAS departures considerably, indicaCng much beIer consistency between the data.
Noise normalised GRAS Departures
Red = with oper. COSMIC
Black = with new COSMIC
Mean (o-b)/ SD (o-b)/
Courtesy: Sean Healy ECMWF
Improvements to CDAAC Inversion SoKware: > 20 km
Oct 27‐29, 2009 Boulder, CO Fourth FORMOSAT‐3/COSMIC Data Users Workshop
Improvements to CDAAC Inversion SoKware: < 6km
StaCsCcal comparison of COSMIC and ECMWF (high resoluCon model) fracConal refracCvity differences versus alCtude (km) for 1 July 2007. The leK panel is for COSMIC inversions that use the sliding median filter and the right panel is for inversions that don’t use the sliding median filter. The red lines are the mean differences, the green lines are the standard deviaCons of the differences, and the blue lines are the number of samples.
OLD Inversion Software (With sliding median filter)
NEW Inversion Software (Without sliding median filter)
COSMIC - ECMWF COSMIC - ECMWF
Oct 27‐29, 2009 Boulder, CO Fourth FORMOSAT‐3/COSMIC Data Users Workshop
QuanCfy the Reproducibility of RO data: EsCmates of Structural Uncertainty of
RO data
(Ho et al. JGR, 2009)
Shu-peng Ben Ho, UCAR/COSMIC http://www.cosmic.ucar.edu/~spho/
Comparison of CHAMP RO data processed by GFZ, JPL, UCAR, and WegC FracZonal RefracZvity Anomalies
Independent of processing procedures : the trend uncertainty from GPS RO data processed by different centers < 0.02%/5yrs
Oct 27‐29, 2009 Boulder, CO Fourth FORMOSAT‐3/COSMIC Data Users Workshop
Electron Density Profiles • Electron Density Profiles, EDPs (level2:ionPrf)
- Abel Inversion: Potentially significant inversion errors below F2 layer, less so above F2 layer - Number: COSMIC (4/06-3/09: 2.1M), CHAMP (5/01-10/08: 300,000) - NOTE: Fixed COSMIC s/w bug, corrected noisy profiles after 2008.001 - Quality Control still needs improvement
Lei et al., 2006JA012240
110 km
Unit: 1×105/cm3
ED at 100km from COSMIC
Simulation
Yue et al., GRL (submitted)
Oct 27‐29, 2009 Boulder, CO Fourth FORMOSAT‐3/COSMIC Data Users Workshop
Absolute TEC Data • Absolute TEC (level1b:podTec)
- Number: COSMIC (4/06‐3/09: 2.5M), CHAMP (5/01‐10/08: 200,000)
- NOTE: Fixed COSMIC s/w bug and reprocessed 2008.001 onwards (700,00 more arcs)
- Absolute accuracy ~ 1‐3 TECU
- RelaZve accuracy ~ 0.0024 TECU at 1‐Hz
- Quality Control is good
Ionospheric response to sudden stratospheric warming event (Jan 09) seen in topside ionosphere - vTEC in the topside increase in the morning - vTEC decreases in the afternoon - This results agrees with study of ground-based GPS [Goncharenko et al. 2009,CEDAR meeting]
- NEW: Topside TEC is good evidence that the SSW affects the ionosphere by changing the tides and then the electric field in ionosphere (only EXB can reach high altitude)
Deviation (TECU) vTEC(SSW month) - vTEC(preSSW month)
Oct 27‐29, 2009 Boulder, CO Fourth FORMOSAT‐3/COSMIC Data Users Workshop
ScinCllaCon Data
• ScinZllaZon S4 (level2:atmPrf) - Computed from 50‐Hz OccultaZon L1 SNR observaZons (60‐80 km alt)
- Number: COSMIC (4/06‐3/09: ~2M), CHAMP (5/01‐10/08: 393,000), GRACE (11/06‐3/09: 90,000), GPSMET (5/95‐2/97: ~5,000: ~4,000 AS)
• COSMIC ScinZllaZon S4 (level1b:scnLv1) - Downloaded 1‐Hz values from all line of sights (based on 50‐Hz data)
- Number: > 4.5M LEO‐GPS arcs
Scintillation from level1b:scnLv1 (9-sec averages)
E layer F layer
Oct 27‐29, 2009 Boulder, CO Fourth FORMOSAT‐3/COSMIC Data Users Workshop
Tiny Ionospheric Photometer (TIP)
• COSMIC Calibrated TIP radiances (level1b:tipLv1) - TIP payload off for FM2/3 - > 1400 hours of quality controlled data
COSMIC TIP: 2006-2009, ~1400 hours
Oct 27‐29, 2009 Boulder, CO Fourth FORMOSAT‐3/COSMIC Data Users Workshop
Future Plans – CDAAC Enhancements
• CDAAC 3.0 System – Improve data processing, storage capability, reliability
– Improve soUware and configuraZon management for mulZ‐mission processing (GRAS)
– Improve data access (FTP site, ..), update website, DB interface
– Improve orbit, clock and excess phase processing
• Neutral Atmospheric Products – ConZnue to invesZgate structural uncertainty in LT
– Explore new methods of BA opZmizaZon
– InvesZgate ionospheric correcZon (large‐scale higher‐order residuals, use of L5)
– Restructure Inversion Process (for mulZ‐missions, efficiency and tailored products)
– Add new BAD flag bitmask
– New 1DVAR procedure
– Provide climate products
• Ionospheric Products – Improve EDPs by using informaZon on horizontal gradients
– Improve Quality Control: add BAD flag, error esZmates
– Provide climate products
Oct 27‐29, 2009 Boulder, CO Fourth FORMOSAT‐3/COSMIC Data Users Workshop
Future Plans – cont
• Data Product ValidaZon / CalibraZon - Develop comparison tools - RO‐RO - NWP/Climate Models - Radiosondes - Ground‐based GPS – PWV - AMSU, AIRS, SSMI, Saber
• Development and Data Processing Schedule - ConZnue development of CDAAC 3.0
- Re‐process CHAMP (2009.2650) 1‐2 months - Re‐process COSMIC (2009.2650) 3‐6 months - Re‐process GRACE (2009.2650) 1 month - Re‐process GPSMET (2009.2650) 1 month - Process TerraSAR‐X (2009.2650) 1 month - Process SAC‐C (2009.2650) 1‐2 months
- Process Metop/GRAS ? - Complete CDAAC 3.0 9 months - Re‐Process
Oct 27‐29, 2009 Boulder, CO Fourth FORMOSAT‐3/COSMIC Data Users Workshop
Summary
• Some level of degradaZon of COSMIC spacecraU and payloads
• ~90% COSMIC observaZons (1500‐2000) available within 3 hr (~90 min average latency)
• CDAAC Reliability: > 99.5% • COSMIC data having posiZve impact on NWP, ionosphere and
climate studies, instrument validaZon/calibraZon
• DisseminaZng new version of data products
• ConZnuing development of CDAAC 3.0 • Re‐processing with consistent soUware underway