COSMIC Data Analysis and Archive Center (CDAAC)

Overview and Status

B. Schreiner, B. Kuo, S. Sokolovskiy, D. Hunt, X. Yue, J. Zeng, T.-K. Wee, J. Braun, T. Vanhove, J. Lin, M. Sleziak, J. Castilleja

UCAR COSMIC Program 2nd International Conference on GPS Radio Occultation in 2013

CDAAC

NESDIS

GTS

NCEP

ECMWF

CWB

UKMO

Canada Met.

JMA

1000-2000 WMO BUFR Files per day with Latency ~ 75-90min

Getting COSMIC Results to Weather Centers

JCSDA

Meteo France

COSMIC Operational Processing

Science & Archive

TACC

AFWA

Input Data -  COSMIC data -  Attitude data -  GPS ground data -  GPS NDM Bits -  GFS Forecast -  IGS/ORB/CLK -  Bernese Config

Research Community

SFTP

UCAR/Unidata’s LDM

WGET

RTSs: Alaska Norway McMurdo Hanscom AFB

Providing  data  to  >  2,087  registered  users  from  71  countries  

COSMIC, C/NOFS, SAC-C

~ 3.7 M COSMIC Profiles in Real Time 4/21/06 – 04/28/2013

Change  in    Satellite  Ops  

3

CDAAC Design

CDAAC File types in RED

Level 0--level 1

LEO POD

Excess Phase

Abel Inversion

1-D Var Moisture

Correction

Atmospheric processing

Excess Phase

Ionospheric Inversion

Ionospheric processing

Profiles

Real time Task Scheduling Software

atmPhs atmPrf (α,N,T,P)

ionPrf (EDPs)

ionPhs

wetPrf bfrPrf

GNSS ground data GNSS NDM Bits IGS/IGU ORB/CLK Bernese Config files GFS Forecast

Absolute TEC

Scint.

podTec (TEC)

scnLv1 (S4/σΦ)

LEO Science and SOH data

GNSS Clock Estimation

TIP

tipLv1 (Radiances)

RINEX

podObs

4

How do we know our CDAAC RO data processing is optimal?

•  Review of algorithms and software internally and via publication

•  Identify/Analyze QC failures

•  Compare results (POD, BA, N, T, WV, TEC, EDP) from mission to mission

•  Comparison of results with other data processing software and investigate and understand differences

–  more detailed RO-Trends studies

–  ROPP comparisons

•  Comparison with correlative data and models

•  Get feedback from science application studies

Excess Phase Bug Reported by Stig S. BA noise between 60-80 km

2007.274

2007.274

Metop/GRAS - ZD COSMIC - SD

OLD Software

NEW Software

2012.090

2012.090

Agrees well With GRAS data from EUMETSAT

Excess Phase Bug Reported by Stig S. UCAR Metop/GRAS - ECMWF

2007.274 OLD Software NEW Software

Software fix has insignificant impact on BA’s as compared to ECMWF

Statistical comparison of COSMIC retrieved refractivities to ECMWF over the tropical ocean

Red lines: mean deviation

Green lines: +/- standard deviation Blue lines: penetration (bottom heights of retrieved profiles)

~ 80% profiles penetrate to 1 km ~ 20% profiles penetrate to 1 km; mean and standard deviation (inversion errors) are substantially larger

GPS azimuth: 50-60 deg SNR ~ 400-600 V/V

GPS azimuth: 0-10 deg SNR ~ 700-900 V/V

NEW Neutral Atmospheric Retrievals SW Undergoing Final test/evaluation

•  Software restructured to be more modular and maintainable •  All occultations are processed and made available •  Wave Optics (Phase Matching) Algorithm •  Fixed height (20 km) for GO/WO transition •  New statistical optimization algorithm •  Filtering of Doppler for GO processing consistent with Fresnel

scale (different in time for different occultations) •  Calculation of lat. & lon. of TP: use of BA from climatology

(CIRA+Q) instead of raw BA obs. •  New QC •  Additional scalar output parameters •  New output files

Fixed Height for GO/WO Transition at 20 km

•  Physical resolution: WO ~ 100 m; GO ~ 1.5 km (Fresnel's zone (FZ))

•  BA calculation height range: WO (Phase Matching) from 0 to 21 km; GO above 20 km

•  WO BA filtering window: 100 m from 0 to 11 km; 200 m from 10 to 21 km; transition between 10 and 11 km

•  GO BA filtering window: FZ (~1.5 km) up to 40 km;

•  Transition from WO BA to GO BA:

between 20 and 21 km

Truncation of bending angle profiles retrieved by wave optics transform

Main assumption for application of WO transform: for a given impact parameter, only one ray exists = = only one stationary point in the integral exists Classical truncation approach: when amplitude of WO transform fades New truncation approach: when stationary point (pronounced maximum of the phase function) does not exist in WO transform

Pha

se F

unct

ion

(rad

) NO stationary point

stationary point

Neutral Atmospheric Retrievals Quality Control

•  COSMIC data from Jan 29-31, 2009

•  Maximal deviation of L1 and L2 Dopplers between 20 and 40 km

•  Maximal L1CA SNR averaged between 40 and 80 km

•  New QC Criteria: -  L1CA SNR > 200 V/V -  Max Doppler deviation < 0.1 m/sample -  fractional deviation of retrieved

refractivity from background (NCAR climatology) < 50%

!

Statistical comparison of COSMIC retrieved N to ECMWF For OLD and NEW Retrieval Software

!

Blue = mean Red = STD

OLD = Dashed Line NEW = Solid Line

•  COSMIC data from Jan 29-31, 2009

•  Mean differences are similar in NEW and OLD retrieval software

•  NEW retrieval software has smaller STD than OLD

•  NEW retrieval software has slight

increase in STD near 20 km

•  NEW retrieval software passes 75% of soundings, compared to 65% for OLD

Area of Interest atmPrf Scalar Variable Description Bending angle uncertainty for an occultation

smean Mean deviation of ionosphere-free BA from climatology

Bending angle uncertainty for an occultation

stdv Standard deviation of ionosphere-free BA from climatology

Signal-to-Noise Ratio for an occultation

snr1avg, snr1del snr2avg, snr2del

Mean SNR, linear trend of SNR, for L1 and L2, between 60 and 80 km

Tropopause Height and Temperature

trtwmo, trhwmo trtwmo2, trhwmo2 trtcp, trhcp

Tropopause height and temperature from WMO definition (incl. 2nd tropopause), and cold point definition

Atmospheric Boundary Layer characterization

balmax zbalmax

max. BA lapse in variable sliding window and median height

Atmospheric Boundary Layer characterization

bpnmax zbpnmax

N break point (lapse of N-gradient by sliding linear regression in 0.5 km window) and median height

Ionospheric Scintillation S4

New Scalar Parameters

Sporadic E-clouds Detection of multiple Es effects: heights of TP and intervals of amplitude fade

Atmospheric Boundary Layer characterization

max. N lapse in sliding window 0.3 km and median height

Convection in the moist troposphere

Characterization of moist convection, depth of convective layer (under revision)

Multiple profile truncation heights More or less conservative truncation heights

Neutral Atmospheric Profiles Scalar Output Parameters

Purpose: running applications such as: different BA optimizations, deducing Tropospheric/ionospheric structures from high-res WO transform; error estimation, QC, etc, without re-running the main inversion code level2/benPrf ; argument: impact height; vectors: 2 - High-res L1 and L2 bending angles for entire profile level2/wotPrf; argument: impact height; vectors: 4 - I and Q of the Wave Optics transform (Phase Matching) at the full resolution 1 m for L1 and L2 (when L2C is available) level2/qcfPrf; argument: time; vectors: 8 - L1 and L2 excess phases after removal of nav. data modulation and re-connection of the phases with the post-processing model based on orbits and BA climatology - L1 and L2 SNRs - height of straight line between GPS and LEO - impact height based on orbits and BA climatology - difference between the receiver and the post-processing Doppler models

Level3/mmcBin; - Monthly mean climatologies binned in lat/lon/alt

Auxiliary Output Files

Output from atmPrf files: - output vectors: height, impact parameter, bending angle, refractivity, latitude, longitude, azimuth - reference vector (argument) in BUFR: height (geometric) - definition of the reference height grid: uniform; 300 values between htop (60 km) and hbot - definition of other vectors (functions): sampled on an auxiliary uniform grid, smoothed (Savitzky-Golay) with 200 m window, down-sampled on the reference height grid - all profiles filled with dummy values below hbot RO data (physical resolution <100 m in LT) are under- sampled in BUFR (~200 m step); anti-aliasing low-pass filter is applied Possible modifications: - impact height instead of impact parameter - fixed impact height grid as the reference grid - discretization consistent with physical resolution of RO (wave optics) 50-100 m - BA error profile based on signal structure - different truncation heights based on signal structure and different approaches - option: providing all retrieved BA profiles down to surface and including truncation height info. in PCV

high resolution profile

simple decimation

decimation with anti-aliasing filtering

Output of COSMIC data (vectors) to BUFR files

CDAAC Near Term Plans (in next ~4-6 months)

•  Excess Phase processing -  Finalize and validate software

•  Neutral Atmospheric Products -  Finalize and validate NEW retrieval software -  Develop improved BA error profiles

•  Continue Re-processing efforts with NEW software –  COSMIC will be processed first

•  Work with IROWG NWP subgroup on any BUFR changes •  Provide test dataset of NEW products to NWPs •  Develop/publish MMCs •  Ionospheric Products

-  Publish Re-Analysis products -  Scintillation improvements (provide 10-sec average S4)

CDAAC Data Access •  New FTP Batch Delivery System

-  New hardware and ftp software

-  Increases download speed significantly

-  Online since Oct. 2, 2012

-  ~5 GB/user/day with unlimited file counts

-  When the job is done, user will get notification email with download links

-  Soon to add daily tar files for selected file types to speed up downloading

•  Other download methods -  GTS, Unidata/LDM, -  HTTP, Loaded HD -  GoogleMaps

http://cdaac-www.cosmic.ucar.edu/cdaac/DBif/cdaac_highlevel.cgi

Algorithm Documentation: http://cdaac-www.cosmic.ucar.edu/cdaac/doc/overview.html

CDAAC Support to the Community •  Supporting 2087 data users from 71

countries

•  Providing NRT data to NWPs, AFWA

•  Providing NRT, Post-Processed, and climate Re-analysis products from 9 RO missions

•  Development of improved RO data

processing algorithms

•  Providing data, user support, and science data processing software to community

Data Downloaded ~250TB (1/3 from university community)

Acknowledgments •  NSF •  Taiwan’s NSPO •  NASA/JPL, NOAA, USAF, ONR, NRL •  Broad Reach Engineering •  Other RO Missions, CHAMP, SAC-C, GRACE,

TerraSAR-X, C/NOFS, Metop-A/GRAS

Questions?