Analysis of Record

Geoff DiMego

5 May 2004

NCEP

where the nation’s climate and weather services begin

T O P I C S

• Chronological Review of Background

• EMC Concept for Analysis of Record

• Why Trust EMC?

• NCEP Data Access

• Anisotropy in 3DVAR

• Leveraging

• ROM COST$

CHRONOLOGY I

• Oct 2002 request from Glahn & Livesey for centrally produced “Analysis of Record” to verify NDFD (unofficial & unfunded)

• Request from Jack Hayes / Brad Colman for 1-page proposals for downscaling approaches: 1 25 year 2 km climo of sensible wx by downscaling 25 year

32 km NARR used as observed basis for MOS/ Neural Net development to downscale NARR & GR forecasts to 2 km

2 Local model with nudging

3 Anomaly techniques (Toth & Lord A & B)

CHRONOLOGY II

• Approach 1 was preferred by ISST, but it had long timeline of 2+ yrs and cost $$

• Brad Colman came to EMC based on intense desire for something to help forecasters with NDFD workload in the short term

• Led to idea of and implementation of DGEX (Downscaled GFS by Eta eXtension) which is essentially Approach 2 applied centrally (nudging component of approach will come when Eta is replaced by WRF)

Analysis of Record (AoR)

• Approach 1 to downscale NARR involved 4DDA which could be applied to AoR

• ISST made it clear that forecasters want an analysis to use in real-time as well as the best analysis for AoR to verify NDFD

• These and other possible uses/requirements (e.g. Surface Transportation or Local Modeling etc) were enough justification for EMC to start “thinking” about the problem

EMC’s AoR Concept

• Can’t just apply 2-D analysis (variational or otherwise) to surface data - we might have 10,000’s of mesonet/ surface obs, B U T we have millions of AoR grid points

• Need a 3-D forecast model to obtain temporally consistent solution dictated among observed data, terrain & lower boundary forcing and synoptic forcing

• Propose to apply tried & true NCEP 4-D data assimilation technique of forecast-analysis cycle at high resolution (~2 km) with cost cutting measures to make feasible in production

EMC’s AoR Concept

• NCEP’s 4DDA will (like the EDAS) use– Full complexity of NOAH Land-Surface Model – Assimilation of observed precipitation data to

ensure lower-boundary states are optimal

• NCEP will use WRF-NMM as assimilating model to efficiently include – Nonhydrostatic effects in the dynamics– Terrain following coordinate (hybrid sigma-

pressure replaces step-mountain eta)– Nudging (not in any of NCEP current models)

EMC’s AoR Concept

• AoR’s emphasis is on sensible weather elements• Focus AoR on surface & sensible weather where

we have majority of mesoscale observations• To save cost, reduce vertical resolution away

from surface (run with 20-30 levels instead of current 60 levels)

• To compensate for less vertical, nudge prediction away from sfc to an existing solution provided by operational North American Mesoscale run (currently 12 km Eta but 10 km WRF-NMM by late FY2005)

Why Should We Trust EMC

• Afterall, EMC dropped use of overland surface temperatures in Eta 3DVAR and North American Regional Reanalysis & got better Eta forecasts when initialized from global analysis!

• NCEP convinced poor performance of 3DVAR due to step-mountain Eta coordinate:– No terrain following surface– Impossible to restrict vertical influence of surface obs

• AoR to use WRF-NMM with a terrain following sigma-pressure hybrid vertical coordinate

• EMC already has adapted global Gridpoint Statistical Interpolation (GSI) to WRF-NMM

NCEP Has Total Data Access• Continuous data collection from all sources

– Radiosondes, dropsondes, pibals, Profilers, RASS, VAD– Surface land (SYNOPs, METARs, mesonets)– Surface marine (ships, fixed & drifting buoys, CMANs, XBTs)– Aircraft (ACARS, AMDAR, AIREP, RECCO)– Satellite cloud-drift winds (visible, microwave, moisture)– Satellite radiances (DOD/NOAA/NASA polar + geostationary)– GPS IPW, SSM/I precip, scatterometer ocean sfc wind speed– Level II 88D radial velocity, reflectivity & spectrum width

• Anything available locally via LDAD can and should be sent to TOC in Silver Spring, MD and on to NCEP – this only takes seconds - most of these data are already getting to NCEP via FSL.

Mesonet (non-AWS) Surface Ob Density

METAR Surface Ob Density

GPS IPW (Integrated Precipitable Water) Ob Density

Boundary Layer Profiler Ob Density

EMC Outreach and Leveraging

• EMC is partnering with Steve Lazarus and others who helped develop / adapt the ADAS to complex terrain at University of Utah

• NCEP’s 3DVAR / GSI is being adapted to use anisotropic covariance structures that follow the terrain and depend on atmospheric flow and stability.

• EMC leverages all the strengths of co-located Joint Center for Satellite Data Assimilation

Isotropic Error Correlation in ValleyPlotted Over Utah Topography

ob’s influence extends into mountains indiscriminately

Anisotropic Error Correlation in ValleyPlotted Over Utah Topography

ob’s influence restricted to areas of similar elevation

Anisotropic Error Correlation on Slope Plotted Over Utah Topography

ob’s influence restricted to areas of similar elevation

Anisotropic Error Correlation on Mt Top Plotted Over Utah Topography

ob’s influence restricted to areas of similar elevation

Anisotropic Error Correlation on Mt Top Plotted Over Utah Topography

ob’s influence restricted to areas of similar elevation

x

EMC Outreach and Leveraging

• EMC’s AoR will leverage efforts to use the newly available Level II 88D data

• NWS collection of all Level II 88Ds (NWS, DOD + FAA sites) arrives at NCEP in seconds

• EMC is partnering with – NSSL/CIMMS’s Qin Xu, J. Gong and L. Wang et

al. on quality control of 88D radial velocities– CIMMS/NSSL’s Kim Elmore, K. Howard, J.

Zhang et al. on quality control & analysis of 88D reflectivity

EMC’s AoR ROM Costs

• Completion of development and adaptation of an operational AoR code and bullet-proof setup to run within NCEP Operational Suite ($250K for 2 FTEs)

• AoR would take ~15% of current computer 24 hours a day (14 min for anal + 44 min for fcst) to do hourly CONUS AoR at 2.4 km with 6 hour delay (to get Stage III precip analysis) ($2.5M increment to current CCS recurring budget)

Real-time Analysis for forecasters

• Our analysis suggests this would cost essentially the same amount as the AoR (without the development costs of course)

• If NWS gets job of supporting the Department of Homeland Security, it is quite possible there will be enough increase in computer capacity to generate BOTH the real-time analysis and the AoR with support needed only for the development and maintenance personnel – stay tuned.

Background Slides

Scaling of CCS resource for analysis piece of AoR Current 12 km Eta 3DVAR for full North American domain

full 60 levels takes 10 minutes on 30% of CCS

• 2.4 km resolution = 125 more “work”

• Anisotropic = 1.1• 15% of CCS = 2

• 275

• CONUS domain = 1/6• ~3 levels = 1/20• No radiances = 1/1.5• 14 minutes = 1/1.4• Fewer iterations = 1.1• 1/277

Scaling of CCS resource for forecast piece of AoR Current 12 km Eta Model for full North American domain

60 hr forecast takes 66 minutes on 75% of CCS

• 2.4 km resolution = 125 more “work”

• WRF/Nonhydro = 1.2• 15% of CCS = 5• 44 minutes = 1.5• 1125

• CONUS domain = 1/6• 1 hr forecast = 1/60• Only 20 levels = 1/3

• 1/1080

Purpose of Downscaling

• Extend the information content of coarse model prediction fields to finer scales that reflect the influence of detailed local effects such as terrain and / or land-states

• Initialization of IFPS for generation of NDFD especially through day 8

• Analysis of Record needed to verify NDFD

Relevance to NDFD

• Immediate need of the NWS Field is for high res grids to initialize GFE/IFPS/NDFD especially at day 8:

• High resolution grids of at least 5 km with a preference for 2.5 km

• Grids with uniform content out to day 8 at least once per day (currently using MRF grids which will be replaced by 4/day GFS so demand may be for more than just once per day)

• NDFD parameters (sensible weather) but preferably full 3-D grids to populate GFE / IFPS in anticipation of improving SMART TOOLS

Extension to 8 Days: Background• Original request asked for a single 24 hour Eta-12

forecast cold-started from 6.5 or 7 day GFS forecast (would have major spin-up problems)

• Next considered an extension from 60 hr for small (1/6th) domain to be run in place of 06z extension (NCEP Dir ruled out option to change current suite)

• EMC offered to compress schedule by pushing primary Eta run from 0-60 hr to 0-84 hr (freeing up slot to do 84-192 hr) when computer was upgraded

• Colman reiterated immediate need of NWS field• EMC relented and proposed to compress schedule &

put in 84-192hr extension on current computer

Extension to 8 Days: Proposal• Eta-12 had been run in 2 pieces:

– Large block of machine to make 0-60 hr fcst– Small block of machine to make 60-84 hr extension

• Proposal (implemented 20 April):– Make first block bigger to make 84 hr fcst in same time

window as 60 hr is taking now – Use small block (no change in cpu resource) to run 4.5

day (84-192 hr) extension for small domain (1/4.5=2/9th)

• Four slots would be made available:– Run CONUS domain (2/9th) at 06z &18z extending the

00z & 12z GFS– Run OCONUS domains for AK (1/7th) [eventually

Hawaii (1/25th) & Puerto Rico (1/25th)] at 00z &12z extending the 18z & 06z GFS

Extension to 8 Days: Feasibility

• Eta extension will produce the desired effect of downscaling the GFS solution because the GFS synoptic scale forecast will dominate the Eta solution in the interior through the effects of the lateral boundary conditions especially for this small a domain and for this long of a prediction

• EMC’s Eric Rogers built and demonstrated this capability with interior results quite similar to GFS forecast used for lateral boundaries

Product Distribution• Jason Tuell said space for 8-day extension

available on new SBN channel being used to test transition to new DVBS together with GRIB2 forms of full Eta-12 and GFS

• This would satisfy all the major concerns over distribution and avoid need to deliver through regional WAN’s & backdoors

EMC’s Downscaling Approach 1.• Apply EMC’s AoR concept to 25-year North American

Regional Reanalysis (2-2.5 FTE+CPU)• Produce 25 years of 2.4 km sensible weather grids• Use with NARR & GR forecasts to produce MOS/NN

coefficients for use with NCEP model guidance to produce downscaled results for every gridpoint at every WFO (? FTE – primarily MDL)– Produce just NDFD variables (sensible weather)– Produce all 3-d variables for high res input to GFE and IFPS

which then allow forecaster to add value in producing the actual NDFD fields

• Need substantial CPU to downscale 32km NARR to 2.4km even if done just every 3 hours – might have to settle for 5 km initially

• NCEP backup machine or “research cluster” are possible sites with enough CPU

Toth/Lord Downscaling Strategy - A• Correct model bias (on model grid)

– Today’s forecast vs current model history (1-2 months)– Current ensemble mean vs Reanalysis climate mean

• Correct model spread (on model grid)– Today’s forecast vs current model history– Current ensemble spread vs Reanalysis climatological spread

• Apply corrections to all ensemble members• Result: forecast anomaly on model grid, corrected for

climatology• Calculate most probable anomaly from ensemble (error

weighted mean)• Given high resolution, gridded climatology for each

forecast element:– Add most probable anomaly to climatology for downscaled forecast element– Not guaranteed to be physically consistent (like model grids)– Forecast anomaly on model grid needs to be transmitted– High resolution climatology resident at WFOs

Toth/Lord Downscaling Strategy – B

• Bias correction directly on NDFD grid– High resolution information still needed from local

climatology

• Can be done locally or centrally– If locally, assumes WFOs receive all ensemble forecast

members

• Neural Network application– Input: ensemble forecasts, lat, lon, elevation, climatology etc– Output: bias corrected ensemble forecasts on NDFD grid– Penalty function: probabilistic measure (e.g. Brier Skill

Score)