regional deterministic prediction system (rdps) technical … · regional deterministic prediction...

© Environment Canada, 2014

1

Regional Deterministic Prediction System (RDPS)

Technical Note

Last update:

November 18, 2014

from version 3.2.1 to version 4.0.0


2

Version Date Author Variation or revocation

1.0 01/10/2014 J.-F. Caron Creation of document

First version of content for Sections 1 to 6.1

2.0 30/10/2014 D. Anselmo Addition of text covering UMOS, SCRIBE, the

subjective evaluation, and dependent sub-systems.

3.0 17/11/2014 J.-F. Caron Some corrections

4.0 18/11/2014 D. Anselmo Final modifications before publishing.


3

Table of Contents Summary .......................................................................................................................................... 4

Nomenclature ................................................................................................................................... 5

1. Introduction .................................................................................................................................. 6

2. Forecast model configurations and cycling strategies .................................................................. 7

3. Changes to the data assimilation system ...................................................................................... 8

4. Changes to the forecast models .................................................................................................. 11

5. Objective evaluation of the final R&D test series ...................................................................... 11

6. Evaluation of the parallel run ..................................................................................................... 27

7. Performance of dependent systems ............................................................................................ 34

8. Availability of products .............................................................................................................. 36

9. Summary of the results ............................................................................................................... 36

10. Acknowledgements .................................................................................................................. 37

11. References ................................................................................................................................ 37


4

Changes to the Regional Deterministic Prediction System (RDPS)

from version 3.2.1 to version 4.0.0

Research, Development, and Operations Divisions at the Canadian Meteorological Centre,

Environment Canada

Summary

The modifications to the data assimilation component of the regional deterministic prediction

system (RDPS) implemented at Environment Canada operations during the fall of 2014 are

described. The main change is the replacement of the limited-area 4DVar data assimilation

algorithm for the limited-area analysis and the associated 3DVar scheme for the synchronous global

driver analysis by the 4D ensemble-variational (4DEnVar) scheme presented in the companion

TechNote (GDPS-TN) for the global deterministic prediction system (GDPS). Further forecast

improvements were also made possible by upgrades in the assimilated observational data and by

introducing the improved global analysis presented in GDPS-TN in the RDPS intermittent cycling

strategy. The computational savings brought about by the 4DEnVar approach are also discussed.


5

Nomenclature

3DVar: Three-dimensional variational data assimilation

4DVar: Four-dimensional variational data assimilation

3DEnVar: Three-dimensional ensemble-variational data assimilation

4DEnVar: Four-dimensional ensemble-variational data assimilation

AD: Adjoint

CMC: Canadian Meteorological Centre

EC: Environment Canada

ECMWF European Centre for Medium-Range Weather Forecasts

EnKF: Ensemble Kalman Filter

GB-GPS ZTD: Ground-based global positioning system zenith tropospheric delay

GDPS Global Deterministic Prediction System

GEM: Global Environmental Multi-scale model

ETS: Equitable threat score

LAM: Limited area model

LBC: Lateral Boundary Condition

RDPS: Regional Deterministic Prediction System

SCRIBE: EC’s software to assist in the preparation of forecast bulletins

TL: Tangent-linear

UMOS: Updateable Model Output Statistics


6

1. Introduction

In the companion technical note (Buehner et al. 2014; hereafter referred to as GDPS-TN), the latest

modifications to the global deterministic prediction system (GDPS) implemented operationally at

Environment Canada (EC) in the fall of 2014 were described. The most notable change is the

replacement of the four-dimensional variational data assimilation (4DVar) scheme by a four-

dimensional ensemble-variational (4DEnVar) scheme in which the background-error covariances

are represented by a blend of climatological covariances and 4D flow-dependent covariances

derived from a global ensemble Kalman filter (EnKF). In this document we report on the

implementation of the same 4DEnVar scheme in the regional deterministic prediction system

(RDPS). Further forecast improvements were obtained from changes to the assimilated

observational data and the introduction of the improved global analysis presented in GDPS-TN in

the RDPS intermittent cycling strategy.

Similarly to global data assimilation applications, considerable effort has been devoted in recent

years to both comparing and combining ensemble-based and variational-based limited-area data

assimilation approaches (Wang et al. 2008a,b; Zhang and Zhang 2012; Zhang et al. 2013; Liu and

Xiao 2013; Schwartz and Liu 2014; Gustafsson and Bojarova 2014, Pan et al. 2014). As reported by

Zhang and Zhang (2012), using 4DVar in combination with an ensemble-derived covariance matrix

(hybrid 4DVar) can outperform a stand-alone EnKF or a 4DVar in a limited-area forecasting system,

in agreement with the results from Buehner et al. (2010) in a global configuration. However, the

latter approach still requires the use of tangent-linear (TL) and adjoint (AD) versions of the forecast

model, whose time integrations dominate the cost of the analysis step and which requires, moreover,

significant development and maintenance effort. On the other hand, as explained in Buehner et al.

(2013; hereafter B13), 4DEnVar uses 4D ensemble covariances in a way that essentially replaces the

use of TL and AD versions of the forecast model and has been shown to produce forecasts with

similar or improved accuracy at short lead times in a global context by B13 and in GDPS-TN of this

paper. Recently Gustafsson and Bojarova (2014) reported that 4DEnVar can outperform both 4DVar

and hybrid 4DVar in the context of the limited-area HIRLAM forecasting system. Therefore, the

4DEnVar approach described in GDPS-TN seemed appropriate for the RDPS in order to 1) improve,

or at least maintain, the RDPS forecast accuracy obtained using the operational limited-area 4DVar

scheme and 2) make more efficient use of resources at EC by moving towards a more unified data

assimilation approach for deterministic and ensemble forecasting.

Since there is currently no operational equivalent to the global EnKF (Houtekamer et al. 2014) at the

regional scale at EC, we simply based our 4DEnVar scheme for the RDPS limited-area analysis on

the use of 4D ensemble covariances derived from the global EnKF as in the GDPS configuration

described in GDPS-TN. Our approach is thus similar to the National Centers for Environmental

Prediction (NCEP), which recently replaced the 3DVar scheme in the North American Mesoscale

(NAM) and the Rapid Refresh (RAP) regional forecasting systems by a 3DEnVar scheme based on

their global EnKF system (see NWS 2014a,b).


7

2. Forecast model configurations and cycling strategies

The forecast model configurations and the cycling strategy of the RDPS remain the same (except for

one bug correction in the models explained in section 4) in v.4.0.0. Nevertheless they are

summarized again here to facilitate the comprehension of this document.

The RDPS represents the main NWP guidance for forecasters of the Meteorological Service of

Canada for day one and two. It produces forecasts over North America (see again Fig. 1) up to T +

48 h (T + 54 h) at 00 and 12 UTC (06 and 18 UTC) using a limited-area (LAM) version of the GEM

model (Côté et al. 1998) on a cylindrical equidistant grid with a horizontal grid spacing of 0.09o

(approximately 10 km) and 80 vertical levels (model lid at 0.1 hPa, as in the GDPS). Unlike the

GDPS, the RDPS still uses a hydrostatic-pressure coordinate (defined in Charron et al. 2012) and a

discretization on a regular (unstaggered) vertical grid, due to compatibility issues between some

physical process parameterizations used in the RDPS and the log-hydrostatic pressure vertical

coordinate defined on a staggered grid proposed by Girard et al. (2014).

Figure 1. Map showing the domain of the limited-area model in the

RDPS using an approximate 10 km horizontal grid spacing.

This forecasting system employs an intermittent upper air cycling strategy (Fig. 2) where the

(approximately 25 km) analysis from the GDPS serves to initialize the 6 h LAM forecast before the

analysis time T (named LB in Fig. 2). This forecast serves as the background state for the analysis

step at time T (named LF in Fig. 2). In parallel, the same procedure is applied to a global GEM

driving model with a horizontal grid spacing of approximately 33 km. The resulting synchronous

global driving analysis (named DF in Fig. 2) and forecast, as proposed and tested by Fillion et al.

(2010; hereafter F10), allows the observations outside the LAM analysis domain to influence the

LAM forecasts through the lateral boundary conditions (LBCs).


8

Figure 2. Schematic of the RDPS intermittent upper air cycling

strategy showing its dependency on the GDPS. The distances refer to

the approximate horizontal grid spacing of the nonlinear models. See

text in section 2 for further details.

The driving model uses the same vertical levels as the LAM in order to minimize interpolations in

the prescription of the LBCs, whose formulation in the LAM version of GEM is based on Thomas et

al. (1998). In each forecast a digital filter procedure (Fillion et al. 1995) is applied to eliminate the

spurious gravity waves triggered by imbalances in the initial conditions. Interested readers are

referred to CMC (2012) and references therein for further details on the model configuration used in

the RDPS, especially regarding the parameterization of the physical processes.

Both the LAM and the driving component use the Interactions between Soil–Biosphere–Atmosphere

(ISBA) land surface scheme, which has its own data assimilation and cycling strategy (see Bélair et

al. 2003a,b for complete details). In the driver, the surface variables from the GDPS at T – 6 h are

used to initialize the DB forecast (see Fig. 2) and no update of these variables is made in DF at time

T (i.e., DF starts from 6 h forecast surface variables). The LAM component uses a continuous data

assimilation cycling strategy where surface temperatures and soil moisture contents are updated

every 24 h at 00 UTC. LAM forecasts (LB and LF) initialized at 06, 12, and 18 UTC simply rely on

the forecast surface variables from the most recent 00 UTC surface analysis.

3. Changes to the data assimilation system

3.1. From 4D/3DVar to 4DEnVar

The previously operational 4DVar scheme used to perform the LAM analysis is based on the

formulation described in T12 and is a temporal extension of the 3DVar scheme proposed by F10. It

operates with an incremental formulation (Courtier et al. 1994) to produce analysis increments on a


9

global 400 200 Gaussian grid (i.e. with a horizontal grid spacing of approximately 100 km). The

homogeneous and isotropic background-error covariances used at the start of the assimilation

window (T – 3 h) are defined with a global spherical-harmonics spectral representation, but the TL

and AD models are defined over a limited-area domain exceeding (by 30% in each direction) the

horizontal grid of the high-resolution (~10 km) nonlinear model (see Fig. 5 in T12). Because the TL

and AD model grid chosen is an exact sub-domain of the global Gaussian analysis grid, the

communication of information (i.e., analysis increments and adjoint sensitivities) is direct, with no

spatial interpolations required.

This configuration mimics the solution of a global 4DVar (but at a reduced computational cost) in

order to prevent distortions in the analysis increments near the lateral boundaries of the nonlinear

model and to optimize the analysis of the large scales in this continental-scale LAM. The TL and

AD models used here include the same dynamical processes as the high resolution forecast model,

but with only two physical processes: simplified vertical diffusion and simplified grid-scale

condensation. Only one outer loop with 45 iterations is performed, and the resulting global analysis

increments at low resolution (~100 km) are then interpolated to the high resolution LAM grid (~10

km) and added to the background state at T – 3 h. A 3 h nonlinear forecast is then needed to carry

this analysis to time T. See T12 and F10 for further details.

The global driver analysis, on the other hand, is obtained from a 3DVar system since a global 4DVar

system would require too much time to meet EC's operational constraints for RDPS products. The

analysis increments are also produced on a global 400 200 (~100 km) Gaussian grid, but using the

same background-error covariances employed by the global 4DVar in the previously operational

version of the GDPS (see Charron et al. 2012) and with all the observational data available over the

entire globe.

The 4DEnVar scheme tested and adopted in the RDPS is identical to the system implemented in the

GDPS and presented in section 2a of GDPS-TN and in B13. Using an analysis grid that matches the

horizontal grid spacing of the 4D background-error covariances obtained from the global EnKF (~66

km in section 3 and 4; ~50 km in the final configuration presented in section 5), it is possible to

obtain analysis increments at a higher resolution than the limited-area 4DVar scheme presented

above. Because no TL or AD model integrations are necessary in 4DEnVar, the computational cost

of performing a LAM analysis at a higher resolution using the ensemble covariances from the global

EnKF is still lower than the limited-area 4DVar scheme. Moreover, the computational efficiency of

the 4DEnVar makes it possible to use this approach for the global driver analysis. A comparison of

timings and computational resources between the different data assimilation approaches is presented

in section 5d.

As in GDPS-TN, the ensemble-derived background-error covariances are blended with the same

homogeneous and isotropic background-error covariances used in the GDPS for each analysis


10

(LAM and global driver) using the same vertical profile of weighting factors and the same spatial

localization design. Unlike the GDPS, the RDPS based on 4DEnVar still uses a cold start strategy

and a non-incremental digital filter initialization procedure in forecasts initialized at time T, simply

because the incremental analysis update and the physics recycling (warm start) capability now used

in the GDPS (see Section 2g in GDPS-TN) are not available in the older version of the GEM model

used in the LAM and the driver components. Therefore, it was decided to select the 4DEnVar

analysis increment valid at time T obtained after 50 (70) iterations in the LAM (driver) analysis,

which further reduces the computational cost compared to 4DVar, since no forecast step is needed to

carry the analysis from T – 3 h to T.

3.2. Changes to the observational data assimilated

The various improvements to the processing and the volume of data assimilated in the GDPS

described in section 2 of GDPS-TN were also implemented in the RDPS. In short, the most

important changes are: 1) coefficients from a revised satellite radiance bias correction scheme were

implemented1, 2) the processing and the assimilation of radiosonde and aircraft data were upgraded

(this includes introducing a bias correction strategy for aircraft temperature data and taking into

account the horizontal drift of radiosonde balloons as in Laroche and Sarazin, 2013), 3) the number

of assimilated channels for AIRS and IASI radiances was increased and, finally, 4) zenith

tropospheric delay (ZTD) data from ground-based GPS (GB-GPS) receivers (sensitive to

precipitable water) over North America are now assimilated. See Sections 2b to 2f and Tables 2 and

3 in GDPS-TN for complete details.

3.3. Computational resources

The forecast improvements in v.4.0.0 are also associated with a major reduction in the

computational cost of the analysis steps in the RDPS. Table 2 shows the average computational

resources of the analysis steps (LAM and global driver) for v.4.0.0 and v.3.2.1, where the

computational cost is defined in terms of the product between the number of CPUs used and the

elapsed wall-clock time. In the LAM component, the replacement of the 4DVar reduces the

computational cost by an order of magnitude due to significant reductions in both the required

number of CPUs (320 vs 2048) and in the wall clock time (7 min vs. 17 min). This is largely the

result of replacing the integration of the TL and AD models in 4DVar by the use of an ensemble of

nonlinear model states to estimate 4D background-error covariances over the assimilation time

window. However, it is important to stress that 4DEnVar analyses are generated using an updated

version of the variational analysis program, and benefit from improvements in the optimization of

the FORTRAN code, especially from a better usage of the Message Passage Interface (MPI)

protocol. Note that the computational cost of 4DVar does not include the 3 h nonlinear model

forecast step (3 min using 1024 CPUs) to carry the analysis increments from T – 3 h to the analysis

1 No bias correction coefficients are computed in the RDPS. The coefficients are simply taken from the GDPS.


11

time T, which is not needed in 4DEnVar. In the global driver, in order to maintain a wall clock time

similar to that in the 3DVar analysis in v.3.2.1 (i.e., 9 min), the 4DEnVar component in V.4.0.0

demands more CPUs (320 vs. 64) and therefore shows an increased cost (see again Table 2).

Nevertheless, the total cost of the two analyses in V.4.0.0 remains well below the total cost in

v.3.2.1.

TABLE 2. Comparison of averaged computational resources used for the

analysis steps of the RDPS in experiments v.4.0.0 and v.3.2.1 on an IBM Power

7 system. The wall clock time represent the elapsed time between the start of

the variational analysis program and the end of the writing of the analysis

increments. The cost is simply defined as the product of the wall clock time and

the number of cores used. The cost of the nonlinear forecast from T – 3 h to T in

4DVar is not included.

cores Wall clock (min) Cost (core min)

LAM

v.3.2.1 (4DVar) 2048 17 34816

v.4.0.0 (4DEnVar) 320 7 2240

Global driver

v.3.2.1 (3DVar) 64 9 576

v.4.0.0 (4DEnVar) 320 9 2880

4. Changes to the forecast models

As in the GDPS v.4.0.0, a unit conversion error in the formulation of the prognostic equations for

the snow canopy density was corrected in the model version used for the LAM and for the driving

forecasts in v.4.0.0, leading to an approximate doubling of the snow density seen by the models.

This correction in the model formulation had a significant impact on the near-surface temperature in

winter, as will be shown later in Section 5.3.

5. Objective evaluation of the final R&D test series

For each winter (February and March) and summer (July and August) period of 2011, a total of 118

48 h forecasts were initialized at 00 and 12 UTC. In order to measure the full impact of the changes,


12

the surface variable cycling strategy in the LAM (see Section 2.1) was activated in both the new

configuration (v.4.0.0) and in the control experiment (v.3.2.1). The only difference between the final

series and the parallel run performed during the Summer and Fall 2014,with the exception of the

time period, is that the assimilated observations were obtained using a data cutoff time of T + 9h

instead of the data cutoff time used at EC operations: T + 2h.

The results reported below show the impact on forecasts from all the updates in the RDPS. Readers

interested in the impact on forecasts from partial updates (e.g. replacing only 4DVar by 4DEnVar)

are referred to Caron et al. (2014).

5.1. Upper air verification

To be consistent with the assimilation of radiosonde data at their reported or estimated geographical

position during the balloon ascent in v.4.0.0, forecasts from both v.4.0.0 and v.3.2.1 where verified

against radiosonde data by taking into account the horizontal drift. However, as in Laroche and

Sarazin (2013), all the observations used for the verification were assumed to be valid at the

synoptic time (00 or 12 UTC) despite that reported and estimated observation times were taken into

account in the data assimilation step of v.4.0.0. This verification strategy is also used in Section 6.1.

Figure 3 and Figure 4 show the verification for the winter period while Figure 5 and Figure 6

show the results during the summer period. Forecasts from v.4.0.0 better match radiosonde

observations at most vertical levels compared to v.3.0.0 for both winter and summer periods.

Figure 7 and Figure 8 show the fit of the analyses to the observations. It can be seen that the

4DEnVar-based analyses are closer to radiosonde observations than the 4DVar-based analyses, but

this DOES NOT necessarily mean that 4DEnVar-based analysis are closer to the true state of the

atmosphere. Independent observations (observations that were not assimilated in either 4DEnVar or

4DVar) would be needed to estimate the accuracy of the analyses. However, based on the forecast

improvements observed above and below, we can infer that 4DEnVar-based analyses represent an

improved estimate of the true state of the atmosphere compared to 4DVar-based analyses.


13

Figure 3. Final series verification for 24 h forecasts from v.4.0.0 (red) and v.3.2.1 (blue) against

North American radiosondes. The standard deviation (solid lines) and bias (observation minus

forecast; dashed lines) are shown for zonal wind (UU; upper left panels), wind module (UV; upper

right panel), geopotential height (GZ; middle left panel), temperature (TT; middle right panel), and

dew point depression (ES; lower left panel). Scores are averaged over 118 winter cases in 2011.

Boxes on the left (right) of the figures indicate statistical significance levels for biases (standard

deviation). Red (blue) boxes indicate that the v.4.0.0 (v.3.0.0) experiment is better. No box indicates

that the null hypothesis (that statistics of the two samples are the same) cannot be rejected. The

radiosondes’ horizontal drift during balloon ascent was taken into account in order to be consistent

with the changes introduced in the data assimilated in v.4.0.0.


14

Figure 4. As in Fig. 3, but for 48 h forecasts.


15

Figure 5. As in Fig. 3, but for 118 summer cases in 2011.


16

Figure 6. As in Fig. 3, but for 118 summer cases in 2011 and for 48 h forecasts.


17

Figure 7. As in Fig. 3, but for 00 h forecasts (analyses).


18

Figure 8. As in Fig. 3, but for 118 summer cases in 2011 and for 00 h forecasts (analyses).


19

5.2. Ground-based GPS

Figure 9 shows the verification of precipitable water forecasts against the values derived from GB-

GPS ZTD observations. An improved fit can be seen at every lead time during the summer period.

These improvements are largely due to the addition of GB-GPS ZTD observations in the data

assimilated in both the RDPS and the GDPS in v.4.0.0 and are in agreement with the findings of

Macpherson et al. (2008). In the winter period, when precipitable water is very low, no change was

observed in the verification scores (not shown).

Figure 9. Final series verification of forecasts from v.4.0.0 (red) and v.3.2.1 (blue) against

precipitable water (mm) derived from GB-GPS ZTD observations over North America as a function

of lead times (h) for the summer period. The standard deviations (biases; forecast minus

observation) are shown as solid (dashed) lines. Scores are averaged over 118 cases and the error bars

represent a 90% confidence interval.


20

5.3. Surface verification

The forecasts were also verified against METAR (aviation routine weather report) and synoptic

observation data available operationally at EC as in Wilson and Vallée (2003) using the USTAT

verification package. Only reports from surface-based stations with an elevation difference with the

LAM topography of less than 100 meters were used, which led to a total of 1282 stations being

considered: 984 in Canada and 210 in the US. Therefore the verification scores presented here are

more representative of the forecast performance over Canada. Statistics were partitioned between

forecasts initialized at 00 UTC and 12 UTC in order to capture the different behavior during daytime

and nighttime regimes.

Figure 10 and Figure 11 show the verification for screen level temperature and dew point,

respectively, for the summer period while Figure 12 and 13 show the scores for the winter period.

While some improvements can be seen in summer, the largest improvements can be seen in winter

due to the bug correction in the prognostic equations of the snow canopy density in the model

formulation.

Precipitation accumulations were also verified using the equitable threat score (ETS; see e.g., Mason

2003) for 5 thresholds (≥0.5, ≥2, ≥5, ≥10, and ≥15 mm) of 12 h accumulations from synoptic

reports. The scores are reported in Figure 14 (for summer) and Figure 15 (for winter). Again, some

improvements in v.4.0.0 compared to v.3.2.1 can be seen at most of the forecast lead times in each

season, although many are barely statistically significant.


21

1.5m T – Summer

00 UTC 12 UTC K

K

Lead Time (hours) Lead Time (hours)

Figure 10. Final series verification for forecasts from v.4.0.0 (red) and v.3.2.1 (blue) against

temperature from surface-based stations over North America as a function of lead times (h) for

forecasts initialized at 00 UTC (12 UTC) in left (right) panel for the summer period. The standard

deviations (biases; forecast minus observation) are shown in the top (bottom) panel and were

obtained from an average over 59 cases. For each comparison, the difference between the two

experiments (red minus blue) is shown in the bottom part of the figure where the grey shaded area

represents a 90% confidence interval.

a) STDE b) STDE

a) Std Dev

c) Bias

b) Std Dev

d) Bias


22

1.5m Td – Summer

00 UTC 12 UTC K

K


Figure 11. As in Fig. 10, but for dew point temperature.

a) Std Dev

c) Bias

b) Std Dev

d) Bias


23

1.5m T – Winter

00 UTC 12 UTC K

K


Figure 12. As in Fig. 10, but for the winter period.

a) Std Dev

c) Bias

b) Std Dev

d) Bias


24

1.5m Td – Winter

00 UTC 12 UTC K

K


Figure 13. As in Fig. 10, but for dew point temperature and the winter period.

a) Std Dev

c) Bias

b) Std Dev

d) Bias


25

00 UTC 12 UTC

ET

S

ET

S

ET

S

ET

S

Figure 14. Final series verification for forecasts from v.4.0.0 (red) and v.3.2.1 (blue) against 12-h

precipitation accumulation measured in terms of the equitable threat score (ETS) as a function of

accumulation thresholds (greater or equal to; mm) for forecasts initialized at 00 UTC (12 UTC) in

left (right) panel for the summer period. Verifications for accumulation from 0h to +12h are shown

in top row; +12h to +24h in second row; +24h to +36h in third row, and +36h to +48 in bottom row.

The scores were obtained from an average over 59 cases. For each comparison, the difference

between the two experiments (red minus blue) is shown in the bottom part of the figure where the

grey shade represents a 90% confidence interval.

a) +00 to +12h

c) +12 to +24h

e) +24 to +36h

g) +36 to +48h

b) +00 to +12h

d) +12 to +24h

f) +24 to +36h

h) +36 to +48h


26

00 UTC 12 UTC

ET

S

ET

S

ET

S

ET

S

Figure 15. As in Fig. 14, but for the winter period.

a) +00 to +12h

c) +12 to +24h

e) +24 to +36h

g) +36 to +48h

b) +00 to +12h

d) +12 to +24h

f) +24 to +36h

h) +36 to +48h


27

6. Evaluation of the parallel run

The parallel run of the RDPS began on 15 July 2014 (although the complete set of sub-systems,

including UMOS and SCRIBE, only started on 24 July 2014) and lasted until November 2014 (more

than 3 months).

6.1. Objective evaluation

The objective verification was computed using 118 cases from the parallel run (as in the final series)

starting on 15 July 2014 00 UTC and ending on 11 September 2014 12 UTC.

Figure 16 and Figure 17 show the comparison with radiosonde observations for 24h and 48h

forecasts, respectively. The improvements are similar to the results from the final series (see Figures

5 and 6), except for the degradation of the geopotential height bias above 500 hPa, especially in the

24h forecasts (note that the same apparent degradation was also observed in the GDPS v.4.0.0; see

GDPS-TN). We remark that temperature measurements from radiosondes (and thus geopotential

height) can be significantly biased (see e.g. Sun et al. 2013) and that no bias correction is currently

applied in our systems. Therefore, it is difficult to draw any conclusion based on radiosonde

observations alone. On the other hand, as shown in the GDPS-TN, a comparison of the GDPS

forecasts with ECMWF analyses (in which radiosonde temperatures are bias corrected) did not

revealed any degradation in geopotential or temperature biases.

Figure 18 show the verification of precipitable water forecasts against the values derived from GB-

GPS ZTD observations. The improvements are similar to the results seen in the final series (see

Figure 7).

Figure 19 and Figure 20 show the verification for screen-level temperature and dew point,

respectively. Improvements can be seen, but not as much as in the final series (see Figures 8 and 9).

Figure 21 shows the precipitation accumulation verification in terms of ETS. Some improvements

can be seen for some thresholds at most lead times.


28

Figure 16. As in Fig. 3, but for 118 cases of the parallel run in summer 2014.


29

Figure 17. As in Fig. 3, but for 118 cases of the parallel run in summer 2014 and for 48 h forecasts.


30

Figure 18. As in Fig. 9, but for 118 cases of the parallel run in summer 2014.


31

1.5m T – Summer

00 UTC 12 UTC

K

K


Figure 19. As in Fig. 10. but for 118 cases of the parallel run in summer 2014.

a) STDE b) STDE

a) Std Dev

c) Bias

b) Std Dev

d) Bias


32

1.5m Td – Summer

00 UTC 12 UTC

K

K



a) Std Dev

c) Bias

b) Std Dev

d) Bias


33

00 UTC 12 UTC

ET

S

ET

S

ET

S

ET

S


a) +00 to +12h

c) +12 to +24h

e) +24 to +36h

g) +36 to +48h

b) +00 to +12h

d) +12 to +24h

f) +24 to +36h

h) +36 to +48h


34

6.2. Subjective evaluation

The output of the RDPS parallel run was evaluated by CMC operational meteorologists for most of

the summer and part of the autumn of 2014. Daily evaluations were performed by verifying

forecasts from RDPS v.4.0.0 and RDPS v.3.2.1 against the operational analysis at 00 and 12 UTC.

Forecasts with lead times of 12, 24, 36, and 48-hours were verified for mass fields (geopotential

height at 500 hPa and sea-level pressure) and for quantitative precipitation forecasts (QPF).

In the shorter lead-times, results showed small differences when it came to the verification of the

geopotential height at 500 hPa and sea-level pressure. The differences became more frequent at 36

hours and more significant at 48 hours in favour of RDPS v.4.0.0. At 48-hour lead time, the two

systems were judged to be equivalent nearly two-thirds of the time, either having the correct

solution or not. The remaining third of the time, the new RDPS was preferred over the operational

model by a small margin.

When considering the verification by region, a slight gain was noted for v.4.0.0 over the Pacific and

slightly more over central North America. It is over the Atlantic regions where we saw a more

significant gain with RDPS v.4.0.0. Where the operational RDPS and new RDPS were different, two

thirds of the cases were in favour of the v.4.0.0 solution. The timing and intensity of systems were

better handled over the Atlantic regions, which had a positive impact for QPF as well. The

subjective evaluation also showed slight gains for v.4.0.0 over the Arctic, which were more

pronounced at 500 hPa.

Results for the QPF showed a more pronounced signal in favour of RDPS v.4.0.0 over the Atlantic

region, whereas they were judged to be equivalent over central regions, and somewhat better over

the Pacific region.

In summary, the subjective assessment concluded that the two systems are essentially equivalent in

the first 24 hours, but RDPS v.4.0.0 performs better on day 2, especially in terms of QPF and mass

fields over regions of eastern Canada.

7. Performance of dependent systems

7.1. UMOS

Training of the UMOS system for RDPS v.4.0.0 was carried out using the hindcasts (final series of

forecasts) for the periods from February 1st to March 31

st, 2011 for the winter season and from July

1st to August 30

th, 2011 for the summer season. In addition, training for the summer season was

advanced using data from the parallel run.

Verifications performed in both the summer (July 24th

to September 30th

, 2014) and winter (March

3rd

to March 31st, 2011) seasons show that the performances of the UMOS-RDPS system are

globally maintained although some individual stations show greater or weaker performances. Such

variations are normal, and moreover, individual performances should generally improve with the

addition of more cases in the UMOS statistical database.


35

7.2. SCRIBE

Since the forecasts for days 1 and 2 in the SCRIBE system originate from the RDPS, the results

from the RDPS v.4.0.0 parallel run were verified using the SCRIBE verification system for the

period from August 1st to September 30, 2014. A statistical post-processing (UMOS) is performed

for cloudiness, temperature at 2 m, probability of precipitation, and the direction and speed of the

wind. Other weather elements are generated directly from model output.

Observing stations used by the SCRIBE verification system.

The following weather elements were verified for the 00, 06, 12, and 18Z model runs, for the 136

stations shown in the figure above, and for the 0-48 hour forecast period:

• Precipitation accumulation

• Cloudiness

• Probability of precipitation or PoP

• Maximum (daytime) and minimum (nighttime) temperatures

• Precipitation type

• Wind speed

• Wind direction

The verification results generally indicated a neutral outcome. However, there was a slight

improvement for the RDPS v.4.0.0 forecast accumulation for the 0 mm category (no precipitation),

as well as for the probability of precipitation, especially for the category ≤ 50%. Improvements were


36

also noticed in the prediction of wind speeds from 0 to 19 km/h and, to a lesser extent, from 20 to 39

km/h.

For the predicted precipitation types and wind direction, the results showed a slight degradation

compared to RDPS v.3.2.1. However, these differences are not considered to be significant.

7.3. Other

The impact of changes to the RDPS (from v.3.2.1 to v.4.0.0) on the various sub-systems which

depend on RDPS output to drive their forecasts and/or analyses was in most cases largely neutral.

This was true for the High-Resolution Deterministic Prediction System (HRDPS) – West window,

the Regional Deterministic Wave Prediction System (RDWPS), the Integrated Nowcasting System

(INCS), the Regional Air Quality Deterministic Prediction System (RAQDPS), and the Regional

Deterministic Air Quality Analysis (RDAQA).

For three other systems, in particular, 1) the Regional Deterministic Prediction Analysis (RDPA),

otherwise known as the Canadian Precipitation Analysis (CaPA), 2) the Regional Deterministic

Prediction System – Coupled to the Gulf of St. Lawrence (RDPS-CGSL), and 3) the Regional

Marine Prediction System for the Gulf of St. Lawrence (RMPS-GSL), major upgrades were tested in

parallel runs over the summer of 2014. Based on the results of these runs, new versions of these

systems were approved for implementation into operations at the same time as the implementation

of RDPS v.4.0.0. For further information concerning the changes made to each of these systems, as

well as the performance of the parallel runs, the reader is referred to the appropriate technical note.

The list of changes to operational systems at the CMC is available on the following web page:

http://collaboration.cmc.ec.gc.ca/cmc/cmoi/product_guide/docs/changes_e.html

8. Availability of products

With this update, there are no changes to the timing or offering of products.

9. Summary of the results

The updated version 4.0.0 of the RDPS provides improved forecast accuracy relative to the previous

system, version 3.2.1, based on results from two sets of two-month R&D data assimilation and

forecast experiments during 2011 and also the parallel run during the summer and fall of 2014. The

improvements were shown to result from the combined impact of numerous changes, notably: the

replacement of 4DVar by 4DEnVar, an improved treatment of radiosonde and aircraft observations,

an improved radiance bias correction procedure, and the assimilation of new radiance and GB-GPS

observations. Due to the replacement of 4DVar with 4DEnVar, the new system is more

computationally efficient and easier to parallelize, facilitating a doubling of the analysis increment

horizontal resolution, which would have been prohibitively expensive with the 4DVar-based system.

http://collaboration.cmc.ec.gc.ca/cmc/cmoi/product_guide/docs/changes_e.html


37

The RDPS adopted the same 4DEnVar configuration as the GDPS for the LAM analysis only

because there is currently no operational equivalent to the global EnKF at the regional scale at EC.

However, significant efforts have recently been devoted at EC to developing limited-area EnKF

systems, and a North American continental scale configuration is currently being tested for

operational implementation in the near future. There are also plans to increase the horizontal

resolution of the forecast model in the RDPS at the convection-permitting scale, i.e., with a grid

spacing of a few km. Our future work will thus be devoted to developing a limited-area 4DEnVar

scheme to improve the analyses and forecasts at small scales, where the importance of moist

processes and nonlinearities should give further advantage to the 4DEnVar approach relative to

4DVar.

10. Acknowledgements

The authors thank the large number of people in the research, development, and operations divisions

at the Canadian Meteorological Centre who made this project possible.

11. References

Bélair, S., L.-P. Crevier, J. Mailhot, B. Bilodeau, and Y. Delage, 2003a: Operational implementation

of the ISBA land surface scheme in the Canadian regional weather forecast model. Part I:

Warm season results. J. Hydromet., 4, 352-370.

Bélair, S., R. Brown, J. Mailhot, B. Bilodeau, and L.-P. Crevier, 2003b: Operational implementation

of the ISBA land surface scheme in the Canadian regional weather forecast model. Part

II: Cold season results. J. Hydromet., 4, 371-386.

Buehner, M., P. L. Houtekamer, C. Charette, H. L. Mitchell, and B. He, 2010b: Intercomparison of

variational data assimilation and the ensemble Kalman filter for global deterministic

NWP. Part II: One-month experiments with real observations. Mon. Wea. Rev., 138,

1567–1586.

Buehner, M., J. Morneau, and C Charette, 2013: Four-dimensional ensemble-variational data

assimilation for global deterministic weather prediction, Nonlin. Processes Geophys., 20,

669-682

Canadian Meteorological Center (CMC), 2013: Improvements to the Global Deterministic

Prediction System (from version 2.2.2 to 3.0.0), and related changes to the Regional

Prediction System (from version 3.0.0 to 3.1.0), Technical note of the Canadian

Meteorological Center. Available at:


38

http://collaboration.cmc.ec.gc.ca/cmc/cmoi/product_guide/docs/lib/op_systems/doc_opc

hanges/technote_gdps300_20130213_e.pdf

Canadian Meteorological Center (CMC), 2012: Improvements to the Regional Deterministic

Prediction System (RDPS) from version 2.0.0 to 3.0.0. Technical note of the Canadian

Meteorological Center. Available at:

http://collaboration.cmc.ec.gc.ca/cmc/cmoi/product_guide/docs/lib/op_systems/doc_opc

hanges/technote_rdps300_20121003_e.pdf

Caron, J.-F., T. Milewski, M. Buehner, L. Fillion, M. Reszka, S. Macpherson and J. St-James, 2014:

Implementation of deterministic weather forecasting systems based on ensemble-

variational data assimilation at Environment Canada. Part II: The regional system.

Submitted to Mon. Wea. Rev.

Charron, M., and Coauthors, 2012: The Stratospheric Extension of the Canadian Global

Deterministic Medium-Range Weather Forecasting System and Its Impact on

Tropospheric Forecasts. Mon. Wea. Rev., 140, 1924–1944.

Côté, J., S. Gravel, A. Méthot, A. Patoine, M. Roch, and A. Staniforth, 1998: The operational CMC-

MRB Global Environmental Multiscale (GEM) model. Part I: Design considerations and

formulation. Mon. Wea. Rev., 126, 1373–1395.

Courtier, P., J.-N. Thépaut, and A. Hollingsworth, 1994: A strategy for operational implementation

of 4D-Var using an incremental approach. Quart. J. Roy. Meteor. Soc., 120, 1367–1387.

Fillion, L., and Coauthors, 2010: The Canadian Regional Data Assimilation and Forecasting system.

Wea. Forecasting, 25, 1645–1669.

Fillion, L., H. L. Mitchell, H. Ritchie, and A. Staniforth, 1995: The impact of a digital filter

finalization technique in a global data assimilation system. Tellus,47A, 304–323.

Girard, C., and Coauthors, 2014: Staggered Vertical Discretization of the Canadian Environmental

Multiscale (GEM) Model Using a Coordinate of the Log-Hydrostatic-Pressure Type.

Mon. Wea. Rev., 142, 1183–1196.

Gustafsson, N. and Bojarova, J., 2014: Four-dimensional ensemble variational (4D-En-Var) data

assimilation for the HIgh Resolution Limited Area Model (HIRLAM), Nonlin. Processes

Geophys., 21, 745-762.


39

Houtekamer, P. L., Xingxiu Deng, Herschel L. Mitchell, Seung-Jong Baek, Normand Gagnon, 2014:

Higher Resolution in an Operational Ensemble Kalman Filter. Mon. Wea. Rev., 142,

1143–1162.

Laroche, S., and R. Sarrazin, 2013: Impact of Radiosonde Balloon Drift on Numerical Weather

Prediction and Verification. Wea. Forecasting, 28, 772–782.

Liu, C., and Q. Xiao, 2013: An ensemble-based four-dimensional variational data assimilation

scheme. Part III: Antarctic applications with the advanced WRF using real data. Mon.

Wea. Rev., 141, 2721–2739.

Macpherson, S. R., G. Deblonde, J. M. Aparicio, and B. Casati, 2008: Impact of NOAA Ground-

Based GPS Observations on the Canadian Regional Analysis and Forecast System. Mon.

Wea. Rev., 136, 2727–2746.

Mason, I. B., 2003: Binary events. Forecast Verification—A Practitioner’s Guide in Atmospheric

Science, I. T. Jolliffe and D. B. Stephenson, Eds., Wiley, 37–76.

National Weather Service (NWS), 2014a: Updates to the Rapid Refresh (RAP) Analysis and

Forecast System, NWS National Technical Implementation Notice 13-38. Available at:

http://www.nws.noaa.gov/os/notification/tin13-38rap_aab.htm

National Weather Service (NWS), 2014b: Changes to the North American Mesoscale (NAM)

Analysis and Forecast System, NWS National Technical Implementation Notice 14-29.

Available at: http://www.nws.noaa.gov/os/notification/tin14-29namcca.htm

Pan, Y., K. Zhu, M. Xue, X. Wang, M. Hu, S. G. Benjamin, S. S. Weygandt, and J. S. Whitaker,

2014: A GSI-Based Coupled EnSRF–En3DVar Hybrid Data Assimilation System for

the Operational Rapid Refresh Model: Tests at a Reduced Resolution. Mon. Wea. Rev.,

142, 3756–3780.

Schwartz, C. S., Z. Liu, 2014: Convection-Permitting Forecasts Initialized with Continuously

Cycling Limited-Area 3DVAR, Ensemble Kalman Filter, and “Hybrid” Variational–

Ensemble Data Assimilation Systems. Mon. Wea. Rev., 142, 716–738.

Sun, B., A. Reale, S. Schroeder, D. J. Seidel, and B. Ballish, 2013: Toward improved corrections for

radiation-induced biases in radiosonde temperature observations, J. Geophys. Res.

Atmos., 118, 4231–4243


40

Tanguay, M., L. Fillion, E. Lapalme, M. Lajoie, 2012: Four-Dimensional Variational Data

Assimilation for the Canadian Regional Deterministic Prediction System. Mon. Wea.

Rev., 140, 1517–1538.

Thomas, S. J., C. Girard, R. Benoit, M. Desgagné, and P. Pellerin, 1998: A new adiabatic kernel for

the MC2 model. Atmos.–Ocean, 36, 241–270.

Wang, X., D. M. Barker, C. Snyder, and T. M. Hamill, 2008a: A hybrid ETKF–3DVAR data

assimilation scheme for the WRF model. Part I: Observation system simulation

experiment. Mon. Wea. Rev., 136, 5116–5131.

Wang, X., D. M. Barker, C. Snyder, and T. M. Hamill, 2008b: A hybrid ETKF–3DVAR data

assimilation scheme for the WRF model. Part II: Real observation experiments. Mon.

Wea. Rev., 136, 5132–5147.

Wilson, L. J., and M. Vallée, 2003: The Canadian Updateable Model Output Statistics (UMOS)

System: Validation against Perfect Prog. Wea. Forecasting, 18, 288–302.

Zhang, F., M. Zhang, and J. Poterjoy, 2013: E3DVar: Coupling an Ensemble Kalman Filter with

Three-Dimensional Variational Data Assimilation in a Limited-Area Weather Prediction

Model and Comparison to E4DVar. Mon. Wea. Rev., 141, 900–917.

Zhang, M., and F. Zhang, 2012: E4DVar: Coupling an ensemble Kalman filter with four-

dimensional variational data assimilation in a limited-area weather prediction model.

Mon. Wea. Rev., 140, 587–600.

regional deterministic prediction system (rdps) technical … · regional deterministic prediction...

Documents