verification of the wrf model for simulating heavy...

Atmospheric Research 135–136 (2014) 172–192

Contents lists available at ScienceDirect

Atmospheric Research

j ourna l homepage: www.e lsev ie r .com/ locate /atmos

Verification of the WRF model for simulating heavyprecipitation in Alberta

Clark Pennelly ⁎, Gerhard Reuter, Thomas FleschDepartment of Earth and Atmospheric Sciences, University of Alberta, Edmonton, Canada

a r t i c l e i n f o

⁎ Corresponding author at: 1-26 Earth Sciences BuildinEdmonton, Alberta, Canada, T6G 2E3. Tel.: +1 805 242 3

E-mail address: [email protected] (C. Pennelly

0169-8095/$ – see front matter © 2013 Elsevier B.V. Ahttp://dx.doi.org/10.1016/j.atmosres.2013.09.004

a b s t r a c t

Article history:Received 25 April 2013Received in revised form 19 August 2013Accepted 3 September 2013

The Weather Research and Forecasting (WRF) model was used to simulate precipitation forthree flooding events in Alberta, Canada. A detailed comparison was made between the48 hour spatial distribution of model rainfall and observations obtained from rainfall gauges.Verification was evaluated in terms of Probability of Detection, False Alarm Ratio, BIAS, andEquitable Threat scores from over 120 observation stations. Evaluation was also performedusing the root-mean-squared-error at each model grid box as well as integration over themajor river basins of Alberta. Simulations with 15 km grid resolution were compared usingfive different cumulus parameterization schemes: Explicit, Kain–Fritsch, Betts–Miller–Janjić,Grell–Dévényi and Grell 3D ensembles.The Kain–Fritsch and explicit cumulus parameterization schemes were found to be the mostaccurate when simulating precipitation across three summer events. Themodel simulations usingthe Kain–Fritsch scheme often overestimated precipitation, resulting in higher Probability ofDetection values. Combined with low False Alarm Ratio values, this typically yielded the highestEquitable Threat scores. Greater precipitation accuracy was generally observed when thehorizontal resolution of the model was increased to 6 km. Model simulations performed withoutusing a cumulus parameterization scheme (i.e. explicit precipitation only) performed with similaraccuracy as simulations using a cumulus parameterization scheme at 6 km resolution.

© 2013 Elsevier B.V. All rights reserved.

Keywords:Weather Research and Forecasting modelWRFPrecipitation simulationCumulus parameterization schemePrecipitation verification

1. Introduction

The Rocky Mountains form the Continental Divideextending some 2500 km from northern Canada to southernTexas. This mountain barrier strongly affects the weatherand precipitation for the province of Alberta, Canada. Theorographic effects are particularly evident during the sum-mer due to differential slope heating which gives rise toconvergence, triggering convective outbreaks (Smith andYau, 1987). The summer season can experience extremerainfall events associated with the passage of an upper aircutoff low and lee cyclogenesis over the Alberta FoothillsRegion (Reuter and Nguyen, 1993). The transport of watervapor to Alberta often occurs in moist warm conveyer belts

g, University of Alberta,146.).

ll rights reserved.

originating from the Gulf of Mexico (Brimelow and Reuter,2005). These extreme rainfall events can lead to flashflooding in southern Alberta.

In June 2005, extensive rainfall caused flooding in southernAlberta (Ou, 2008). Sixteen municipalities declared states ofemergency. Thousands of people were forced to leave theirhomes along the rivers. The floods claimed four casualties andthe estimated damage was 400 million Canadian dollars. Theprecipitation fell from four distinct storms with similar tracks.The dates and recorded maximum rainfall amounts were: 1–5June (140 mm), 5–9 June (248 mm), 16–19 June (152 mm),and 27–29 June (90 mm). This paper focuses on numericalsimulation of two of these extreme events: 5–9 June (Storm A)and 16–19 June (Storm B).

Storm A showed synoptic conditions that are typical forlarge Alberta rain storms (Ou, 2008). On 5 June 2005, anupper-air blocking high was stationed over Alberta. With anupper-air trough approaching from the west, a surface low

http://crossmark.crossref.org/dialog/?doi=10.1016/j.atmosres.2013.09.004&domain=pdf

http://dx.doi.org/10.1016/j.atmosres.2013.09.004

mailto:[email protected]

http://dx.doi.org/10.1016/j.atmosres.2013.09.004

http://www.sciencedirect.com/science/journal/01698095

173C. Pennelly et al. / Atmospheric Research 135–136 (2014) 172–192

pressure center developed over Montana, forming a trough oflow pressure extending into Alberta. A secondary low formedin this trough in southeastern Alberta late on 6 June. This lowmoved slowly to the northwest on 8 June, causing heavyprecipitation across southern Alberta. The most intenseprecipitation fell from 00 UTC 06 June to 12 UTC 08 June.The intense radar echoes were organized in a precipitationband that approached Alberta from the southwest pushingnortheastwards across the province. Heavy precipitation fellover the foothills of the Rocky Mountains, while lighterprecipitation occurred throughout the southern part of Alberta.The Oldman River basin received an average precipitationamount of 107 mmduring a 48 hour periodwhich started at 06UTC 06 June. The southeastern border between Alberta andSaskatchewan had precipitation amounts around 50 mm,considerably smaller than the accumulation over the OldmanRiver basin. The northern part of the domain, above 52°N,received relatively low precipitation amounts.

Storm B followed a common pattern for heavy rainfall overAlberta. A cutoff cold low supported a well developed surfacelow pressure center. The vertically stacked system slowlymoved northwards from Montana into southern Alberta.During the early stage, the system was quite convective andcontained lightning, hail and squall lines across southernAlberta. Storm B produced an observed 48 hour maximumrainfall accumulation of 152 mm at Springbank, about 25 kmnorthwest of Calgary, and it was estimated that an area of about50,000 km2 received ≥50 mm of rain (Ou, 2008).

A third modeling case (Storm C) was added to have anexample of a highly convective event. Storm C occurred on12–13 July 2010, withmaximum recorded rainfall of 110 mm.On 12 July 2010, the metropolitan city of Calgary suffered themost damaging hailstorm in Canada's recent history. Themaximum hail size was 4 cm in diameter, and damages wereassessed at 400 million Canadian dollars in insurance claims(Phillips, 2010). Storm C developed in the exit region of the250 mb jet in southern Alberta. The cold front alignednortheast to southwest, and produced numerous thunder-storms across central and southern Alberta, which causedsignificant damage. The Strathmore Radar recorded reflectiv-ity values above 55 dBZ passing over the metropolitan city ofCalgary, which indicated heavy precipitation with large hail(Smith, 2011). These large hail stones produced damage tostructures, vehicles, trees, and crops. This storm also pro-duced heavy precipitation over the North Saskatchewan Riverbasin, with an average of 47 mm of rainfall. While thisamount is far less than the precipitation which Storm A andStorm B produced for the river basin with the heaviestprecipitation, the North Saskatchewan River basin was thelargest basin we studied, and a lower precipitation valuewould be expected when sampled over a much larger area.

Hydrological models estimating water flow for riversin Alberta need a high spatial and temporal resolution ofprecipitation data. Rain gauge measurements alone do notprovide adequate resolution, particularly in the orographicregions of south west Alberta. Weather radar imagery canestimate rainfall rate, but not over mountainous terrainbecause ground clutter distorts radar echoes. In addition,radar images have limited forecast skill, as they cannot beproduced prior to the precipitation event. In recent yearsthere have been efforts to use precipitation estimates from

Numerical Weather Prediction (NWP) models as an input forhydrological models.

With the advances of computing power and data assimila-tion, it is possible to run NWP models as a tool for floodforecasters. An important issue is to assess the skillfulness ofthese models in predicting the spatial distribution of rainfall toobtain reliable estimates of the total watermass falling over thecatchment areas of the river systems. One of the standaloneNWPmodels used for mesoscale precipitation forecasting is theWeather Research and Forecasting model (WRF). Flesch andReuter (2012) usedWRF to simulate heavy precipitation eventsover Alberta and examined the role of the topography insimulating and organizing the precipitation. Specifically, theyperformed simulations using the actual topographic grid andother simulations with reduced mountain elevations. Theyconcluded that a reduction of mountain elevation decreasesmaximum precipitation by about 50% over the mountains andfoothills.

NWP models often use cumulus parameterization schemes(CPS) to mimic the effects of cumulus clouds which are notresolved as they are smaller than individual model grid cells.These schemes attempt to trigger the convection and modifythe temperature and moisture profiles within a model columnbased on the grid-scale (i.e. resolved) meteorological informa-tion. Some common cumulus parameterization schemes are:Betts and Miller (1986), Kain and Fritsch (1990), and Grell(1993). How cumulus parameterization schemes operate inNWPmodels is particularly important for hydrological applica-tions, because the total volume of rainwater is sensitive to thecumulus parameterization scheme (Wang and Seaman, 1997).Kerkhoven et al. (2006) compared different cumulus parame-terization schemes for an intense monsoon rainfall event inChina and Japan and found that the Grell scheme was the mostrobust, performingwell at different rainfall intensities. TheGrellscheme was also used by Litta et al. (2012) to simulate severestorms over east India using the WRF model.

The results of a NWP model can be quite dependent onthe spatial resolution of the numerical grid. Intuitively, onewould expect that simulations using the highest spatialresolution would provide the most accurate model simula-tion. Wang and Seaman (1997) and Done et al. (2004) indeedfound that a finer grid resolution yielded the most accurateresults, but Grubišić et al. (2005) and Roberts and Lean(2008) showed cases for which the finer grid spacing did notimprove simulation accuracy. Furthermore, the finer gridspacing requires significantly more computation time andresources when performing simulations.

The purpose of this paper is to simulate intense Albertasummer rainstorms with the emphasis on evaluating theskillfulness of the model to accurately predict the spatialdistribution of rainfall. A secondary objective is to determinethe optimum choice of cumulus parameterization schemesfor grid resolution of 15 km and 30 km. Furthermore, weinvestigate whether a fine grid resolution of 6 km yieldsmore accurate precipitation amounts. An inter-comparisonbetween model precipitation and rain gauge observationswill be performed on the model grid and also integratedacross the watershed basins. Three storms will be simulatedusing the Weather Research and Forecasting model. Themodel output will be examined for accuracy of locationand amounts of precipitation by comparing the simulated

174 C. Pennelly et al. / Atmospheric Research 135–136 (2014) 172–192

48-hour precipitation output with the observed 48-hourprecipitation amounts.

2. Methodology and model description

To test how well we can predict precipitation for heavyrainfall events in Alberta, we use the Weather Research andForecasting (WRF) model (Michalakes et al., 1999). WRFfeatures non-hydrostatic dynamics, multi-nest capability,and several physics options for boundary layer processes,radiation schemes, cloud microphysics, and cumulus param-eterization schemes. Fig. 1 shows our domain setup using aninter-active nested domain inside the parent domain. Theinner grid covers southern Alberta, and only the meteoro-logical information from the inner grid was used in thisstudy. Table 1 lists the simulations performed with the modelstart and end times for the three storms when simulatedusing different grid resolutions and cumulus parameteriza-tion schemes. The model simulations were initialized at 1200

Fig. 1. Numerical Weather Prediction domain setup using an outer grid resolutionanalysis was from Alberta land south of 54° N latitude inside the 15 km grid.

UTC from the North American Regional Reanalysis dataset(NARR, Mesinger et al., 2006) the day before the first day forwhich precipitation was to be evaluated, to allow for 18 h ofmodel spin-up. Boundary conditions for the outer domainwere updated every 3 h from the NARR dataset.

The WRF model was used in an off-the-shelf manner, usingthe Environmental Modelling System's (Rozumalski, 2006)default configuration. The default configuration of WRF usesthe Lin et al. (1983) bulk water microphysics scheme, the Noah(Skamarock et al., 2008) land surface scheme, and the YonseiUniversity planetary boundary layer scheme (Hong et al.,2006). The Lin et al. microphysics scheme resolves watervapor, cloud, and precipitation processes using 6 hydrome-teors: water vapor, cloud water, rain, cloud ice, snow, andgraupel. The Noah land surface model uses atmosphericinformation from the surface layer to provide heat andmoisturefluxes for 4 layers of soil. The Yonsei University planetaryboundary layer scheme is responsible for the vertical sub-gridfluxes due to eddy transport in the entire vertical column. The

of 45 km, and inner grid resolution of 15 km. All precipitation data used for

Table 1WRF simulation's initialization and end times for Storm A, Storm B, and Storm C, using different cumulus parameterization schemes and grid resolutions for thenested domain. The model simulation (e.g. A6) is the combination of the grid resolution (6 km) for the particular Storm (A). Model precipitation wasaccumulated for the nested domain starting from 06 UTC the day following initialization, and ended 48 h later.

Model simulation Initialization time End time Grid spacing(km)

Cumulus parameterization schemes used

A6 12 UTC 5 Jun 2005 9 UTC 8 Jun 2005 6 EX, KFA15 12 UTC 5 Jun 2005 9 UTC 8 Jun 2005 15 EX, KF, BMJ, GD, G3DA30 12 UTC 5 Jun 2005 9 UTC 8 Jun 2005 30 EX, KF, BMJ, GD, G3DB6 12 UTC 16 Jun 2005 9 UTC 19 Jun 2005 6 EX, KFB15 12 UTC 16 Jun 2005 9 UTC 19 Jun 2005 15 EX, KF, BMJ, GD, G3DC6 12 UTC 11 Jul 2010 9 UTC 14 Jul 2010 6 EX, KFC15 12 UTC 11 Jul 2010 9 UTC 14 Jul 2010 15 EX, KF, BMJ, GD, G3D


Yonsei University planetary boundary layer and Lin et al.microphysics schemes have been shown to be skillful whensimulating WRF precipitation events (e.g. Efstathiou et al.,2013). We selected the Advanced Research WRF dynamicmodel core to perform our simulations. The model has 45vertical pressure levels, with the top level at 50 mb. For moredetails of model choices, we refer to Flesch and Reuter (2012).We used a 3-1 nesting option between outer and innerdomains, as well as the same cumulus parameterizationscheme for both domains for a given model simulation in anattempt to minimize inconsistencies at the interface of thecomputation grids (Warner et al., 1997). Three different spatialresolutions were used for the inner domain (outer domain):6 km (18 km), 15 km (45 km) and 30 km (90 km). Other thanslightly different domain resolution and cumulus parameteri-zation schemes, the simulations with different spatial resolu-tions used identical configurations and initialization data.

2.1. Cumulus parameterization schemes

By releasing latent heat and transporting water vaporand sensible heat, cumulus clouds modify the vertical profileof the environment. This takes place from subsidence ofthe environmental air, induced by the convection of massupwards, as well as from detrainment of water substancefrom clouds (Ooyama, 1971). Deep convection results inwarming and drying of the environmental air as it is forced tosink (Yanai and Johnson, 1993). Shallow convection willmoisten and cool the environmental air from the detrain-ment of water vapor that evaporates.

Mesoscale models are unable to explicitly resolve convec-tive cloud formation when the grid resolution is coarser thanabout 10 km. Thus model simulations using grid resolutions of10 km or larger require parameterization of cumulus clouds.The goal of cumulus parameterization is to determine thecollective effects of cumulus clouds, rather than to resolvehow a single cloud affects the vertical profile (Arakawa andSchubert, 1974). Different cumulus parameterization schemeshave been developed based on different assumptions regardinghow convection is triggered and how intense and deep theresulting convection is. All cumulus schemes have a triggeringmechanism, which is the set of requirements for activationof the cumulus scheme. Cumulus schemes, once triggered,will change the vertical profile of the grid column, oftenby modifying the moisture and temperature values. Themodification will continue until the closure assumptions aremet, with these assumptions being a set of requirements to

deactivate the parameterized convection inside the modelcolumn. Each cumulus scheme has different trigger assump-tions, modification processes and closure assumptions, thoughdifferent cumulus schemes may have some similar aspects.

In this study we will perform some simulations withoutusing a cumulus parameterization scheme. We will termthese simulations explicit, identifying the simulations bythe notation EX. We will compare those simulations withsimulations using the Kain–Fritsch (KF), Betts–Miller–Janjić(BMJ), Grell–Devinji (GD) and Grell Three-Dimensional (G3D)cumulus parameterization schemes.

The KF scheme is a mass-flux parameterization schemewhich determines the strength of convection from convec-tive available potential energy (CAPE) when deep convectionis triggered (Kain and Fritsch, 1990). It is an extension of theearlier Fritsch–Chappell scheme (Fritsch and Chappell, 1980)that modulates updrafts/downdrafts as well as entrainmentand detrainment rates. The KF scheme triggers deep convec-tion when a mixed parcel has positive vertical velocity over adepth that exceeds a specified cloud depth, typically 3 to4 km (Kain, 2003). KF mixes the air due to convection as wellas related updrafts and downdrafts, and rigorously conservesmass, thermal energy, total moisture and momentum (Kainand Fritsch, 1993). Convection triggered using the KF schemewill eliminate at least 90% of the CAPE over a time periodwhich is typically between 0.5 and 1 h long (Kain et al.,2003), which is the closure assumption. The removal of CAPEis performed by rearranging the mass in a column usingthe updraft, downdraft and environmental mass fluxes. Byremoving CAPE from rearranging mass, convective precipi-tation is introduced into the model column.

The Betts–Miller–Janjić scheme is the extension of theBetts–Miller scheme (Betts and Miller, 1986). Similarly tothe KF scheme, BMJ is triggered when a parcel of air ascendsa certain distance, as well as having positive CAPE. TheBMJ scheme then adjusts the atmospheric temperature andmoisture profile towards a reference structure by usingdeep and shallow convection. The reference structures arepre-determined profiles of temperature and moisture insidethe cumulus scheme. Janjić (1994) introduced a cloud ef-ficiency factor into the Betts–Miller scheme to avoid spuriousdeep convection over warm oceans, a problem that happenedusing the older Betts–Miller scheme. Vaidya and Singh(2000) compared the previous Betts–Miller scheme to thatof the BMJ and found that the new changes made to the BMJscheme were shown to be more useful when forecastingprecipitation over land. Janjić (2000) also introduced a


variable relaxation time, as well as more moisture profiles.Gilliland and Rowe (2007) verified WRF cumulus parame-terization schemes at a high resolution and found that theBMJ scheme had difficulty producing precipitation undersome environments, such as a warm and dry environment.The BMJ scheme has been found favorable in other settingsand was used operationally in the WRF–NMM at NCEP.

The Grell–Dévényi (GD) scheme and the Grell threedimensional (G3D) scheme make use of ensemble parame-terization with different closure assumptions and parameters(Grell and Dévényi, 2002). The goal of the two Grell schemesis to determine the best configuration for an ensemble ofparameters and closure schemes to feed back into the NWPmodel to statistically arrive at more accurate amount ofprecipitation than using only a single cumulus scheme. Sincethe Grell schemes are ensemble in nature, they featurenumerous triggering mechanisms, adjustment processesand closure assumptions, many of which are found in otherparameterization schemes. From comparing to observedwork, an optimal mixture of the sub-ensembles can befound. While there are configurations for the Grell schemeswhich the user can change to fine tune model simulations,there was no attempt at using anything other than thedefault settings during this study. The G3D scheme wasdesigned to be suitable for grid sizes less than 10 km(Skamarock et al., 2008), in addition to coarser resolutions,whereas the GD scheme was not designed for resolutionsfiner than 10 km while simulated inside the WRF model.

2.2. Model verification

The observations used to assess precipitation wereweather station archived data available from EnvironmentCanada's Climate website (http://climate.weatheroffice.gc.ca/climateData/canada_e.html). Data were also provided byAlberta Agriculture and Rural Development AgroClimaticInformation Service (ACIS) at http://agriculture.alberta.ca/acis/ (March 2012). These two sources of observed precipi-tation measurements accounted for approximately 120 ob-servation stations used in this study. However, due to the lowdensity of observation stations in Alberta, this resulted in anobserved grid resolution of approximately 60 km. Automat-ically recorded precipitation observations can suffer fromunder-catchment during windy conditions (Colle et al.,2000), though there was no attempt to modify the observeddata. The observation station data were interpolated to gridpoints of the same resolution as the model simulation byusing the default Cressman (1959) function, Oacres, as partof the Grid Analysis and Display System (GrADS) mappingsoftware (http://www.iges.org/grads/). While the Cressmanfunction can suffer from producing more precipitation thanother interpolation methods (5.7%, Hewitson and Crane,2005), other studies have used the Cressman function withinthe GrADS software (e.g. Davolio et al., 2009) with success.

Evaluation of the simulated precipitation was performed intwo different ways — at the grid point and at the observationpoint.

2.2.1. Grid pointThe 48 hour simulated precipitation at each grid point

was compared to the corresponding observed grid point's

interpolated 48 hour precipitation amount, in order to findthe root-mean-squared-error (RMSE, Appendix A) at thatgrid point. The total RMSE was then found for each of theseven major river basins of southern Alberta (Fig. 2). Datawere only evaluated on grid points south of 54° N latitude.Analysis also determined the percent simulated precipitationcompared to observed precipitation for each river basin. Thisallowed for a spatial evaluation of each cumulus scheme interms of over-simulating or under-simulating the precipita-tion amounts, while the RMSE gave information regardinghow accurately WRF simulated the precipitation across theAlberta river basins.

2.2.2. Observation pointThe second method of evaluation compared the observed

amount of precipitation from over 120 observation stationswith the simulated precipitation at those observation stations.This method used known values for observed precipitation,while the grid-point method used interpolated values deter-mined by the Cressman function. Theobservation point analysiswas performed at three 48-hour precipitation accumulatedthresholds: above 10 mm, 25 mm, and 50 mm. The observeddata and the simulated data were evaluated to determine theProbability of Detection (POD), False Alarm Ratio (FAR), BIAS,and Equitable Threat (ET, Schaefer, 1990) for each thresholdacross the entire domain, by using a 2 × 2 contingency table(Wilks, 1995). POD determined the percentage of stationswhich WRF correctly simulated precipitation when precipita-tion was observed. FAR determined the percentage of stationswhichWRF incorrectly simulated precipitationwhen comparedto the total number of simulated precipitation events. BIASdetermined whether WRF simulated precipitation at morestations than observed, or fewer than observed. A BIAS value of1 would indicate that the same number of stations hadsimulated precipitation as were observed with precipitation,whereas a BIAS value of 2 would implyWRF simulated twice asmany stations with precipitation than were observed. ETdetermined overall skill when simulating precipitation andincludes a correction term which reduces the effect of correctprecipitation at an observation station by chance. An ET score of0 indicates the same accuracy as a random precipitationsimulation, and positive ET scores indicate some level ofskill, while a perfect precipitation simulation would haveET equal to 1. Appendix B contains formulas for the aboveprecipitation statistics. We focused primarily on the higherthreshold of precipitation, due to the flooding consequences ofhigh precipitation rates.

3. Storm A

3.1. Simulations with 15 km grid resolution

The WRF model was used to simulate Storm A with fivedifferent cumulus parameterization schemes. Fig. 3 comparesthe observed 48 hour rainfall amounts with the model'ssimulated precipitation using five different cumulus param-eterization schemes (EX, BMJ, KF, GD, G3D). All schemeswere skillful in reproducing the major features of the spatialdistribution of precipitation, yet there were differences in thespatial amounts of precipitation between simulations. TheEX scheme (Fig. 3b) produced the maximum precipitation

http://climate.weatheroffice.gc.ca/climateData/canada_e.html

http://climate.weatheroffice.gc.ca/climateData/canada_e.html

http://agriculture.alberta.ca/acis/

http://agriculture.alberta.ca/acis/

http://www.iges.org/grads/

Fig. 2. Major river basins of Alberta, Canada. The seven used in this study, ordered form the largest area to smallest, are the North Saskatchewan, Red Deer, Bow,Athabasca, Oldman, South Saskatchewan, and Milk. This figure was produced by data from the Department of Natural Resources Canada.


furthest south compared against the other cumulus schemes,as well as a distinct secondary location of precipitationnear the southeastern border to Saskatchewan. However, theEX scheme placed the zone of maximum precipitation(136 mm) incorrectly inside the river basin to the north,the Bow River, 75 km from the observed maximum. The BMJscheme (Fig. 3c) simulated more intense precipitation alongthe foothills, with a larger maximum precipitation amount of163 mm. However, the location was inside the Red DeerRiver basin, also 170 km from the observed maximum. TheKF scheme (Fig. 3d) produced more widespread precipitationthan the EX, with precipitation further north as well asheavier precipitation in the southeastern part of Alberta. Thezone of maximum precipitation (140 mm), when simulatedusing the KF scheme, was noticeably north compared to the

EX scheme, inside the Red Deer River basin, 170 km from theobserved maximum. The GD (Fig. 3e) and G3D (Fig. 3f)produced similar areas of precipitation, with their maximumprecipitation in close spatial agreement with the observedmaximum, though not as intense. These five cumulus schemesproduced most of their precipitation within a similar range oflatitudes, between 50°N and 53°N, and all schemes producedthe precipitation further northwhen compared to the observedprecipitation. However, the simulations must be comparedagainst the observed precipitation (Fig. 3a) which had amaximumprecipitation amount of 224 mmwithin theOldmanRiver basin.

Each cumulus parameterization scheme simulated differ-ent amounts of precipitation for the domain of study duringStorm A (Fig. 4). The BMJ and KF schemes simulated more

image of Fig.�2

49°N

51°N

52°N

53°N

54°N

50°N

110°W120°W 118°W 116°W 114°W 112°W 110°W120°W 118°W 116°W 114°W 112°W

49°N

51°N

52°N

53°N

54°N

50°N

49°N

51°N

52°N

53°N

54°N

50°N

(a) (b)

(c) (d)

(e) (f)

Precipitation accumulation [mm]

Fig. 3. 48 hour precipitation amounts for Storm A for (a) the observed precipitation interpolated to 15 km resolution, and the simulations using 15 km gridresolution with the following cumulus parameterization schemes: (b) EX, (c) BMJ, (d) KF, (e) GD, and (f) G3D.


domain averaged precipitation than what was observed,with BMJ simulating 122% of the observed volume and KFsimulating 132%. The KF scheme has been noted to over-simulate precipitation amounts from an overactive triggermechanism of the cumulus scheme (e.g. Colle et al., 2003;

Gochis et al., 2003; Liang et al., 2004; Wang and Seaman,1997). The EX scheme simulated the closest amount ofprecipitation at 101% of the observed amount. GD simulated106% while G3D simulated 109%. River basins with lowamounts of observed precipitation, such as the Athabasca

image of Fig.�3

EX BMJ KF GD G3D

700

600

500

400

300

200

100

0

0

10

20

30

40

50

60

70

80

90

RM

SE [

mm

]Pe

rcen

t sim

ulat

ed o

f ob

serv

ed [

%]

Athaba

sca

[4 m

m]

Bow R

iver

[46 m

m]

Red D

eer

[43 m

m]

N. Sas

katch

ewan

[8 m

m]

S. Sas

katch

ewan

[38 m

m]

Milk

Rive

r

[34 m

m]Oldm

an

[107

mm]

Domain

avera

ge

[30 m

m]

(a)

(b)

Fig. 4. Precipitation verification statistics, for Storm A, at each river basin, simulated using 15 km grid resolution. Percent simulated precipitation of observed(a) and RMSE values (b) for each cumulus scheme. Observed basin precipitation values are indicated below the basin's name in (b).



(4 mm) and North Saskatchewan (8 mm), received moresimulated precipitation than was observed. When assessedover the entire domain, all cumulus schemes either properlysimulated the amount of precipitation or simulated moreprecipitation than was observed.

The domain average analysis (Table 2) shows that EX hadthe lowest RMSE at 29 mm, with KF the next closest at32 mm. BMJ had the highest error with 39 mm. KF simulated32% more precipitation than was observed, yet had lowerRMSE values than some cumulus schemes which bettersimulated the total amount of precipitation. Thus, KF schemewas able to simulate precipitation more accurately comparedto some of the other cumulus schemes, though the EXscheme was more accurate than KF when analyzed across thegrid points.

The Oldman River received the most precipitation duringthis storm, at 107 mm when averaged across all grid pointsinside the river basin. While simulating only 58% of theobserved basin precipitation (Table 3), KF simulated moreprecipitation than any other cumulus scheme for this riverbasin. The next closest was EX at 42%. KF had the lowestRMSE value (60 mm), with EX the next closest at 73 mm. GD,G3D, and BMJ performed similarly for this river basin, with30%, 29% and 29% respective simulated precipitation, whencompared to the total observed precipitation over the riverbasin. Their RMSE values were also similar; GD with 83 mm,G3D with 84 mm and BMJ with 88 mm. The KF cumulusparameterization scheme simulated this river basin moreaccurately than the remaining cumulus schemes, by havingthe lowest RMSE value.

Table 4 shows the results from the observation pointanalysis for each cumulus scheme with the three differentprecipitation accumulation thresholds. The table shows similarresults as the grid point analysis above. Due to KF and BMJ bothsimulating more precipitation than observed, both of theseschemes would be expected to have higher POD and BIASvalues than the other schemes, which is observed in Table 4. Itwould also be expected that KF andBMJwould have higher FARvalues than other schemes, due to simulating more precipita-tion than being observed, and thus falsely simulating moreprecipitation as well. This was the case for BMJ, but not for KF.KF was able to maintain a low FAR, at all thresholds, whilesimulating 32% more precipitation than was observed. ThoughKF had high POD (0.77) and low FAR (0.41) at the highestprecipitation threshold, the EX scheme had a slightly higher ETscore (0.39) than KF (0.38) due to EX's low FAR (0.32); thus EXwas slightly more accurate when simulating the precipitation

Table 2Domain averaged verification results for Storm A, Storm B, and Storm C, when simulais identified in column OBS. Bold values indicate the two most accurate cumulus sc

Storm Statistic OBS

A15 Average precipitation (mm) 30RMSE (mm) –

Percent simulated of observed (%) –

B15 Average precipitation (mm) 36RMSE (mm) –


C15 Average precipitation (mm) 31RMSE (mm) –


from this storm when analyzed across the observation points.The BMJ, GD, and G3D schemes all had similar ET scores at 0.23,0.21 and 0.23 respectively, at the highest precipitationthreshold. The observation point analysis using POD, FAR,BIAS and ET shows that EX and KF were able to simulate thisstorm more accurately than the other cumulus schemes.

Table 4 illustrates some general results when precipita-tion is analyzed using different precipitation thresholds. ETscores were generally higher at the 25 mm threshold for allschemes than at both the 10 mm and 50 mm thresholds. Thisis consistent with other studies (e.g. Cherubini et al., 2002),though Colle et al. (1999, 2000) had the highest ET values atthe lowest threshold when simulating precipitation over thePacific Northwest. In addition, cumulus schemes generallyhad lower POD and higher FAR at higher thresholds whichlead to lower ET scores. Other studies have found thatnumerical weather prediction models have higher FAR andlower POD and ET scores at higher precipitation thresholds(e.g. McBride and Ebert, 2000), indicating that heavyprecipitation may be more difficult for models to accuratelysimulate than moderate or light precipitation.


Fig. 5 shows the spatial distribution of the 48 hoursimulated precipitation amounts using the KF scheme atdifferent grid resolutions. The 30 km resolution (Fig. 5d) hadskill when simulating this storm, though comparing with theobservations (Fig. 5a), it is clear that the 30 km simulation islacking areas of heavy precipitation. Fig. 6a shows thepercentage of simulated precipitation compared to observedprecipitation. At 30 km resolution, each cumulus schemesimulated more precipitation across the domain than wasobserved. The values range from the EX scheme with theclosest value at 116% of observed, to KF with 137% ofobserved (Table 5). No scheme showed improvement whensimulating the total amount of precipitation over the riverbasins when simulated using a coarse resolution of 30 kmcompared to a grid resolution of 15 km.

The A30-EX simulation had a lower RMSE value (Table 5)than the A15-EX simulation (Table 2) when averaged acrossthe domain,while the remaining schemes had higher errors forthe lower resolution simulation. KF and BMJ had marginallyhigher errors at 30 km resolution than for 15 km resolution,but were lower in the magnitude of difference than GD andG3D. Generally, the cumulus schemes are more accurate at thegrid resolution of 15 km rather than 30 km when evaluated

ted at 15 km grid resolution. Averaged observed precipitation for each stormhemes for the given analysis.

EX BMJ KF GD G3D

30 37 39 32 3329 39 32 34 34

101 122 132 106 10935 47 44 43 4230 25 23 21 2297 129 122 119 11630 35 34 22 2420 23 23 23 2397 115 109 73 78

Table 3Verification results for Storm A, Storm B, and Storm C, when simulated at 15 km grid resolution, for the river basin which received the most precipitation for thegiven storm. Storm A was over the Oldman, Storm B over the Red Deer, and Storm C over the North Saskatchewan. Averaged observed precipitation for each riverbasin is identified in column OBS. Bold values indicate the two most accurate cumulus option for the given analysis.

Storm Statistic OBS EX BMJ KF GD G3D

Storm A15: Oldman River Average precipitation (mm) 107 45 31 61 33 31RMSE (mm) – 73 88 60 83 84Percent simulated of observed (%) – 42 29 58 30 29

Storm B15: Red Deer Average precipitation (mm) 56 40 69 66 57 49RMSE (mm) – 34 29 25 22 26Percent simulated of observed (%) – 72 123 118 102 87

Storm C15: North Saskatchewan Average precipitation (mm) 47 47 55 48 28 31RMSE (mm) – 24 29 26 30 30Percent simulated of observed (%) – 100 118 102 61 67


using the grid-point analysis. In addition, the 15 km simula-tions better simulated the domain averaged precipitationamounts, whereas the 30 km simulations all over-simulatedthe precipitation.

The Oldman River basin had the heaviest precipitation forthis storm, and EX had simulated this basin with accuracy at15 km resolution. When analyzed across the Oldman Riverbasin, the EX and BMJ schemes were slightly more accuratewhen simulated at a resolution of 30 km, with lower RMSEvalues (Table 5), than for a resolution of 15 km (Table 3).However the remaining three cumulus parameterizationschemes (KF, GD, G3D) had less accuracy when simulatedat 30 km resolution across the Oldman River basin.

There were differences between the POD, FAR, BIAS, andET when simulating Storm A at 15 km (Table 4) and at 30 kmresolution (Table 5). The EX scheme was the only simulationat 30 km resolution to have a higher POD (0.73) than for thesame simulation using 15 km grid resolution (0.63). This mayhave been due to the larger BIAS value for the 30 kmsimulation (1.37) than at 15 km (0.93). An increase in BIASindicates more simulated precipitation which could increasethe POD value if the precipitation is simulated where it is alsoobserved. However, other schemes also experienced a largerBIAS at 30 km resolution without increasing the POD; G3Dhad a BIAS of 0.87 at 15 km resolution and a BIAS of 1.43 at30 km resolution, though experienced a decrease in POD. Thehigher BIAS values also caused much greater FAR values atthe grid resolution of 30 km than at 15 km for all schemes.Generally, the 30 km grid resolution simulations had lower

Table 4Precipitation scores for Storm A when simulated at 15 km grid resolution,for three precipitation thresholds. Bold values indicate the two mostaccurate cumulus schemes for the given analysis.

Threshold (mm) EX BMJ KF GD G3D

POD 10 0.90 0.93 0.95 0.86 0.8125 0.77 0.80 0.88 0.71 0.7350 0.63 0.53 0.77 0.53 0.47

FAR 10 0.27 0.35 0.28 0.31 0.3425 0.23 0.35 0.32 0.38 0.3950 0.32 0.50 0.41 0.53 0.46

BIAS 10 1.25 1.42 1.32 1.25 1.2325 1.00 1.23 1.29 1.14 1.2050 0.93 1.07 1.30 1.13 0.87

ET 10 0.34 0.22 0.35 0.26 0.1825 0.43 0.32 0.39 0.25 0.2450 0.39 0.23 0.38 0.21 0.23

POD and ET, and higher BIAS, FAR, and RMSE than the samesimulation using a grid resolution of 15 km.


At 6 km resolution, most storm dynamics are resolvableand there is no need to invoke a cumulus parameterizationscheme. Nevertheless, we thought that in addition to the EXsimulation run, we would also make a 6 km resolution runusing the KF scheme. Fig. 5 compares the 48 hour precipita-tion amounts for different resolutions. While WRF simulatedthis storm at 6 km resolution successfully (Fig. 5b), it is notobvious whether the refinement of resolution yields moreaccurate precipitation coverage. The EX scheme had a muchlarger change in simulated precipitation than the KF schemewhen the grid resolution was changed from 15 km to 6 km.A6-EX (Table 6) simulated 118% of the observed precipita-tion, while A15-EX (Table 2) simulated 101%. A6-KF simu-lated 133% of the observed precipitation, while A15-KFsimulated 132%. For Storm A, increasing the resolutionresulted with a simulation with more precipitation, and theEX scheme was more accurate than the KF scheme whensimulating the amount of precipitation across the grid pointsinside the domain.

The high resolution 6 km simulations had higher errorsthan the 15 km simulations. The domain averaged RMSE forA6-EX was 35 mm (Table 6), compared to A15-EX with29 mm (Table 2). The KF simulations show similar behavior.Higher resolution simulations often suffer from larger errorswhen evaluated at the individual grid points (Mass et al.,2002). This is because higher resolution corresponds tosmaller grid points, and small displacement differences inthe precipitation field can yield a larger error than comparedto a coarser resolution.

For the observation point analysis, A6-EX had lower PODat 6 km (0.57, Table 6) than at 15 km resolution (0.63,Table 4) and higher FAR at 6 km (0.47) than 15 km (0.32).This led to a lower ET score at 6 km resolution (0.26) than15 km (0.39). The KF simulation experienced similar changesbetween the resolutions, though was less accurate than EX,with higher RMSE, FAR, and BIAS. While KF had higher PODthan EX, the KF scheme had a lower ET value at 6 kmresolution. This storm was simulated with less accuracy at6 km resolution for both cumulus schemes than at 15 kmresolution, though EX simulated this storm with higheraccuracy than KF using 6 km grid resolution. Among the

Precipitation accumulation [mm]110°W120°W 118°W 116°W 114°W 112°W 110°W120°W 118°W 116°W 114°W 112°W

49°N

51°N

52°N

53°N

54°N

50°N

49°N

51°N

52°N

53°N

54°N

50°N

(a) (b)

(c) (d)

Fig. 5. 48 hour precipitation amounts for Storm A when simulated at three different grid resolutions using the Kain–Fritsch cumulus parameterization scheme.The observed precipitation was interpolated to 15 km resolution (a), and simulations had the following resolutions: (b) 6 km, (c) 15 km, and (d) 30 km.


three different grid resolutions and five cumulus parameter-ization schemes, this storm was best simulated at 15 kmresolution by using the EX scheme, with the KF scheme beingslightly less accurate at a 15 km resolution.

4. Comparison of the three storm events

All three storm events were simulated with the WRFmodel using five different cumulus parameterizationschemes at a 15 km grid resolution. The most intense rainfallfrom Storm B (Fig. 7) occurred over the Red Deer River basin,with a basin-averaged amount of 56 mm. This is about halfthe rainfall observed for Storm A, which averaged 107 mmacross the Oldman River basin, though the Red Deer Riverbasin is approximately twice as large as the Oldman Riverbasin. Storm B had a larger amount of precipitation averagedover the entire domain (36 mm) than Storm A (30 mm), forthe same length of time, 48 h. As for Storm A, with theexception of the EX scheme (97% of observed), each cumulusparameterization scheme simulated Storm B with moreprecipitation that was observed (Fig. 8). For Storm B, the EX

and BMJ schemes had the largest domain averaged RMSE, at30 mm and 25 mm (Table 2). The RMSE values for Storm Bwere generally lower than for Storm A. The EX schemecorrectly simulated the amount of precipitation but was notable to simulate the precipitation in the correct location,resulting with the highest (i.e. poorest) RMSE values, andwas the least accurate scheme to simulate Storm B whenanalyzed across the grid points. BMJ, which simulated 29%more precipitation than was observed, was slightly moreaccurate than EX. KF, G3D, and GD all performed similarly toone another; GD had the lowest RMSE value (21 mm), whileG3D simulated a more correct amount of precipitationbetween these three schemes. The RMSE values for KF(23 mm) and G3D (22 mm) were close to each other invalue, and lower than EX and BMJ. KF simulated the mostprecipitation of these three schemes, but had errors approx-imately the same as G3D. The GD scheme simulated thisstorm the most accurately when analyzed using the gridpoint method over the entire domain, with the lowest RMSEand a relatively correct amount of precipitation (119%)compared against the observed amount.

0

20

40

60

80

100

0

100

200

300

400

500

600

700

800

900

1000

1100

1200

Perc

ent s

imul

ated

of

obse

rved

[%

]

Athaba

sca

[2 m

m]

Bow R

iver

[77 m

m]

Red D

eer

[35 m

m]

N. Sas

katch

ewan

[7 m

m]

S. Sas

katch

ewan

[39 m

m]

Milk

Rive

r

[37 m

m]Oldm

an

[112 m

m]Dom

ain

avera

ge

[30 m

m]

(a)

(b)

EX BMJ KF GD G3D

RM

SE [

mm

]

Fig. 6. Precipitation verification statistics, for Storm A, at each river basin, when simulated using 30 km grid resolution. Percent simulated precipitationof observed (a) and RMSE values (b) for each cumulus scheme across all river basins. Observed basin precipitation values are indicated below the basin'sname in (b).


Table 5Verification results for different cumulus parameterization schemes when simulating Storm A at 30 km grid resolution. Observed domain and river basin valuesare identified in column OBS. Domain POD, FAR, BIAS, and ET values were calculated at a precipitation threshold of 50 mm. Bold values indicate an increase inaccuracy from the 15 km resolution simulation of the same storm.

Statistic OBS EX BMJ KF GD G3D

Domain averaged precipitation accumulation (mm) 30 33 37 39 36 38Domain averaged precipitation RMSE (mm) – 27 41 37 44 46Domain averaged percent simulated precipitation of observed (%) – 116 129 137 127 133Oldman River basin precipitation (mm) 107 60 32 57 20 21Oldman River basin RMSE (mm) – 62 86 63 98 97Oldman River basin percent simulated precipitation of observed (%) – 54 29 51 18 19Probability of Detection (POD) – 0.73 0.43 0.67 0.33 0.30False Alarm Ratio (FAR) – 0.46 0.66 0.57 0.76 0.79BIAS – 1.37 1.27 1.57 1.37 1.43Equitable Threat (ET) – 0.32 0.10 0.20 0.02 −0.01


The Red Deer River basin received the greatest amount ofprecipitation during Storm B (Table 3). GD closely simulatedthe amount of observed precipitation at 102% and had thelowest RMSE for this basin (22 mm). EX has the highestRMSE (34 mm), further indicating that EX simulated thisstormwith lower accuracy than the other schemes. For StormB, the observation point precipitation analysis shows that KFand GD simulated this storm the most accurately. Table 7shows the results for each scheme, at the highest precipita-tion threshold. KF had the highest POD at the 50 mmthreshold (0.92), and also the lowest FAR (0.49) comparedto the other schemes. Even with a high BIAS (1.81), KF hadthe highest ET score of 0.37 and was the most accuratescheme when evaluated using the observation point analysis.

Storm C (Fig. 9) had the largest difference in precipitationwhen simulated using the five cumulus schemes whencompared against Storm A and Storm B. Storm C wasconvectively unstable, with nearly 400 J/kg of ConvectiveAvailable Potential Energy (CAPE) determined from the 1200UTC 12 July 2010 Stony Plain sounding. The large differencesin precipitation suggest that days with a high CAPE value maybe more sensitive to different cumulus parameterizationschemes. Since each cumulus parameterization scheme usesdifferent triggering mechanisms as well as closure assump-tions, environments which favor convective days may besimulated very differently by different cumulus parameter-ization schemes. From the domain average values (Fig. 10),both the EX and KF schemes simulated the precipitation closeto the observed amount (97% and 109%). EX averaged thelowest RMSE of 20 mm (Table 2), while the other four

Table 6Verification results for different cumulus parameterization schemes when simulatinET values were calculated at a precipitation threshold of 50 mm. Bold values indicastorm and cumulus scheme.

Statistic A6EX

Domain averaged precipitation accumulation (mm) 38Domain averaged precipitation RMSE (mm) 35Domain averaged percent simulated precipitation of observed (%) 118Probability of Detection (POD) 0.57False Alarm Ratio (FAR) 0.47BIAS 1.07Equitable Threat (ET) 0.26

cumulus schemes averaged 23 mm. Storm C had a similaramount of precipitation when averaged across all grid pointsinside the domain (31 mm) as Storm A (30 mm), whileStorm C had lower RMSE values when analyzed across thedomain. The EX scheme simulated this storm the mostaccurately when analyzed using the grid point analysis acrossthe domain of study, with the KF and BMJ schemes beingslightly less accurate. The North Saskatchewan River basinreceived the most precipitation during Storm C, with anaverage of 47 mm of precipitation across all grid points insidethe river basin (Table 3). EX simulated 100% of the observedprecipitation, and had a RMSE value of 24 mm. With lowRMSE and accurate simulation of the accumulated precipita-tion, the North Saskatchewan River basin was simulated themost accurately using the EX scheme, with the KF havingslightly less accuracy using the grid point analysis.

Storm C was analyzed for accuracy by using the observa-tion point analysis for a precipitation threshold of 50 mm(Table 7). The BMJ scheme had the greatest POD value (0.57),with KF and EX the next closest at 0.43. The GD and G3D hadvery low BIAS scores, indicating very little simulatedprecipitation above 50 mm, which resulted in low POD andhigh FAR scores. The observation point analysis shows thatthe EX, BMJ, and KF scheme all simulated this storm withsimilar ET scores, and thus, skill.

Numerical simulations of Storm C revealed some weak-ness in capturing the spatial distribution of the observedrainfall. The POD values for Storm C were relatively low incomparison to Storm A and Storm B. This led to cumulusschemes generally having lower ET scores for Storm C than

g Storm A, Storm B, and Storm C at 6 km grid resolution. POD, FAR, BIAS, andte an increase in accuracy from the 15 km resolution simulation of the same

A6KF B6EX B6KF C6EX C6KF

42 40 43 38 3341 25 22 25 26

133 103 110 115 990.67 0.73 0.85 0.63 0.500.52 0.42 0.46 0.49 0.401.40 1.27 1.58 1.23 0.830.24 0.37 0.37 0.27 0.27

(a) (b)

(c) (d)

(e) (f)

49°N

51°N

52°N

53°N

54°N

50°N

49°N

51°N

52°N

53°N

54°N

50°N

49°N

51°N

52°N

53°N

54°N

50°N

110°W120°W 118°W 116°W 114°W 112°W 110°W120°W 118°W 116°W 114°W 112°WPrecipitation accumulation [mm]

Fig. 7. 48 hour precipitation amounts for Storm B for (a) the observed precipitation interpolated to 15 km resolution, and the simulations using 15 km gridresolution with the following cumulus parameterization schemes: (b) EX, (c) BMJ, (d) KF, (e) GD, and (f) G3D.


for Storm A and Storm B. In addition, the GD and G3Dschemes simulated this storm poorly, simulating much lessprecipitation than was observed indicated from small BIASvalues at the highest precipitation threshold. The lack of

precipitation resulted in ET scores only marginally above zerofor the GD and G3D schemes. While it has been noted that theGrell cumulus parameterization scheme has skillfully simu-lated heavy precipitation (e.g. Kerkhoven et al., 2006; Yang

EX BMJ KF GD G3D

0

100

200

300

400

(a)

50

40

30

20

10

0

(b)

Athaba

sca

[21 m

m]

Bow R

iver

[41 m

m]

Red D

eer

[56 m

m]

N. Sas

katch

ewan

[39 m

m]

S. Sas

katch

ewan

[23 m

m]

Milk

Rive

r

[7 m

m]Oldm

an

[24 m

m]Dom

ain

avera

ge

[36 m

m]

Perc

ent s

imul

ated

of

obse

rved

[%

]R

MSE

[m

m]

Fig. 8. Precipitation verification statistics, for Storm B, at each river basin, simulated using 15 km grid resolution. Percent simulated precipitation of observed(a) and RMSE values (b) for each cumulus scheme. Observed basin precipitation values are indicated below the basin's name in (b).


Table 7Precipitation scores for Storm A, Storm B, and Storm C, when simulated at15 km grid resolution, at the highest precipitation threshold of 50 mm. Boldvalues indicate the two most accurate cumulus schemes for the givenanalysis.

Storm Statistic EX BMJ KF GD G3D

A15 POD 0.63 0.53 0.77 0.53 0.47FAR 0.32 0.50 0.41 0.53 0.46BIAS 0.93 1.07 1.30 1.13 0.87ET 0.39 0.23 0.38 0.21 0.23

B15 POD 0.54 0.81 0.92 0.85 0.77FAR 0.59 0.58 0.49 0.51 0.58BIAS 1.31 1.92 1.81 1.73 1.85ET 0.19 0.25 0.37 0.33 0.24

C15 POD 0.43 0.57 0.43 0.03 0.10FAR 0.43 0.50 0.50 0.80 0.70BIAS 0.77 1.13 0.87 0.17 0.33ET 0.23 0.24 0.20 0.00 0.02


and Tung, 2003), the Grell scheme has also been noted tounderperform (e.g. Ratnam and Kumar, 2005), often simu-lating a drier and colder atmosphere than observed (Gochiset al., 2000).

Storm B and Storm C were also simulated at 6 km gridresolution (Fig. 11). B6-EX (B15-EX) simulated 103% (97%) ofthe total observed precipitation that occurred over the entiredomain, while B6-KF (B15-KF) simulated 110% (122%), asshown in Table 6 (Table 2). B15-EX had a RMSE value of30 mm, which reduced to 25 mm when simulated at 6 kmresolution. KF showed a much smaller improvement betweenthe two simulations, decreasing from 23 mm at 15 kmresolution, to 22 mm at 6 km resolution. Storm B, whenevaluated over all grid points inside the domain, had higherprecipitation accuracy when simulated at 6 km resolutionthan at 15 km.

Simulation B6-EX had higher POD (0.73, Table 6) whencompared to B15-EX (0.54, Table 7). The EX scheme also hadlower FAR and higher ET when simulated at 6 km resolution.While the KF simulation at 15 km resolution had higher POD,it also had higher FAR than was simulated at 6 km resolution.Consequently, the ET score (0.37) for simulations usingthe KF scheme did not change when the grid resolutionwas increased from 15 km to 6 km. The observation pointanalysis shows that a simulation of Storm B using theEX scheme at 6 km grid resolution was more accuratethan at 15 km grid resolution. The KF scheme showedminor improvements in accuracy when the resolution wasincreased from 15 km to 6 km. While both KF and EXsimulated this storm with the same ET score using 6 kmresolution, the KF scheme was more accurate with a lowerRMSE of 22 mm compared to EX with 25 mm, thus havingmore accuracy.

For Storm C, both EX and KF had higher RMSE values whensimulated at 6 km resolution than at 15 km resolution. The KFscheme showed a slight improvement when simulating theamount of precipitation at higher resolution, simulating 99%(109%) at 6 km (15 km) grid resolution. The EX schemeshowed the opposite. From the grid point analysis, higherresolution did not increase accuracy for Storm C for eithercumulus scheme.

The C6-EX and C6-KF simulations had nearly identical ETscores, at 0.27 (Table 6), which were higher than the ET scorefor the same storm when simulated at 15 km resolution(Table 7). When compared against the 15 km resolutionsimulations, the EX scheme had higher POD and FAR valuesat 6 km resolution, whereas KF had higher POD andlower FAR at the higher resolution. While both schemes hadhigher ET scores at the 6 km grid resolution simulations, KFexperienced a larger increase than EX when compared to the15 km resolution simulations. Thus, both schemes showedimprovements in accuracy when simulating precipitation at agrid resolution of 6 km when compared to a resolution of15 km. Both schemes simulated Storm C with very similarlevels of accuracy at 6 km resolution, with nearly identical ETand RMSE values.

5. Discussion and conclusions

Simulations performed by the WRF model showed thatusing a grid resolution of 15 km provides a compromisebetween computational efficiency and accurate resolution ofthe observed precipitation field. Refining the grid resolutionfrom 15 km to 6 km drastically increases the computationtime, while the accuracy of the precipitation distribution wasonly slightly better than simulations of 15 km grid resolution,with the difference in accuracy depending on the cumulusscheme. The EX scheme showed a slight improvement inaccuracy for higher resolution simulations, while the KFshowed less improvement.

Our findings were that for heavy rainfall events it wasbest to either use the Kain–Fritsch (KF) cumulus parameter-ization scheme, or alternatively, use the explicit schemewhich does not parameterize convection. The KF schemeand the explicit scheme consistently had low errors, highProbability of Detection and Equitable Threat scores. GDand G3D simulated very similar to each other and had lessskill than KF and EX for these three storms when simulatingheavy precipitation. BMJ simulations were also less accuratethan using the KF or EX scheme. Generally, all cumulusparameterization schemes showed some degree of skill whensimulating precipitation at a grid resolution of 15 km, thoughthe ET scores were not similar across all cumulus schemesand precipitation thresholds.

When comparing the 6 km resolution simulations against15 km, some interesting findings emerged. Storm Awas bettersimulated at 15 km resolution rather than at 6 km. In contrast,Storm B and Storm C were simulated more accurately at 6 kmresolution than at 15 km. The EX scheme performed fairlysimilarly to the KF scheme at the higher resolution, furthershowing that a cumulus parameterization scheme may not beneeded at high resolutions.

Of the true cumulus parameterization schemes (non-EX),the KF cumulus parameterization scheme best simulated theheavy precipitation events across Alberta during the summerseason. The strength of the KF scheme has been observed inother studies (e.g. Gochis et al., 2002). The KF scheme isthought to simulate convective precipitation more accuratelybecause the scheme conserves mass while using the param-eterization of convective downdrafts (Gochis et al., 2002) aswell as using CAPE as part of the closure assumptions (Wangand Seaman, 1997).

49°N

51°N

52°N

53°N

54°N

50°N

49°N

51°N

52°N

53°N

54°N

50°N

49°N

51°N

52°N

53°N

54°N

50°N

110°W120°W 118°W 116°W 114°W 112°W 110°W120°W 118°W 116°W 114°W 112°W

(a) (b)

(c) (d)

(e) (f)


Fig. 9. 48 hour precipitation amounts for Storm C for (a) the observed precipitation interpolated to 15 km resolution, and the simulations using 15 km gridresolution with the following cumulus parameterization schemes: (b) EX, (c) BMJ, (d) KF, (e) GD, and (f) G3D.


While the results indicate that these three heavy precip-itation storms were generally better simulated whenusing the finer grid resolution of 6 km, the increase in

accuracy was marginal. An objective of this study wasto determine whether finer resolution would result inhigher accuracy for precipitation events. However, the user

Perc

ent s

imul

ated

of

obse

rved

[%

]

Athaba

sca

[39 m

m]

Bow R

iver

[9 m

m]

Red D

eer

[39 m

m]

N. Sas

katch

ewan

[47 m

m]

S. Sas

katch

ewan

[3 m

m]

Milk

Rive

r

[1 m

m]Oldm

an

[10 m

m]Dom

ain

avera

ge

[31 m

m]

(a)

(b)

EX BMJ KF GD G3D

500

400

300

200

100

0

0

10

20

30

40

RM

SE [

mm

]

Fig. 10. Precipitation verification statistics, for Storm C, at each river basin, simulated using 15 km grid resolution. Percent simulated precipitation of observed(a) and RMSE values (b) for each cumulus scheme. Observed basin precipitation values are indicated below the basin's name in (b).


110°W120°W 118°W 116°W 114°W 112°W 110°W120°W 118°W 116°W 114°W 112°W

49°N

51°N

52°N

53°N

54°N

50°N

49°N

51°N

52°N

53°N

54°N

50°N

(a) (b)

(c) (d)


Fig. 11. 48 hour precipitation amounts for Storm B and Storm C at 6 km resolution using the KF scheme. Observed precipitation for Storm B (a) and Storm C (c) isshown above the simulated amounts for Storm B (c) and Storm C (d).

Table B.1Precipitation contingency table, where each element (HIT, MISS, FALSE, andNONE) hold the number of observation stations in which the observationand simulation exceed or fail to exceed a precipitation threshold. Forexample, for a precipitation threshold of 5 mm, if there was 7 mm ofobserved precipitation (rain) at an observation station, and 3 mm of


of the model needs to consider the additional resourcesrequired to process a higher resolution simulation. The15 km resolution simulations required slightly longerthan 1 h of computational time to perform a simulationon a new computer from 2010, from start to finish. Thecomputer was a 32-bit machine, using 2 CPUs, and 4 GBof RAM. The 6 km resolution simulations required a differentcomputer, which used 8 processors and 8 GB of RAM,and consumed over 18 h of computational time to startand finish each storm simulation. While a researcher mayprefer a higher resolution simulation that takes more thanhalf a day to perform, an operational flood forecastermay choose the coarser resolution with similar accuracy togive them more time to prepare any warnings that may berequired.

simulated precipitation (non-event) at the same observation station, thiswould be a MISS, increasing the MISS counter by one.

Observed:

Rain Non-event

Simulated: Rain HIT FALSENon-event MISS NONE

Appendix A. Grid point analysis

The grid point precipitation analysis is performed byusing the root-mean-squared-error (RMSE) between theobserved grid point precipitation value, O, and the simulated

grid point precipitation value, S, for every grid point, N, in theriver basin:

RMSE ¼ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi1=Nð Þ

XN1S−Oð Þ2

q: A:1

Appendix B. Observation point analysis


For a given precipitation threshold, the observation pointstatistics can be found for each storm using the numberof hits, misses, and false events from the contingency tableabove. Hits will be designated in formula form as H, misseswith M, false as F, and N will be the total number ofobservation stations verified.

The Probability of Detection (POD) is calculated as:

POD ¼ H= H þMð Þ: ðB:1Þ

The False Alarm Ratio (FAR) is calculated as:

FAR ¼ F= H þ Fð Þ: ðB:2Þ

The BIAS is calculated as:

BIAS ¼ H þ Fð Þ= H þMð Þ: ðB:3Þ

The Equitable Threat (ET) is calculated as:

ET ¼ H−Eð Þ= H þM þ F−Eð Þ: ðB:4Þ

Where the variable E is:

E ¼ H þ Fð Þ � H þMð Þ½ �=N: ðB:5Þ

References

Arakawa, A., Schubert, W.H., 1974. Interaction of a cumulus cloudensemble with the large-scale environment, part 1. J. Atmos. Sci. 31,674–701.

Betts, A.K., Miller, M.J., 1986. A new convective adjustment scheme. Part II:Single column tests using GATE wave, BOMEX, ATEX and arctic air-massdata sets. Q. J. R. Meteorol. Soc. 112, 693–709.

Brimelow, J.C., Reuter, G.W., 2005. Transport of atmospheric moisture duringthree extreme rainfall events over the Mackenzie River basin. J. Hydrol.6, 423–440.

Cherubini, T., Ghellia, A., Lalaurette, F., 2002. Verification of precipitationforecasts over the Alpine region using a high-density observing network.Weather. Forecast. 17, 238–249.

Colle, B.A., Westrick, K.J., Mass, C.F., 1999. Evaluation of MM5 and Eta-10precipitation forecasts over the Pacific Northwest during the coolseason. Weather. Forecast. 14, 137–154.

Colle, B.A., Mass, C.F., Westrick, K.J., 2000. MM5 precipitation verificationover the Pacific Northwest during the 1997–99 cool seasons*. Weather.Forecast. 15, 730–744.

Colle, B.A., Olson, J.B., Tongue, J.S., 2003. Multiseason verificationof the MM5. Part II: evaluation of high-resolution precipitationforecasts over the northeastern United States. Weather. Forecast. 18,458–480.

Cressman, G.P., 1959. An operational objective analysis system. Mon.Weather Rev. 87, 367–374.

Davolio, S., Mastrangelo, D., Miglietta, M.M., Drofa, O., Buzzi, A., Malguzzi, P.,2009. High resolution simulations of a flash flood near Venice. Nat.Hazards Earth Syst. Sci. 9, 1671–1678.

Done, J., Davis, C.A., Weisman, M., 2004. The next generation of NWP:explicit forecasts of convection using the weather research andforecasting (WRF) model. Atmos. Sci. Lett. 5, 110–117.

Efstathiou, G.A., Zoumakis, N.M., Melas, D., Lolis, C.J., Kassomenos, P.,2013. Sensitivity of WRF to boundary layer parameterizations insimulating a heavy rainfall event using different microphysicalschemes. Effect on large-scale processes. Atmos. Res. 132–133,125–143.

Flesch, T.K., Reuter, G.W., 2012. WRF model simulation of two Albertaflooding events and the impact of topography. J. Hydrometeorol. 13,695–708.

Fritsch, J.M., Chappell, C.F., 1980. Numerical prediction of convectivelydriven mesoscale pressure systems. Part I: convective parameterization.J. Atmos. Sci. 37, 1722–1733.

Gilliland, E.K., Rowe, C.M., 2007. A comparison of cumulus parameterizationschemes in the WRF model. Preprints, 21st Conference on Hydrology,San Antonio, TX, Amer. Meteor. Soc., P2.16.

Gochis, D.J., Yang, Z.-L., Shuttleworth, W.J., Maddox, R.A., 2000. Quantifica-tion of error in a pseudo-regional climate simulation of the NorthAmerican Monsoon. Proc. Second Southwest Weather Symp., Tucson,AZ, National Weather Service.

Gochis, D.J., Shuttleworth, W.J., Yang, Z.-L., 2002. Sensitivity of the modeledNorth American monsoon regional climate to convective parameteriza-tion. Mon. Weather Rev. 130, 1282–1298.

Gochis, D.J., Shuttleworth, W.J., Yang, Z.-L., 2003. Hydrometeorologicalresponse of the modeled North American monsoon to convectiveparameterization. J. Hydrometeorol. 4, 235–250.

Grell, G.S., 1993. Prognostic evaluation of assumptions used by cumulusparameterizations. Mon. Weather Rev. 121, 764–787.

Grell, G.A., Dévényi, D., 2002. A generalized approach to parameterizingconvection combining ensemble and data assimilation techniques.Geophys. Res. Lett. 29, 1693–1696.

Grubišić, V., Vellore, R.K., Huggins, A.W., 2005. Quantitative precipitationforecasting of wintertime storms in the Sierra Nevada: sensitivity to themicrophysical parameterization and horizontal resolution. Mon. Weath-er Rev. 133, 2834–2859.

Hewitson, B.C., Crane, R.G., 2005. Gridded area-averaged daily precipitationvia conditional interpolation. J. Clim. 18, 41–57.

Hong, S.-Y., Noh, Y., Dudhia, J., 2006. A new vertical diffusion package withan explicit treatment of entrainment processes. Mon. Weather Rev. 134,2318–2341.

Janjić, Z.I., 1994. The step-mountain Eta coordinate model: furtherdevelopments of the convection, viscous sublayer, and turbulenceclosure schemes. Mon. Weather Rev. 12, 927–945.

Janjić, Z.I., 2000. Comments on “development and evaluation of a convectionscheme for use in climate models”. J. Atmos. Sci. 57, 3686–3687.

Kain, J.S., 2003. The Kain–Fritsch convective parameterization: an update.J. Appl. Meteorol. 43, 170–181.

Kain, J.S., Fritsch, J.M., 1990. A one-dimensional entraining/detraining plumemodel and its application in convection parameterization. J. Atmos. Sci.47, 2784–2802.

Kain, J.S., Fritsch, J.M., 1993. Convective parameterization for mesoscalemodels: the Kain–Fritsch scheme. The Representation of CumulusConvection in Numerical Models. Meteorol. Monogr., No 46, Amer.Meteor. Soc. 165–170.

Kain, J.S., Baldwin, M.E., Weiss, S.J., 2003. Parameterized updraft massflux as a predictor of convective intensity. Weather. Forecast. 18,106–116.

Kerkhoven, E., Gan, T.Y., Shiiba, M., Reuter, G., Tanaka, K., 2006. Acomparison of cumulus parameterization schemes in a numericalweather prediction model for a monsoon rainfall event. Hydrol.Process. 20, 1961–1978.

Liang, X.-Z., Li, L., Kunkel, K.E., Ting, M., Wang, J.X.L., 2004. Regional climatemodel simulation of U.S. precipitation during 1982–2002. Part I: annualcycle. J. Clim. 17, 3510–3529.

Lin, Y.-L., Farley, R.D., Orville, H.D., 1983. Bulk parameterization of the snowfield in a cloud model. J. Clim. Appl. Meteorol. 22, 1065–1093.

Litta, A.J., Mahanty, U.C., Das, S., Idicula, S.M., 2012. Numerical simulation ofsevere local storms over east India using WRF–NMM mesoscale model.Atmos. Res. 116, 161–184.

Mass, C.F., Ovens, D., Westrick, K., Colle, B.A., 2002. Does increasing horizontalresolution produce more skillful forecasts? Bull. Am. Meteorol. Soc. 83,407–430.

McBride, J.L., Ebert, E.E., 2000. Verification of quantitative precipitationforecasts from operational numerical weather prediction models overAustralia. Weather. Forecast. 15, 103–121.

Mesinger, F., DiMego, G., Kalnay, E., Mitchell, K., Shafran, P.C., Ebisuzaki, W.,Jović, D., Woollen, J., Rogers, E., Berbery, E.H., Ek, M.B., Fan, Y., Grumbine,R., Higgins, W., Li, H., Lin, Y., Manikin, G., Parrish, D., Shi, W., 2006. NorthAmerican Regional Reanalysis. Bull. Am. Meteorol. Soc. 87, 343–360.

Michalakes, J., Dudhia, J., Gill, D., Klemp, J., Skamarock, W., 1999. Design of anext-generation regional weather research and forecast model. TowardsTeracomputing, World Scientific, River Edge, New Jersey. 117–124.

Ooyama, V.K., 1971. A theory on parameterization of cumulus convection.J. Meteorol. Soc. Jpn. 49, 744–756.

Ou, A.A., 2008. Meteorological Analysis of Four Rainstorms that CausedSevere Flooding in Alberta During June 2005. MSc. Thesis Univ. ofAlberta, Edmonton, Canada.

Phillips, D., 2010. Canada's Top 10 Weather Stories for 2010. Available at http://www.ec.gc.ca/meteo-weather/default.asp?lang=En&n=53E29740-1 (April2013).

Ratnam, J.V., Kumar, K.K., 2005. Sensitivity of the simulated monsoons of1987 and 1988 to convective parameterization schemes in MM5. J. Clim.18, 2724–2743.

http://refhub.elsevier.com/S0169-8095(13)00245-7/rf0225















































































































http://www.ec.gc.ca/meteo-weather/default.asp?lang=En&n=53E29740-1

http://www.ec.gc.ca/meteo-weather/default.asp?lang=En&n=53E29740-1





Reuter, G.W., Nguyen, C.D., 1993. Organization of cloud and precipitation inan Alberta storm. Atmos. Res. 30, 127–141.

Roberts, N.M., Lean, H.W., 2008. Scale-selective verification of rainfallaccumulations from high-resolution forecasts of convective events.Mon. Weather Rev. 136, 78–97.

Rozumalski, R.A., 2006. SOO/STRCWRF EMS User's Guide. Available online athttp://strc.comet.ucar.edu/software/newrems (April 2013).

Schaefer, J.T., 1990. The critical success index as an indicator of warning skill.Weather. Forecast. 5, 570–575.

Skamarock, W.C., Klemp, J.B., Dudhia, J., Gill, D.O., Barker, D.M., Duda, M.G.,Huang, X.-Y., Wang, W., Powers, J.G., 2008. A description of the advancedresearch WRF version 3. NCAR Technical Note NCAR/TN-475 + STR.(113 pp.).

Smith, H.D., 2011. Diagnosis of a large hail event over Calgary, Alberta, 12July 2010. Internal Report. Prairie and Northern Region. EnvironmentCanada.

Smith, S.B., Yau, M.K., 1987. The mesoscale effect of topography on thegenesis of Alberta hailstorms. Beitr. Phys. Atmos. 60, 371–392.

Vaidya, S.S., Singh, S.S., 2000. Applying the Betts–Miller–Janjic scheme ofconvection in prediction of the Indian monsoon. Weather. Forecast. 15,349–356.

Wang, W., Seaman, N.L., 1997. A comparison study of convective parame-terization schemes in a mesoscale model. Mon. Weather Rev. 125,252–278.

Warner, T.T., Petersen, R.A., Treadon, R.E., 1997. A tutorial on lateralboundary conditions as a basic and potentially serious limitation toregional numerical weather prediction. Bull. Am. Meteorol. Soc. 78,2599–2617.

Wilks, D.S., 1995. Statistical Methods in the Atmospheric Sciences. AcademicPress (467 pp.).

Yanai, M., Johnson, R.H., 1993. Impact of cumulus convection on thethermodynamic fields. The Representation of Cumulus Convection inNumerical Models. Meteorol. Monogr., No 46, Amer. Meteor. Soc.39–62.

Yang, M.-J., Tung, Q.-C., 2003. Evaluation of rainfall forecasts over Taiwanby four cumulus parameterization schemes. J. Meteorol. Soc. Jpn. 18,1163–1183.

East Bay. He moved to Edmonton, Alberta in2010 to pursue his Masters degree in Atmo-

Clark Pennelly grew up near San Francisco,California. He completed his undergraduatedegree in physics at California State University

spheric Sciences, which he completed in 2013.His research focuses on verification of numericalweather prediction models.

Gerhard Reuter grew up in Namibia, southernAfrica, where he became fascinated by rain, hailand thunderstorms. In 1982, he moved toMontreal where he completed a PhD on cumu-lus clouds at McGill University. Knowing thatAlberta is home to the world's most violenthailstorms, he came to Edmonton in 1990 for acareer as a University of Alberta professor in theDepartment of Earth and Atmospheric Sciences.His research focuses on hailstorms, flashflooding, tornadoes, and storm forecasting.






http://strc.comet.ucar.edu/software/newrems






























Unlabelled image

verification of the wrf model for simulating heavy...

Documents