15 / 05 / 2008

Model ensembles for the simulation of air quality over Europe

Robert VautardLaboratoire des Sciences du Climat et de

l’EnvironnementAnd many colleagues from IPSL, LISA, INERIS,

EURODELTA and TRANSCOM projects

15 / 05 / 2008

Why air quality modelling?

Short-term forecasts (0-3 days)

Long-term predictions of emission scenarios (climate?): 2010 or 2020 or more

Increase knowledge on processes together with observations…

15 / 05 / 2008

Air Quality forecastingPrevention

• 10 Years ago: statistical models, actions based on observations

• Now many deterministic forecasting systems

• Data assimilation in some cases

• In France, PREV’AIR system

• European GEMS/MACC projects (GMES)

15 / 05 / 2008

What are regional AQ models?

Transport

Chemistry

CTMWeather

BoundaryConditions

Emissions

LanduseConcentrations

Many many uncertainties…

15 / 05 / 2008

Regional air quality forecastnot really an initial value problem

Timmermans et al 2007

Assimilation experiments

without assimilation

with assimilation

Blond et al 2004

15 / 05 / 2008

What are the skill of regional AQ forecasts?

PREV’AIR Operational AQ forecasts (3 Summers):Average skill over >200 stations in Europe

Honoré et al. 2008

15 / 05 / 2008

Ensembles with perturbed meteorology (ARPEGE), chemistry

Carvalho et al., in preparation

20 MEMBERS

0.00

0.10

0.20

0.30

0.40

0.50

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

Bin

Rel

ativ

e fr

equ

ency

RUR_D1 RUR_D2 PUR_D1 PUR_D2 URB_D1 URB_D2

CHEMISTRY

0.000.100.200.300.400.500.600.700.80

1 2 3 4 5 6 7 8 9 10 11

Bin

Rel

ativ

e fr

equ

ency

RUR_D1 RUR_D2 PUR_D1 PUR_D2 URB_D1 URB_D2

METEOROLOGY

0.000.100.200.300.400.500.600.700.80

1 2 3 4 5 6 7 8 9 10 11 12

Bin

Rel

ativ

e fr

equ

ency

RUR_D1 RUR_D2 PUR_D1 PUR_D2 URB_D1 URB_D2

15 / 05 / 2008

Emissions controlAction

Loss in life expectancy attributable to PM2.5, and 2020simulation with current legislation, Amann et al 2005

15 / 05 / 2008

But some species are very poorly simulated

PM Episode intercomparisonStern et al. 2008

15 / 05 / 2008

Hopes from ensembles

Represent the « unpredictable part » of the system

Meteorological/emission « noise », knowledge gaps

Provide better deterministic predictions by « error cancelation »Delle Monache and Stull 2003; Galmarini et al., 2004; McKeen et al., 2005

Predict the uncertainty (in forecasts, in scenarios), using the rangeUsing one perturbed mode Hanna et al., 2001; Mallet and Sportisse 2006, Deguillaume et al., 2008, … or a

model ensemble; Vautard et al., 2006;

How to evaluate ? Easy for deterministic predictions More difficult for uncertainty: tools borrowed from ensemble

weather forecasting

15 / 05 / 2008

EuroDelta Experiment

• Regional, european scale evaluation of emission scenarios for 2010 or 2020

• Control experiment: simulation of Year 2001

• 7 models: CHIMERE, DEHM, EMEP, LOTOS-EUROS, MATCH, RCG, TM5,

• Comparison with rural stations (EMEP or AIRBASE)

• Results in– Van Loon et al., 2007 (Atmos. Env.)– Schaap et al., 2008 (in revision…)– Vautard et al., 2006 (Geophys. Res. Lett.)– Vautard et al., 2008 (AE, submitted)

15 / 05 / 2008

Example of improvement by ensemble averaging: Mean diurnal cycles

30

40

50

60

70

80

90

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Observed EMEP LOTOS MATCH CHIMERE

RCG DEHM TM5 Ensemble

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Observed EMEP LOTOS MATCH CHIMERE

RCG DEHM TM5 Ensemble

Ozone Ox=O3+NO2

15 / 05 / 2008

Seasonal skill scores for ozoneTable 5: Correlation coefficients for daily average and daily maximum O3.

  daily average daily maximum

  year DJF MAM JJA SON year DJF MAM JJA SON

EMEP

0.72 0.67 0.55 0.50 0.55 0.75 0.60 0.59 0.61 0.53

LOTOS

0.70 0.49 0.54 0.49 0.43 0.76 0.47 0.70 0.66 0.48

MATCH

0.80 0.68 0.66 0.60 0. 0.81 0.58 0.68 0.7 0.61

CHIMERE

0.76 0.62 0.58 0.64 0.60 0.84 0.62 0.71 0.77 0.62

RCG

0.71 0.58 0.59 0.52 0.36 0.76 0.56 0.70 0.61 0.44

DEHM

0.64 0.45 0.41 0.56 0.31 0.75 0.45 0.60 0.68 0.45

TM5

0.67 0.69 0.44 0.35 0.62 0.72 0.63 0.47 0.51 0.58

Ensemble

0.79 0.74 0.66 0.68 0.58 0.84 0.69 0.76 0.78 0.59

15 / 05 / 2008

The skill of the ensemble mean

• Perfect ensemble: Assume that the ensemble of K values xk is drawn from a distribution of physically possible states: Then the observation xa has the same statistical properties than any member of the ensemble, and the RMSE of the ensemble average can be written:

b is the ensemble bias, is the ensemble spread (standard deviation)

The RMSE is a decreasing function of the number of members K The RMSE (ensemble skill) is linearly linked to the ensemble spread

2211 bK

RMSEens

,

15 / 05 / 2008

Evaluation of uncertaintyConcepts and tools borrowed from ensemble weather forecasting

• Reliability: observation could be one of the members– Observation compatible with predicted distribution

Rank histogram: count the times the rank is 1, 2, …, n:frequencies should be equal

But predicted distributions can have no information content (random or climatological)

• Resolution: the smaller the ensemble spread, the higher the resolution

15 / 05 / 2008

Examples: time series

Too large spread

Too small spread

15 / 05 / 2008

Mean Rank Histograms

Stability of the ensemble

15 / 05 / 2008

Reliability and Resolution

Resolution index: Normalized spread = spread/stdevReliability index: (extreme counts – central counts) / total counts

15 / 05 / 2008

Spread - Skill relation

15 / 05 / 2008

CO2 Modelling : TRANSCOMWork in progress

CO2 modeling important for understanding and inverting fluxes

TRANSCOM ensemble (Law et al., 2008) : Evaluation of model ability to simulate CO2 at regional scale

2 Simulation Years: 2002 and 2003

17 atmospheric models/model versions differing by resolution, input biospheric fluxes (2), anthropogenic CO2 fluxes (2)

6 monitoring sites from CARBOEUROPE-IP

15 / 05 / 2008

Lack of spreadModel or/and data representativeness problems?

15 / 05 / 2008

Origin of ensemble spread and skill

15 / 05 / 2008

Conclusions

• Develop methods to evaluate uncertainty prediction

• European ensemble displays relatively complementary aspects

• For ozone, poor resolution in Atlantic areas, poor reliability in complex terrain, balanced ensemble in Northern Europe.

• For NO2, poor reliability, for secondary inorganic aerosols reliable ensemble. For nitrate, poor reliability in gaz/solid balance.

• For CO2: model ensemble mean spread too small. Analysis coming soon.

15 / 05 / 2008

European papers on evaluation and AQ model ensembles(several missing, most probably!)

Many individual model evaluations (to be reviewed)

EUROTRAC reports…

Tilmes et al., 2002: Forecasts over 1 month of ozone in Germany

Galmarini et al., 2004a,b; 2007 (ENSEMBLE project, dispersion models, ETEX)

EMEP review report Van Loon et al., 2004

Vautard et al., 2007, AE (CityDelta project): City-Scale (5 EU cities, 1 year), eulerian approach

Thunis et al., 2007, AE (CityDelta): Scenario ensembles at city scale

Van Loon et al., 2007, AE (EuroDelta project): Regional scale, Eulerian, ozone, 1 year

Vautard et al., 2006, GRL (EuroDelta, ozone): Ensemble uncertainty

Schaap et al., 2008, AE (EuroDelta): PM10 and components evaluation

Stern et al., 2008 (UBA exercise): PM10 extreme case in Germany