comparison between the gmi gprof precipitation product of...

Comparison between the GMI GPROF precipitation product of GPM and ground reference data over Europe by the H-SAF Precipitation Products Validation Group

7th Workshop of International Precipitation Working Group Tsukuba (Japan), 17-21 November 2014

Italian Civil Protection Department

Contributors:

Silvia Puca1, Angelo Rinollo1, Walter A. Petersen6, David B. Wolff6, Erich F. Stocker6, Ján Kaňák4, Georgy Koshinchanov5, Hristo

Hristov5, Judit Kerényi2, Rafał Iwański3, Bożena Łapeta3, Federico Porcù7, Gianfranco Vulpiani1.

1Italian Department of Civil Protection, [email protected], 2OMSZ-Hungarian Meteorological Service, 3Satellite Research Department, Institute of Meteorology and Water Management, 4Slovak Hydrometeorological Institute,

5National Institute of Meteorology and Hydrology, 6National Space and Aeronautics Administration (NASA), 7Universita' di Ferrara - Dipartimento di Fisica.

www.protezionecivile.gov.it

EUMETSAT SAF on Support to Operational Hydrology and Water

Management http://hsaf.meteoam.it

H-SAF management

Objectives: •1) To provide operational high quality level 2/3 products and develop new products from existing and future satellites with sufficient time and space resolution to satisfy the needs of operational hydrology;

• precipitation (liquid, solid, rate, accumulated); Leader, Italy (ISAC-CNR, CNMCA); exploitation of the full GPM Constellation of PMW radiometers for optimal monitoring and coverage of precipitation, and it will release advanced products for the next generation of EUMETSAT satellites, i.e. MTG and Post-EPS

• soil moisture (at large-scale, at local-scale, at surface and in roots region); Leader Austria (TU-Wien);

• snow parameters (detection, cover, melting conditions, water equivalent); Leader Finland, Turkey)

• 2) To provide independent validation to verify the usefulness of the products for civil protection purposes (floods, landslides, etc..), and for monitoring water resources, and the impact in hydrological models.

The consortium is composed by 11 countries (Austria, Bulgaria, Belgium, Finland, France, Germany, Hungary, Italy, Poland, Slovakia, Turkey) and ECMWF. USAM/CNMCA, Italy: H-SAF management, and data service


Proposal promoting collaboration between H-SAF and GPM

•The goal is to contribute toward the establishment of a long term collaboration between EUMETSAT H-SAF and GPM on the following aspects:

• precipitation retrieval algorithm development, through a fruitful interaction on several critical aspects of interest both to H-SAF and GPM (ISAC-CNR, CNMCA); Scientific coordinator: Giulia Panegrossi

• validation activity, through the connection between the well established H-SAF product validation (DPC, IMGW, and PPVG) and hydrological validation ((IMGW) programs and the Ground Validation/Calibration activity of GPM; Scientific Coordinator: Silvia Puca

•Active participation of H-SAF to GPM EM phase: • Daily download of “Europe” subset of GPM products • Algorithm development for all GPM constellation of radiometers; • Analysis of case studies and extensive validation over Europe of GPM products; • Use of H-SAF code for GMI product validation within GPM GV Team


5

The H-SAF precipitation products validation service

Leader : Italian Civil Protection Department

For all products generated by the project, the product validation cluster is responsible for:

• monitoring the progress in product quality as further development by evaluating statistical scores and case study analysis on the base of comparison between satellite products and ground data (radar and rain gauge);

• providing a validation service to end users by publishing on the H-SAF web-page the statistical scores evaluated and the case studies analysed;

• providing online quality control to end users by generating NRT quality maps; • monitoring operational features of the products as actual arrival, timeliness, etc..; • providing a ground data service inside the project for algorithm calibration and validation

activities.

The H-SAF PRECIPITATION PRODUCT VALIDATION GROUP (PPVG) is composed by 26 experts in hydrology, rain gauge data, radar data, and meteorology coming from 8 countries.

The H-SAF PPV Raingauge network is composed by 4100 stations

VS ‘Validation of the H-SAF precipitation products over Greece using rain gauge data’, Haralambos Feidas, Aristotle University of Thessaloniki, Department of Meteorology and Climatology. Starting 2014

Sources of uncertainty: very light/very high rain rates, drifting wind, and solid precipitation (snow or hail). •different geographical distributions, densities, quality

The rain gauge measurements are interpolated (GRISO method developed at CIMA, Italy) onto a uniform grid, with grid cells of size 5 Km (MSG SEVIRI resolution).

The H-SAF PPV Radar network is composed by 59 C-band radars

•All radars have Doppler capability, however, not all of them have dual polarization. •All radars available to PPVG are regularly maintained and calibrated (quality controlled by each owner Institute) • Each country has its own processing chain to estimate the Surface Rainfall Intensity

PPVG has defined a common processing chain methodology to evaluate the quality index and to calculate the Surface Rain Intensity product directly from the radar raw data available in the different countries, in order to unify precipitation field and quality index generation


The heterogeneity due to climatology, land cover, orography, and type of ground observations available for each country, represents an important resource of the PPVG, as it allows to investigate different aspects of the satellite product accuracy, but it requires also the definition and agreement of a common validation protocol across different countries. A COMMON SOFTWARE is used by all the validators, in order to ensure homogeneity in the validation results. Up-scaling techniques, inter-comparison methodology, validation statistics, are some of the elements defined in the common validation procedure and already applied by all the members of PPVG.

H-SAF Validation Protocol

•It started since launch time of the GPM Core satellite; •The comparison methodology has been decided in collaboration; •Exchange of information and codes:

– the H-SAF PPVG has shared with the GPM Ground Validation group all the adapted common codes.

Collaboration with GPM Ground Validation Team


• COMMON VALIDATION:

• Satellite product (GPROF V03) averaged onto a 0.5°x 0.5° grid; • Rain gauge masurements: spatial interpolation and re-mapped to same grid; • Radar estimates: up-scaling to match 0.5°x 0.5° grid; • Statistical scores evaluation (continuous and multi-categorical);

• CASE STUDY ANALYSIS

• carried out indipendently by each Institute on the base of the knowledge and experience of the Institute itself following a common format. Ancillary data such as lightning, MSG SEVIRI images, NWP models, nowcasting products might be used.

GPM product validation

Precipitation product : GMI GPROF V03 Period validated: March-June 2014

www.protezionecivile.gov.it COMMON VALIDATION • Evaluated on monthly basis; • Statistics are calculated over each country and then over the whole area; • Three different statistics for Land, Coast and Sea; • Rain/no rain threshold (0.20 mm/h for the GPM products); • Statistics evaluated for different precipitation classes. For GPM products:

-Class 1 (no rain): 0.00 – 0.20 mm/h -Class 2: 0.20 – 1 mm/h -Class 3: 1.00 – 110 mm/h -Class 4: all events above 0.20 mm/h -Class 5: all events above 1.00 mm/h Due to low RR found: Class 3 is actually 1.00 – 10 mm/h; one class was introduced to distinguish very low RR (0.2 – 1.0 mm/h); (No RR > 10 mm/h was found)

DATA USED • Radar data: Italy, Hungary, Slovakia (Poland for case studies) • Raingauge data: Italy (30 min cum. rain), Bulgaria (60 min cum. rain, very limited

area) (Poland for case studies)


Hungary 0.00< Rad < 0.20 0.20 < Rad < 1.00 1.00 < Rad < 10.00 10.00 < Rad

0.00< Sat < 0.20 -

0.20 < Sat < 1.00 -

1.00 < Sat < 10.00 -

10.00 < Sat - - - -

Slovakia 0.00< Rad < 0.20 0.20 < Rad < 1.00 1.00 < Rad < 10.00 10.00 <Rad

0.00< Sat < 0.20 -

0.20 < Sat < 1.00 -

1.00 < Sat < 10.00 -

10.00 < Sat - - - -

Bulgaria 0.00< RG < 0.20 0.20 < RG < 1.00 1.00 < RG < 10.00 10.00 < RG

0.00< Sat < 0.20 -

0.20 < Sat < 1.00 -

1.00 < Sat < 10.00 -

10.00 < Sat - - - -

Italy 0.00< RG < 0.20 0.20 < RG < 1.00 1.00 < RG < 10.00 10.00 < RG

0.00< Sat < 0.20 -

0.20 < Sat < 1.00 -

1.00 < Sat < 10.00 -

10.00 < Sat - - - -

R A D A R

R A I N

GA UG E

The percentages shown in a given column are computed with respect to the total number of satellite samples and represent how the satellite product classifies the events assigned to that class by the radar/rain gauges.

More than 94% (94–99 %) of no-rain events are correctly identified by GMI. However, there are also some high percentages in the other cells of the first row in Hungary and Bulgaria and Italy for light rain, indicating that some rain pixels are missed by the satellite products.

There is a general tendency of the GMI to underestimate the precipitation. Precipitation higher than 10 mm/h has been not observed by both satellite and ground instruments.

CONTINGENCY TABLES

Italy 0.00< Rad < 0.20 0.20 < Rad < 1.00 1.00 < Rad < 10.00 10.00 < Rad

0.00< Sat < 0.20 -

0.20 < Sat < 1.00 -

1.00 < Sat < 10.00 -

10.00 < Sat - - - -

98%

2% 0%

99%

1% 0%

99%

1% 0%

99%

1% 0%

94%

5% 1%

90%

8% 3%

38%

51% 12%

64%

32% 4%

74%

23% 4%

42%

42% 16%

82%

15% 3%

6%

33% 61%

23%

48% 29%

30%

50% 20%

0%

55% 45%


RADAR Land Coast Sea

rr>0.20 mm/h NS 1428 380 227

NR 2159 334 134

Mean Error -0.38 0.12 0.48

Std. Dev. 0.73 0.84 1.05

MAE 0.54 0.50 0.75

Bias 0.49 1.19 1.73

RMSE 0.82 0.85 1.15

Continuous statistical score

RAIN GAUGE Land Coast

rr>0.20 mm/h NS 1117 101

NR 1796 110

Mean Error -0.60 -0.48

Std. Dev. 1.05 0.82

MAE 0.76 0.61

Bias 0.39 0.40

RMSE 1.21 0.95

The scores show a good agreement between GMI and both radar and rain gauge data. There is a general underestimation for land areas and overestimation for sea areas. Mean Absolute Error 0.50 -0.76 mm/h and RMSE of 0.82-1.21mm/h are obtained. Better results are obtained for comparison with radar data.


PDF over Land areas, and overall scatterplots

The PDF presented shows good performance of GMI for land areas. The overall scatterplots (Land, Coast and Sea) do not show pronounced overestimation or underestimation patterns.

GMI vs Radar – All areas

GMI vs RG – All areas

PDF GMI and Radar – LAND

PDF GMI and RG – LAND

www.protezionecivile.gov.it Scatterplots among different countries

Very different results are obtained among different countries. We obtained a quite good results in Slovakia (land only), where stratiform precipitations occurred during the validation period (with overestimation by GPROF), and quite good in Italy (land, coast, and sea) where also convective precipitation has been registered, and bad results in Hungary. The difference between the scatterplots are mainly due to the different precipitation regimes in the countries (Please, note, this are preliminary results subject to further verification)

GMI vs Radar – Slovakia GMI vs Radar – Hungary GMI vs Radar – Italy


• The GMI GPROF has a good capability to detect no rain areas (threshold 0.2 mm/h) especially over land.

• High % of missed light precipitation (0.2 mm/h – 1.00 mm/h) compared to both

radar and rain gauges. • Good performances have been evaluated in term of continuous statistical scores

(Mean error from -0.38 mm/h (over land) to 0.48 mm/h (over sea)) and Root Mean Squared Error from 0.82 mm/h to 1.15 mm/h) using as reference the radar data.

• Good results have been obtained generally for the stratiform precipitation

occurred in Slovakia (with some overestimation), and for Italy (overall better with respect to radar than to raingauges).

• More verifications on the correct use of the GMI validation code used by all the

countries is necessary (Hungary).

Some conclusions on common validation

www.protezionecivile.gov.it Case studies analysis

Bożena Łapeta and Rafał Iwański

Satellite Remote Sensing Department

Institute of Meteorology and Water Management – National Research Institute, Poland

(and members of H-SAF PPVG)

POLAND

www.protezionecivile.gov.it Ground data sources used

• The ATS network consists of 1057 Automatic Telemetric Stations. The precipitation data is recorded and accumulated to 10 minutes time slots.

• Each precipitation measurement post is equipped with two automatic tipping bucket rain gauges: heated and non-heated

• The meteorological radar network POLRAD consists of 8 devices and covers almost whole territory of Poland.

POLRAD radars are automated, unattended and remotely-controlled.

• Each of the radars provides unified precipitation field for its perimeter. Also a composite map of Poland is generated for operational use.

• 24 basic radar products are generated. To validate the satellite precipitation products the Surface Rainfall Intensity (SRI) was chosen.

www.protezionecivile.gov.it Methodology

• Check for availability of GMP overpasses covering Poland with its swath (especially Southern part of the country);

• Only valid GMI pixels were used for comparison with ground data; • All available data sources were presented on maps using Natural Neighbour

spatialisation method (precipitation values were smoothed). • GMI GPROF pixels were combined with ATS measurements and radar grid data

using the following simple method. • For each satellite pixel, the rain gauges posts and radar grid values situated

within that pixel were found. • The pixel size was chosen to be of 0.065 deg diameter and its shape was

assumed to be circular. • If more than one rain gauge (radar data point) was found within one satellite

pixel, the ground rain rate value was calculated as a mean of all rain gauges measurements in this pixel.


The conformity of precipitation areas location and structures present on all maps is very good. The max ATS and radar precipitation were properly detected and estimated.

Case study on

16.05.2014 (stratiform case)

GMI GPROF V03

RADAR ATS

0

2

4

6

8

0 2 4 6 8

GPRO

F rr

[mm

/h]

ATS rr [mm/h]

2A-CS-EUROPE.GPM.GMI.GPROF2014v1-3.20140516-S031635-E033247.001205.V03A

0

2

4

6

8

10

12

0 2 4 6 8 10 12

GPRO

F rr

[mm

/h]

Radar rr [mm/h]

2A-CS-EUROPE.GPM.GMI.GPROF2014v1-3.20140516-S195730-E201224.001216.V03A

RADAR

GM

I

ATS

The liquidPrecipFraction (not shown) distribution shows small number of pixels with values below 1 (snow precipitation events reported in Tatra Mountains). ConvectPrecipFraction (not shown) presents very few pixels with convective precipitation pixels (expected in case of stratiform precip)


Statistical scores

Dicotomous statistics: Threshold: 0.20 mm/h The higher values of POD than FAR indicates that GMI GPROF product ability to recognize the stratiform precipitation is good for ATS and very good for radar data. This is also confirmed by high CSI and ACC scores. The results obtained for radar data are much better than for ATS.

Max Sat

Max ground

ME MAE RMSE ST.Dev

ATS 7.79 6.6 -1.021 1.219 1.614 1.253 RD 7.79 10.32 -0.09 0.63 1.009 1.004

Continuous statistics: The average precipitation measured during this event by rain gauges is about 1.85 mm/h while the GMI GPROF averaged is of 0.83 mm/h. In the case of radar data, the average precipitation is about 1.11 mm/h and 1.01 mm/h for radar and GMI GPROF, respectively. Very good results are obtained for radar data.


Some Conclusions on Case Studies in Poland

The analysis performed for situation with stratiform precipitation occurred on the 16th of May 2014 showed very good ability of GMI GPROF product in

recognition and estimation of precipitation. The GMI GPROF conformity with radar data is excellent.

Max Sat

Max ground

ME MAE RMSE ST.Dev

ATS 7.29 17.7 -1.352 1.526 3.305 3.033 RD 12.33 64.02 -1.191 2.013 5.248 5.116

For Convective case 28/5/2014: showed quite good ability of GMI GPROF product in recognition and estimation of

precipitation, but worse scores than for stratiform. The ground moderate and high precipitation is

underestimated.

www.protezionecivile.gov.it Case studies analysis

Angelo Rinollo, Silvia Puca, Gianfranco Vulpiani

Italian Department of Civil protection

Federico Porcù University of Ferrara

Emilio Guerriero

SELEX

ITALY


Reference data and methodology

• The same reference data used for the long series comparison (Italian national

radar mosaic and Italian national rain gauge network) has been used for the case study analysis.

– 18 C-band radars (product: SRI – mosaic) – About 2500 rain gauge stations. Measurements every 30 minutes.

• Same methodology used for the long series comparison. (GMI GPROF product is

re-mapped onto a 50 km x 50 km grid via a simple arithmetical average. Then: – Radar data are up-scaled onto the same grid; – Rain gauge data are first interpolated using the GRISO technique and up-

scaled onto the same grid.

• The comparison with radar data is carried out for all available GMI overpasses of the day, while the one with rain gauges is done for the most representative overpasses.


The GMI shows fairly good ability to correctly identify the precipitation patterns. It is observed an underestimation over land compared to the radar, and overestimation over sea, the main precipitation structure over the Tyrrhenian sea is correctly identified.

GMI Radar Rain gauges

Case study on 15.6.2014

11:30 UTC On the 15th of June 2014 the presence of fresh Atlantic air in the upper atmosphere, together with the

hot and moist pre-existing air in the lower layers has determined instability conditions over Italy, especially on the Tyrrhenian areas, with localized showers and formation of diffuse thunderstorms.


Good agreement is observed between the GMI GPROF and the radar field, while the rain gauges seem to identify smaller precipitation areas and lower amounts. Again, some overestimation over sea is evident, the precipitation pattern detected by the radar is quite well reconstructed by the GMI GPROF product.

GMI Radar Rain gauges

Case study on 15.6.2014

19:30 UTC


Statistical scores

• A POD of 0.49 over land has been obtained both for rain gauge and radar data (the values over sea might not be representative due to a low number of data). Slightly better results obtainedwith radar data in terms of FAR and CSI.

• Continuous scores show underestimation over land and overestimation over sea by the GMI GPROF product, for both radar and rain gauges. Correlation coefficients are better over sea than over land.

RG RD (Land)

RD (Sea)

POD 0.49 0.49 1.00

FAR 0.26 0.13 0.26

CSI 0.42 0.46 0.73

Score (RR≥0.20)

Rain gauges

Radar (Land)

Radar (Sea)

NSat 62 45 15

NGround 94 79 11

Mean Error

- 0.510 - 0.404 0.421

Bias 0.312 0.370 1.408

MAE 0.552 0.422 0.451

Corr. Coeff.

0.328 0.356 0.773

RMSE 0.808 0.600 1.168

Dicotomous statistics: Threshold: 0.20 mm/h


Some Conclusions on case studies in Italy

Case study on 15 June 2014: •good ability of GMI GPROF product in precipitation detection; •underestimation over land wrt both radar and rainguages, and overestimation over sea (wrt radar); •better results are obtained for comparison with radar; Similar results were obtained for other two case studies analyzed: •3 May 2014 (long-lasting and intense precipitation causing extensive flooding in Central Italy •25 May 2014 (localized showers and formation of diffuse thunderstorm in Central-Southern mountainous areas)


Next steps in GPM/H-SAF validation activity

Contact: [email protected]

• More H-SAF countries participating to the validation (i.e., Germany, Turkey, Belgium) are expected ;

• Further tests on ground data quality;

• Comparison at finer grid resolution (need to check ability to capture fine precip. structure);

• Validation of other GPM products (to be defined);

• Check for consistency of precip. between all radiometers for optimal monitoring and applications (i.e., merging with GEO-IR, hydrology, nowcasting)

• Snowfall detection capabilities (and rate or accumulations?)

• Comparisons with H-SAF PMW precipitation products

• Hydrological validation


References: Puca S. et al.,.: The validation service of the Hydrological SAF geostationary and polar satellite precipitation products, Nat. Hazards Earth Syst. Sci., 14, 871-

889, doi:10.5194/nhess-14-871-2014, 2014 Capacci, D., and Porcù, F.: Evaluation of a satellite multispectral VIS-IR daytime statistical rain-rate classifier and comparison with passive microwave rainfall

estimates. J. Appl. Meteor. Climat., 48, 284-300, 2009. Casella D., G. Panegrossi, P. Sanò, A. Mugnai, E. A. Smith, G.J. Tripoli, S. Dietrich, M. Formenton, W.Y. Leung, A. Mehta, Transitioning from CRD to CDRD in

Bayesian Retrieval of Rainfall from Satellite Passive Microwave Measurements: Part 2. Overcoming Database Profile Selection Ambiguity by Consideration of Meteorological Control on Microphysics. IEEE Trans. Geo. Rem. Sens., vol.51, no.9, 4650-4671, 2013 doi: 10.1109/TGRS.2013.225816

Casella D., P. Sanò, G. Panegrossi, L. Milani, M. Petracca, S. Dietrich, A new scheme for the detection of precipitation in tropical regions using PMW radiometers, Atmos. Meas. Tech., submitted.

Lábó, E.: Validation studies of precipitation estimates from different satellite sensors over hungary – Analysis of new satellite-derived rain rate products for hydrological purposes. Journal of Hydrology, 468-469, 173-187, 2012.

Leitinger, G., N. Obojes and U. Tappeiner: Accuracy of winter precipitation measurements in alpine areas: snow pillow versus heated tipping bucket rain gauge versus accumulative rain gauge, EGU General Assembly 2010, held 2-7 May, 2010 in Vienna, Austria, p.5076, 2010.

Mugnai, A., Smith, E.A., Tripoli, G.J., Bizzarri, B., Casella, D., Dietrich, S., Di Paola, F., Panegrossi, G., and Sanò, P.: CDRD and PNPR satellite passive microwave precipitation retrieval algorithms: EuroTRMM / EURAINSAT origins and H-SAF operations, Nat. Hazards Earth Syst. Sci., 13, 887–912, doi: 10.5194/nhess-13-887-2013, 2013.

Mugnai, A., Casella, D., Cattani, E., Dietrich, S., Laviola, S., Levizzani, V., Panegrossi, G., Petracca, M., Sanò, P., Di Paola, F., Biron, D., De Leonibus, L., Melfi, D., Rosci, P., Vocino, A., Zauli, F., Puca, S., Rinollo, A., Milani, L., Porcù, F., and Gattari, F.: Precipitation products from the Hydrology SAF, Nat. Hazards Earth Syst. Sci., 13, 1959–1981, 2013.

Panegrossi G., D. Casella, S. Dietrich, P. Sanò, M. Petracca, and A. Mugnai, A verification study over Europe of AMSU-A/MHS and SSMIS passive microwave precipitation retrievals, Proc. 2013 Joint EUMETSAT/AMS EUMETSAT Meteorology Satellite Conference, Vienna, Austria, 2013

Pignone F., Rebora N., SilvestroF. and Castelli F., GRISO (Generatore Random di Interpolazioni Spaziali da Osservazioni incerte)-Piogge, Relazione delle attività del I anno inerente la Convenzione 778/2009 tra Dipartimento di Protezione Civile e Fondazione CIMA (Centro Internazionale in Monitoraggio Ambientale), report n° 272/2010, pp 353, 2010.

Porcù F., L. Milani and M. Petracca: On the uncertainties in validating satellite instantaneous rainfall estimates with raingauge operational network. Atmos. Res., 144, 73-81, 2014

Rinollo, A., G. Vulpiani, S. Puca, J. Kaňák, E. Lábó, Ľ. Okon, E. Roulin, P. Baguis, E. Cattani, S. Laviola, V. Levizzani: Definition and impact of a quality index for radar-based reference measurements in the H-SAF precipitation product validation, NHESS, 13, 2695-2705, 2013.

Sanò P., D. Casella, A. Mugnai, G. Schiavon, E.A. Smith, G.J. Tripoli, Transitioning from CRD to CDRD in Bayesian Retrieval of Rainfall from Satellite Passive Microwave Measurements: Part 1. Algorithm Description and Testing. IEEE Trans. Geo. Rem. Sens., vol. 51, no. 7, 4119-4143, july 2013, doi: 10.1109/TGRS.2012.2227332.

Sanò P., Casella D., Panegrossi G., Di Paola F., L. Milani, A. Mugnai, Petracca M., S. Dietrich, The Passive microwave Neural network Precipitation Retrieval (PNPR) algorithm for AMSU/MHS observations: description and application to European case studies, Atmos. Meas. Techn., submitted

Thank you!

comparison between the gmi gprof precipitation product of...

Documents