interpolation of real-time ozone measurements in europe
DESCRIPTION
Interpolation of real-time ozone measurements in Europe. Results of a feasibility exercise for the Neighbourhood project Bill Oates [email protected]. Objectives. Review of available interpolation method and operational constraints Tests on archive data Tests on real-time data. - PowerPoint PPT PresentationTRANSCRIPT
Interpolation of real-time ozone Interpolation of real-time ozone measurements in Europemeasurements in Europe
Results of a feasibility exercise for the Neighbourhood project
Bill [email protected]
ObjectivesObjectives
• Review of available interpolation method and operational constraints
• Tests on archive data• Tests on real-time data
Review of Methods and ConstraintsReview of Methods and Constraints
• What methods can be used• What is the most feasible for EEA• Numbers and types of stations:
• What is the minimum density of stations needed?
• How many stations per country does this represent?
• Do we need to make a distinction between rural and urban stations?
Tests on Archive DataTests on Archive Data
• Datasets (from AirBase)• 12th August 2003: greatest number of stations exceeding the alert
threshold• 1st April 2003: contrasting, spring concentrations
• Methods tested• Simple – nearest neighbour• Interpolation – IDW, Kriging
• Station selection• All• Random• 3 densities: 1 station per 100x100km / 200x200km / 300x300km• Exclusion of urban and roadside concentrations
• Validation• Internal (Jack-knife) and External (test dataset) RMSE
Results (Simple Interpolation)Results (Simple Interpolation)
Number of StationsRMSE (ug/m3)
Cell Size: 50km102 55.73157 49.75261 48.10465 45.58
Number of Stations
RMSE (ug/m3)
Cell size (km)
10 25 50102 41.78 41.39 38.96
157 40.40 39.72 36.74261 37.61 35.75 34.01
465 36.93 35.09 33.59
Nearest Neighbour
IDW
Results (Kriging Interpolation)Results (Kriging Interpolation)
DAY TIME DENSITY SELECTIONNumber of stations
Cross-validation
RMSE ug/m3
External Valdiation
RMSE ug/m301 April 2 PM all valid points all 1491 16.7 01 April 2 PM 100k no urban no roadside 260 18.2 17.501 April 2 PM 100k all 311 20.1 17.501 April 2 PM 300k all 37 20.4 20.201 April 2 PM 200k no urban no roadside 67 20.8 20.201 April 2 AM 300k no urban no roadside 31 22.1 24.701 April 2 AM 100k all 303 22.1 21.9
12 August 2 PM 200k all 133 22.4 28.801 April 2 PM 300k no urban no roadside 34 23.0 18.801 April 2 PM 200k all 76 23.1 18.501 April 2 AM 100k no urban no roadside 250 23.2 23.001 April 2 AM 200k no urban no roadside 124 23.6 24.001 April 2 AM 300k all 34 23.7 26.301 April 2 AM 200k all 132 24.0 23.1
12 August 2 PM 100k no urban no roadside 264 24.6 25.812 August 2 PM 200k no urban no roadside 124 26.0 28.212 August 2 PM 100k all 314 26.4 25.112 August 2 AM 200k all 129 31.3 36.312 August 2 AM 300k all 26 31.6 40.312 August 2 PM 300k all 37 31.9 34.012 August 2 AM 100k all 309 33.6 33.212 August 2 AM 200k no urban no roadside 121 36.1 37.912 August 2 PM 300k no urban no roadside 33 37.2 31.412 August 2 AM 300k no urban no roadside 32 37.2 38.212 August 2 AM 100k no urban no roadside 257 38.5 32.7
Interpolation Errors
10.0
15.0
20.0
25.0
30.0
35.0
40.0
0 50 100 150 200 250 300 350 400 450 500
Number of Points
RM
SE
01 April 2 AM
01 April 2 PM
12 August 2 AM
12 August 2 PM
Nearest Neighbour
50km grid
IDW
10km grid
Methods and Constraints – Findings Methods and Constraints – Findings #1#1
• Accuracy levels can be improved from 50 ug/m3 to 30 ug/m3 (and better) by selection of improved interpolation methods• Kriging techniques delivered highest accuracy
results
• Station number and density has less impact on accuracy than the time of year / day:• Optimum station spacing of 200km
• No consistent pattern from the different type of station included
Methods and Constraints – Findings Methods and Constraints – Findings #2#2
• Geostatistical Analyst methods not available in automation routines• No “de-trending” functions
• Spatial Analyst methods are available in automation routines
• Kriging proven as the most accurate from the tests
• Choice of automatic vs. manual determination of variogram parameters
Real-time data objectivesReal-time data objectives
• How much real-time data exists?• Is real-time interpolation for ozone
feasible and practical?• What are the accuracy levels?• Automated vs. manual determination of
variogram parameters?
MethodsMethods
• Manual data “scraping” from existing AQ sites• Existing OzoneWeb stations
• Interpolation• Spatial Analyst within ArcGIS• Ordinary Kriging
• Semi-variogram: Spherical, self-optimising nugget and sill
• Lag-size: 50,000m• Search Radius: variable, 12 nearest neighbours
• 10km grid
14th July
Countries: 7
Valid Stations: 801
2nd August
Countries: 9
valid Stations: 921
1st September
Countries: 14
valid Stations: 985 (Airbase only)
14th July
Countries: 7
Valid Stations: 801
2nd August
Countries: 9
valid Stations: 921
1st September
Countries: 14
valid Stations: 985 (Airbase only)
Semivariograms and Semivariograms and Accuracy – 17Accuracy – 17thth July July
14 July 2005 0900hrs (RMSE=126.28)
0
200
400
600
800
1000
1200
1400
1600
1800
0 200 400 600 800 1000 1200 1400 1600 1800
Thousands
Distance between pairs (km)
Se
miv
ari
og
ram
Actual
Fitted
14 July 2005 1000hrs (RMSE=306.26)
0
100
200
300
400
500
600
700
800
0 200 400 600 800 1000 1200 1400 1600 1800
Thousands
Distance between pairs (km)
Se
miv
ari
og
ram
Actual
Fitted
14 July 2005 1100hrs (RMSE=252.72)
0
100
200
300
400
500
600
700
800
900
0 200 400 600 800 1000 1200 1400 1600 1800
Thousands
Distance between pairs (km)
Se
miv
ari
og
ram
Actual
Fitted
14 July 2005 1200hrs (RMSE=54.20)
0
200
400
600
800
1000
1200
1400
0 200 400 600 800 1000 1200 1400 1600 1800
Thousands
Distance between pairs (km)
Se
miv
ari
og
ram
Actual
Fitted
14 July 2005 1300hrs (RMSE=376.18)
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
0 200 400 600 800 1000 1200 1400 1600 1800
Thousands
Distance between pairs (km)
Se
miv
ari
og
ram
Actual
Fitted
14 July 2005 1400hrs (RMSE=440.36)
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
0 200 400 600 800 1000 1200 1400 1600 1800
Thousands
Distance between pairs (km)
Se
miv
ari
og
ram
Actual
Fitted
14 July 2005 1500hrs (RMSE=469.57)
0
500
1000
1500
2000
2500
3000
3500
4000
4500
0 200 400 600 800 1000 1200 1400 1600 1800
Thousands
Distance between pairs (km)
Se
miv
ari
og
ram
Actual
Fitted
14 July 2005 1600hrs (RMSE=613.18)
0
500
1000
1500
2000
2500
3000
3500
4000
4500
0 200 400 600 800 1000 1200 1400 1600 1800
Thousands
Distance between pairs (km)
Se
miv
ari
og
ram
Actual
Fitted
14 July 2005 1700hrs (RMSE=503.50)
0
500
1000
1500
2000
2500
3000
3500
4000
4500
0 200 400 600 800 1000 1200 1400 1600 1800
Thousands
Distance between pairs (km)
Se
miv
ari
og
ram
Actual
Fitted
14 July 2005 1800hrs (RMSE=520.29)
0
500
1000
1500
2000
2500
3000
3500
4000
4500
0 200 400 600 800 1000 1200 1400 1600 1800
Thousands
Distance between pairs (km)
Se
miv
ari
og
ram
Actual
Fitted
DAY TIME Sites
In Sites Tested
RMSE (ug/m3)
14 July 8 AM 198 44 32.4114 July 9 AM 446 92 25.6414 July 10 AM 498 107 23.6314 July 11 AM 528 109 22.8214 July 12 PM 597 116 21.1314 July 1 PM 637 125 15.4414 July 2 PM 648 128 15.9414 July 3 PM 596 112 16.7114 July 4 PM 620 122 17.4214 July 5 PM 618 121 16.1514 July 6 PM 629 122 17.62
Semivariograms and Semivariograms and Accuracy – 2Accuracy – 2ndnd August August
2 August 2005 0900hrs (RMSE=59.04)
0
100
200
300
400
500
600
0 200 400 600 800 1000 1200 1400 1600 1800
Thousands
Distance between pairs (km)
Se
miv
ari
og
ram
Actual
Fitted
2 August 2005 1000hrs (RMSE=136.56)
0
200
400
600
800
1000
1200
1400
1600
0 200 400 600 800 1000 1200 1400 1600 1800
Thousands
Distance between pairs (km)
Se
miv
ari
og
ram
Actual
Fitted
2 August 2005 1100hrs (RMSE=530.53)
0
500
1000
1500
2000
2500
0 200 400 600 800 1000 1200 1400 1600 1800
Thousands
Distance between pairs (km)
Se
miv
ari
og
ram
Actual
Fitted
2 August 2005 1200hrs (RMSE=682.71)
0
500
1000
1500
2000
2500
3000
3500
0 200 400 600 800 1000 1200 1400 1600 1800
Thousands
Distance between pairs (km)
Se
miv
ari
og
ram
Actual
Fitted
2 August 2005 1300hrs (RMSE=109.21)
0
100
200
300
400
500
600
700
0 200 400 600 800 1000 1200 1400 1600 1800
Thousands
Distance between pairs (km)
Se
miv
ari
og
ram
Actual
Fitted
2 August 2005 1400hrs (RMSE=783.28)
0
500
1000
1500
2000
2500
3000
3500
4000
0 200 400 600 800 1000 1200 1400 1600 1800
Thousands
Distance between pairs (km)
Se
miv
ari
og
ram
Actual
Fitted
2 August 2005 1500hrs (RMSE=1297.29)
0
500
1000
1500
2000
2500
3000
3500
4000
4500
0 200 400 600 800 1000 1200 1400 1600 1800
Thousands
Distance between pairs (km)
Se
miv
ari
og
ram
Actual
Fitted
2 August 2005 1600hrs (RMSE=1314.76)
0
500
1000
1500
2000
2500
3000
3500
4000
4500
0 200 400 600 800 1000 1200 1400 1600 1800
Thousands
Distance between pairs (km)
Se
miv
ari
og
ram
Actual
Fitted
2 August 2005 1700hrs (RMSE=1227.25)
0
500
1000
1500
2000
2500
3000
3500
4000
4500
0 200 400 600 800 1000 1200 1400 1600 1800
Thousands
Distance between pairs (km)
Se
miv
ari
og
ram
Actual
Fitted
2 August 2005 1800hrs (RMSE=188.93)
0
200
400
600
800
1000
1200
0 200 400 600 800 1000 1200 1400 1600 1800
Thousands
Distance between pairs (km)
Se
miv
ari
og
ram
Actual
Fitted
DAY TIMESites
InSites
Tested RMSE
(ug/m3)02 Aug 8 AM 297 49 23.9802 Aug 9 AM 656 111 19.5102 Aug 10 AM 706 118 16.9502 Aug 11 AM 750 119 14.6402 Aug 12 PM 761 122 11.0202 Aug 1 PM 605 98 9.4302 Aug 2 PM 747 123 11.7502 Aug 3 PM 760 119 11.5602 Aug 4 PM 793 123 12.2802 Aug 5 PM 787 121 12.5902 Aug 6 PM 335 52 12.61
1st September Tests1st September Tests
• Earlier tests and results discussed with ETC – suggestions for further analysis:
• Stratify by type of station (NB all “background” sites)• Urban• Rural• Suburban
• Run tests within the class of station, and between the classes of stations
• e.g. Rural stations for interpolation map, test the accuracy at the urban stations
• 10 Daylight Hours• Test #1: Average of the RMSE• Test #2: Sum of the Maximum Residuals
Test #1: Average RMSE (ug/mTest #1: Average RMSE (ug/m33))
• The least accurate: rural interpolation – urban test site • These can be mitigated for by including suburban sites
in additional to the rural sites. • This does however slightly decrease the accuracy for
the rural sites themselves.
Tested SitesInterpolated Sites ALL RURAL SUBURBAN URBAN
ALL 16.3 16.4 17.1 16.5
RURAL - 15.7 20.5 23.0
RURAL & SUBURBAN - 18.3 17.6 16.0
Test #2: Sum of Max. Residuals (ug/mTest #2: Sum of Max. Residuals (ug/m33))
Tested Sites
Interpolated Sites ALL RURAL SUBURBAN URBAN
ALL 477 386 419 377
RURAL - 442 530 559
RURAL & SUBURBAN - 552 517 468
• Same pattern as the RMSE results
Overall FindingsOverall Findings
• Interpolation of real-time Ozone concentration data is feasible
• Even from a relatively small number of monitoring stations,
• Results of acceptable accuracy can readily be generated using the standard interpolation techniques found within the ESRI software selected for the Neighbourhood Project
• Next steps:• Improved accuracy through meteorological
parameters• Increased resolution through differential interpolation
for station groups