Adaptive Neural Network Metamodel for Short-term Prediction of Background Ozone Level

Herman Wahida,b

aFaculty of Electrical Engineering Universiti Teknologi Malaysia

Skudai, Johor, Malaysia. hbinwahi@eng.uts.edu.au

Q. P. Hab

bFaculty of Engineering University of Technology, Sydney

Broadway, NSW, Australia. quangha@eng.uts.edu.au

Hiep Nguyen-Ducc

cDepartment of Environment and Climate Change and Water,

NSW, Australia. hiep.duc@environment.nsw.gov.au

Abstract—Modelling is important in air quality forecasting and control. Before applying an air quality model, it is required to accurately estimate the biogenic emission. The assessment of the background ozone concentration is essential for this estimation. It has been known that the biogenic ozone level in urban areas is changing over the years, and hence information about the temporal trends in air quality data is helpful for the assessment. This paper presents a neural-network metamodel for prediction of the background ozone level in the Sydney basin. Based on measured monitoring data under non-photochemical conditions collected at a number of monitoring stations, the proposed model can reliably provide short-term predictions in the biogenic ozone trends to be used for analysis of ground-level emission impact on air quality.

Keywords-Metamodelling, radial basis function neural network, adaptive spread, background ozone trend, Sydney basin

I. INTRODUCTION

Ozone (O3) is a gas that is naturally produced and present at the earth’s atmosphere. The stratospheric ozone is very useful as it could prevent us from the harmful influence of the sunlight ultraviolet. However, an excessive amount of the tropospheric ozone (or surface ozone) exposure may be harmful to living organisms as it can damage living tissues of humans, animals and plants in direct contact, and might contribute to the warming of the earth’s surface [1].

Most of the investigation on ozone from the literature has focused in the urban areas where it is typically formed by the photochemical reactions from anthropogenic source emission, such as that of nitrogen oxides (NO), dioxides (NO2) and volatile organic compounds (VOCs). The background ozone level is referred to the level of the ozone formed naturally via mostly biogenic processes, i.e. from natural processes free from anthropogenic influences, occurring at the troposphere [2]. According to the US Environmental Protection Agency (EPA), the background ozone is named Policy-Relevant Background (PRB), referred to as the concentration that would occur in the absence of anthropogenic emissions [3]. As to the health risk concern, in the 1996 ozone review, the EPA used 40 ppb (parts per billion) as the 8-hour daily maximum background ozone level in its assessment evaluation. Altshuller et al. [4] suggest that the background ozone is somewhat dependent on a number of conditions such as the

nature of upwind flow, lack of pollution sources, and terrain conditions, including deposition with respect to forest or agricultural areas.

In this paper, data for the non-photochemical background ozone level at some sites in Sydney are derived from the ambient measurements during nighttime and early morning hourly values (i.e. from 19:00pm to 08:00am the next morning). This would exclude any photochemical processes that normally occur during the daytime as if only natural sources were present. The idea here is to use a metamodelling approach to design a computationally effective model for each monitoring station, and from this information, a more generic model could be constructed to extend over the whole area in the Sydney basin.

II. AIR QUALITY MODELLING OVERVIEW

An air-quality model is the mathematical description of the atmospheric transport, diffusion, and chemical reactions of pollutants, which could help determine measures for reducing emission and pollutants to achieve a desired air-quality standard, such as the National Ambient Air Quality Standard (NAAQS). Some of the early surveys in this field can be found in [5,6].

Since the beginning of the 21st century, more sophisticated approaches have been developed owing to advances in computer simulation. The most popular model has been released by US EPA in 2001, called Community Scaled Air Quality (CMAQ). In its latest version, CMAQ contains three types of modelling components, known as CMAQ-Model-3 [7]. The CMAQ output is annual emission inventory data, useful for long-term trend analysis and reporting. The Comprehensive Air-quality Model with extensions (CAMx) [8] is a photochemical grid model developed in late 1990s to study the urban and regional scale air quality problems. The model is an ideal platform for extension to a variety of air quality issues including ozone, particulate matters (PM), visibility, acid deposition and air toxics. Recently, the US EPA uses a global model called GEOS-CHEM to estimate the background ozone level. This model has been developed by a group of researchers at Harvard University since 2004 [9]. However, it is unable to successfully reproduce the temporal

978-1-4244-8075-3/10/$26.00 ©2010 IEEE

changes in hour-by-hour concentrations [10]. Most of the afore-mentioned techniques are able to give

good results for long term prediction. Unfortunately, it is not so accurate when dealing in a short-term basis. Furthermore, the mathematical modelling formulation requires a substantial effort, and the computing time could be very long. To deal with the complexity of an air quality model, neural network approaches may offer an alternative way to simplify the mathematical modeling process.

III. METAMODELLING METHODOLOGY

A. Metamodelling Metamodelling is a technique for determination of simpler

models from complex models that involve less computation but represent adequately a good approximation for nonlinear behaviours. There have been a number of techniques for meta-modeling: neural networks [11], Kriging [12], Radial Basis Function (RBF) [13], and Multivariate Adaptive Regression Splines (MARS) [14]. Among them, Kriging models, Gaussian and RBF processes are intensively investigated (as reported in [15] and [16]).

In metamodelling, an initial dataset should be firstly constructed. Next, data are sampled and grouped uniformly or randomly via an experimental design such as the Latin hypercube design (LHD), full factorial design method, orthogonal array sampling, etc. The sampling method selection depends on a specific approximation problem. By sequential sampling, a target metamodel is obtained. Next, the system error is determined by computing the difference between the metamodel output and the target output. The iteration process begins with adding the new sample group, and the routine will continue until satisfying the prescribed error goal. B. Radial Basis Function Neural Network (RBFNN)

A radial basis function network consists of two layers: a hidden layer of radial basis function and a linear output layer. Each of the hidden neurons implements a radial activated function. Various types have been tested as the activation function such as Gaussian, multiquadratic, polyharmonic and thin-plate spline [17].

By denoting R the number of inputs and Q the number of outputs, the output of the RBFNN for Q = 1 is calculated as

( ),),(1

211

=

−=R

kkk cxwwx φη (1)

where 1×ℜ∈ Rx is an input vector, ( )⋅φ is a basis function,

2⋅ denotes the Euclidean norm, w1k are the weights in the

output layer, 1×ℜ∈ Rkc are the RBF centres in the input vector

space. For S1 neurons in the hidden layer, (1) can also be written as,

wxwx T )(),( φη = , (2)

where ( ) ( )[ ]1111)( SST cxcxx −−= φφφ (3)

[ ]111211 ST wwww = , (4)

with most common being a Gaussian function given by:

,exp)( 2

2−−=

σφ kcx

x (5)

in which σ is the spread parameter of the RBF. Advantages of RBFNN over the Multilayer Perceptrons

include the ability to effectively generate multidimensional interpolative approximations while yielding robustness and reliability in a computationally-aided design. As compared to a conventional method incorporating a fixed algorithm, the RBFNN metamodel will learn intelligently the nonlinear mapping between the input and output of the system, while solving the particular problem [13].

C. Adaptive Radial Basis Neural Network In MATLAB, the spread parameter σ is quite often set

manually by trial and error. There still remains a question as to if the results obtained are in the optimum point for various spread constants. As mentioned in [18], it is suggested that σshould be large enough that neurons respond strongly to the overlapping region of the input space. Also, it must be selected greater than 0.1 of the interval between inputs, and less than 2 of the distance between the leftmost and rightmost inputs. Thus, it may be interesting to tune for some optimal value of σ [19]. This idea can be illustrated here by testing a simple quasi-sinusoidal function. The n-dimensional quasi-sinusoidal function is generally defined as [20]:

[ ],)15/16(sin)15/16sin(3.0)(1

2

=

−+−+=n

iii xxxf εε (6)

where 7.0=ε ,, i=1 for a one dimensional problem, and

[ ]2,1−∈ix . The initial sample set is constructed by eight randomly distributed points. By using a standard RBFNN in MATLAB and varying the spread parameter values, it is observed that the produced performance index, the root mean square error (RMSE) decays until reaching a certain minimum point. This minimum point is considered as the optimal point of σ , as shown in Fig. 1.

This illustration prompts to the suggestion that an optimal value for the spread parameter is a function of RMSE at the minimum global point. Motivated by this, we propose to use the gradient criteria to adjust σ till the first derivative of RMSE approaching 0, i.e.

0)( =∂

∂=∇σ

σ RMSE. (7)

From a given initial value sp0, the optimal point can be achieved by an optimisation technique such as steepest descent, Newton’s method, and Marquardt method. It has been determined by many experiments that the sp0 must be chosen between 0.1 and 10 in order to get the best convergence. For this stage of development, we choose the steepest descent [21] technique for simplicity. The procedure can be written as

,)()( σγ∇−= oldnew spsp (8)

where sp is the updated spread parameter, γ >0 is the step size (or learning rate) and σ∇ is the gradient defined in (7). In order to terminate the iterative process, the following criterion was used, where the change in the function value for two consecutive iterations becomes smaller than a threshold.

.1

)(

)()( ε≤−

old

oldnew

RMSERMSERMSE (9)

The advantageous feature here is that the best performance is determined at every neuron, instead of using a fixed spread value in the standard RBFNN.

IV. BACKGROUND OZONE LEVEL PREDICTION

Our scope is to model a network to predict the background ozone layer at several urban sites in the Sydney basin, namely Blacktown, Lidcombe and St. Mary. A proper selection of the input and output characteristic is essential in order to make the RBFNN learn with a fast convergence. Typically, more input data are better as to make the model more comprehensive and the interpretation more convincing. The entire inputs and targets have to be normalised (e.g. for a minimum of 0 to a maximum of 1), e.g. by using mapminmax function in MATLAB, in order to make them contribute with same influence to the RBFNN. This also allows the Gaussian activation function to squash all incoming data and to make the execution faster.

In this work, four input parameters are used, namely time, NO, NO2 and ozone concentrations as measured at the monitoring stations. For the target output, we set the ozone concentration without nitrogen oxide components after some interval time. The size of the interval depends on the prediction horizon. Typically, one hour is adopted as the time interval for this evaluation. The test set should follow the same manner as with the training set, i.e. the time interval for the test is the same as the time interval used for the training set. Once the learning process is finished and the accuracy by some test sets is satisfactory, the network can be used for prediction with respect to other available data.

V. SYDNEY BASIN BACKGROUND O3 LEVEL AND ANALYSIS

In this work, the hourly data, collected by the NSW Department of Environment, Climate Change and Water at

various stations in the Sydney basin, have been post-processed and quality controlled to produce good data sets [22]. A. Case 1: Blacktown station

We used the data recorded in the year of 1998. Using adaptive RBFNN, we initialize the training process by setting the initial spread parameter and prescribed error goal. After several epochs, the network is constructed once having met the set goal. The network will then be tested with the sample testing data and the results of prediction on the testing set are shown in Fig. 2. As can be seen, most of the values have shown good results of prediction, where it follows the pattern of the actual values derived from ambient measurements. Results for this short-term prediction can be extended to determine the long term trend for the background ozone level, as other modelling systems do. This trend analysis is reported in [23]. Notably, it is quite difficult to get the non-photochemical condition at every hour, thus information of the background ozone level during that hour cannot be obtained. By using this model, the background ozone still can be predicted for events as in past hours under the non-photochemical conditions. Therefore, it is envisaged that the model could be used on-line for continuous prediction of the background ozone levels.

B. Case 2: Lidcombe station The same routine was applied for Lidcombe, a site located

in the central west region of Sydney. The example of prediction results is depicted in Fig. 3, showing good results where only few points were different from the values obtained from measurements. The short-term background ozone level was found between 2ppb to a maximum of 30ppb, both for actual and predicted values, which are within the health-risk tolerance. The model has been trained a priori by set the goal of adaptive RBFNN to an error goal of 0.02 and initial spread parameter of 0.1. After several epochs, the modeling process was terminated giving an optimal spread value of 2.7 at the final hidden neuron. Furthermore, the produced mean absolute error is 5.1, implying the model gives better performance than the previous case, probably due to more non-photochemical data were available during the training stages.

C. Case3: St. Mary station Another urban site in the west of Sydney, St. Mary, was

considered. Figure 4 shows the short-term prediction results from a sampled data in the year of 2004. The networks have been constructed by using the proposed RBFNN metamodelling approach after 5 iterations, and it has been determined that the optimal spread parameter of 2.30 was used at the last hidden node. The prediction follows the pattern of background ozone obtained from actual measurements. The predicted values are varying from the minimum of 5ppb to a maximum of 35ppb, whereas the true values show a fluctuation between 5ppb to 30ppb.

VI. CONCLUSION

We have presented a new metamodel using adaptive radial basis function neural networks for predicting the hourly

0 0.2 0.4 0.6 0.8 1 1.2 1.40

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

Spread parameter (sp)

RM

SE value

1 neuron2 neurons3 neurons4 neurons5 neurons6 neurons7 neurons8 neurons

Fig. 1. RMSE value versus the RBFNN’s spread parameter with various neuron numbers from a case study.

RMSE

val

ue

background ozone level with a good accuracy. The results obtained indicate the promising application of the proposed method in the short-term analysis of the background ozone level and emission impact assessment for air quality modeling. The current methodology depends on a limited number of para-meters, namely time, NO, NO2 and O3 concentrations. To obtain a more comprehensive model, other parameters especially the meteorological data should be taken into consideration such as air temperature, wind speed and wind direction as they have a direct effect to the level of background ozone, and thus exploiting the capability of the proposed metamodelling approach in handling multiple parameters. For further work, the concentration of other anthropogenic components can also be used as input parameters to see if they have any influence to the accuracy of the prediction.

REFERENCES

[1] D.W. Fahey, “Twenty Questions and Answers About The Ozone Layer: 2006 Update,” Scientific Assessment of the Ozone Depletion: 2006.

[2] H. Duc, and M. Azzi, “Analysis of Background Ozone in the Sydney Basin,” Proc. 18th IMACS & MODSIM Congress, Cairns Australia 2009, pp. 2307-2313.

[3] U.S. Environmental Protection Agency (US-EPA), “Air Quality Criteria for Ozone and Related Photochemical Oxidants (2006 Final),” U.S. EPA, Washington, DC, EPA/600/R-05/004aF-cF, 2006.

[4] A.P. Altshuller and A.S. Lefohn, “Background ozone in the planetary boundary layer over the United States,” Journal Air Waste Management Association, Vol. 46, 1996, pp. 134-141.

[5] C. L. Blanchard, “Methods For Attributing Ambient Air Pollutants To Emission Sources,” Annual Review of Energy and the Environment, Vol. 24, 1999, pp. 329-365.

[6] S. Vardoulakis, B.E.A. Fisher, K. Pericleous, and N. Gonzalez-Flesca, “Modelling air quality in street canyons: a review,” Atmospheric Environment, Vol. 37, 2003, pp. 155-182.

[7] D. Byun and K.L. Schere, “Review of the governing equations, computational algorithm, and other components of the Model-3 Community Multiscale Air Quality (CMAQ) Modelling System,” Applied Mechanics Reviews EPA, Research Triangle Park, NC, 2004.

[8] R.E. Morris et al., “Recent advances in CAMx Air Quality Modelling,” Paper #934, 2001; http://www.camx.com/publ/

[9] Harvard University, Atmospheric Chemistry Modelling Group, 2004; http://acmg.seas.harvard.edu/geos/index.html

[10] A.H. Goldstein, D.B. Millet, M. McKay, L. Jaegle, L. Horowitz, O. Cooper, R. Hudman, D.J. Jacob, S. Oltmans, and A Clarke, “Impact of Asian emissions on observations atTrinidad Head, California,” Journal of Geophysical Research, D23S17, doi:10.1029/2003JD004406, 2004.

[11] A. Khosravi and S. Nahavandi, “Developing Optimal Neural Network Metamodels Based on Prediction Intervals,” Proc. Int. Joint Conf. Neural Networks, Atlanta, Georgia, USA, 2009, pp. 1583-1589.

[12] W.E. Biles, J.P.C Kleijnen, W.C.M. van Beers, I. van Nieuwenhuyse, “Kringing metamodeling in constrained simulation optimization: An explorative study,” Proc. 2007 Winter Simulation Conference,Washington D.C 2007, pp. 355-362.

[13] M.Papadrakakis, N.Lagaros & Y.Tsompanakis, “Structural Optimization Using Evolution Strategies and Neural Networks,” Computer Methods in Applied Mechanics & Eng., Vol. 156, 1998, 309-333.

[14] S. Crino and E. Brown, “Global optimization with multivariate adaptive regression splines,” IEEE Transactions on Systems, Man, And Cybernetics-Part B: Cybernetics, Vol. 37, No. 2, 2007, pp. 333-340.

[15] S. Varadarajan, W. Chen, and C. J. Pelka, “Robust Concept Exploration of Propulsion Systems with Enhanced Model Approximation Capabilities,” Engineering Optimiz., Vol. 32, No. 3, 2000, pp. 309-334.

[16] G. Wang and S. Shan, "Review of Metamodeling Techniques in Support of Engineering Design Optimization," Journal of Mechanical Design,Vol 129, 2007, pp. 370-380.

[17] C. Harpham, C.W. Dawson, “The effect of different basis functions on radial basis function network for time series prediction: A ccomparative study,” Neurocomputing, Vol. 69, 2006, pp. 2161-2170.

[18] H. Demuth, M. Beale and M. Hagan, “Neural Network ToolboxTM 6,” MATLAB User’s Guide, 2009.

[19] G. Poshal and P. Ganesan, “An analysis of formability of aluminium performs using neural network,” J. of Materials Processing Technology,Vol. 205, 2008, pp. 272-282.

[20] A.A. Giunta and L.T. Watson, “A comparison of approximation modeling techniques: polynomial versus interpolating models”, In Proc.7th Symp. on Multidisciplinary Analysis and Optimization, 1998.

[21] R.A. Jacob, “Increased rates of convergence through learning rate adaptation,” Neural Networks, Vol. 1, 1988, pp. 295-307.

[22] V. Anh, H. Duc and M. Azzi, “Modelling anthropogenic trends in air quality data”, J. Air & Waste Management, Vol. 47, 1997, pp. 66-71.

[23] H. Wahid, Q.P. Ha and H. Duc, “A Metamodel for background Ozone level using Radial Basis Function Neural Networks,” Proc. Int. Conf. Control, Automation, Robotics &Vision, Singapore Dec. 2010.

0 10 20 30 40 50 60 70 80 90

0

5

10

15

20

25

30

Case number

Bac

kgro

undo

zone

leve

l (pp

b)

Predicted values

Actual values

Fig. 3. Predicted background ozone level at Lidcombe in 1998.

0 10 20 30 40 50 60 70 80 90

5

10

15

20

25

30

35

Case number

Bac

kgro

undo

zone

leve

l (pp

b)

Predicted valuesActual values

Fig. 4. Predicted background ozone level at St. Mary in 2004.

0 10 20 30 40 50 60 700

5

10

15

20

25

30

Case number

Bac

kgro

undo

zone

leve

l (pp

b)

Predicted values

Actual values

Fig. 2. Predicted background ozone level at Blacktown in 1998.

Back

grou

nd o

zone

con

cent

ratio

n (p

pb)

Back

grou

nd o

zone

con

cent

ratio

n (p

pb)

Back

grou

nd o

zone

con

cent

ratio

n (p

pb)