Available online at www.sciencedirect.com

www.elsevier.com/locate/asr

Advances in Space Research 42 (2008) 1469–1474

Application of support vector machine combinedwith K-nearest neighbors in solar flare and solar proton

events forecasting

Rong Li *, Yanmei Cui, Han He, Huaning Wang

National Astronomical Observatories, Chinese Academy of Sciences, 20A Datun Road, Chaoyang District, Beijing 100012, China

Received 1 November 2006; received in revised form 29 November 2007; accepted 30 December 2007

Abstract

The support vector machine (SVM) combined with K-nearest neighbors (KNN), called the SVM-KNN method, is new classingalgorithm that take the advantages of the SVM and KNN. This method is applied to the forecasting models for solar flares and protonevents. For the solar flare forecasting model, the sunspot area, the sunspot magnetic class, and the McIntosh class of sunspot group and10 cm solar radio flux are chosen as inputs; for the solar proton event forecasting model, the inputs include the longitude of activeregions, the flux of soft X-ray, and those for the solar flare forecasting model. Detailed tests are implemented for both of the proposedforecasting models, in which the SVM-KNN and the SVM methods are compared. The testing results demonstrate that the SVM-KNNmethod provide a higher forecasting accuracy in contrast to the SVM. It also gives an increased rate of ‘Low’ prediction at the same time.The ‘Low’ prediction means occurrence of solar flares or proton events with predictions of non-occurrence. This method show promisefor forecasting models of solar flare and proton events.� 2008 COSPAR. Published by Elsevier Ltd. All rights reserved.

Keywords: Feature space; Prediction accuracy; Separating hyperplane

1. Introduction

Solar flares explosively release radiations covering awide wavelength range, which heat up the terrestrial atmo-sphere within minutes and make satellites to drop intolower orbits. Solar energetic particles accelerated to near-relativistic energies during major solar storms severelyendanger astronauts in the interplanetary space (Schwenn,2006). For this reason, the forecasts of solar flares and pro-ton events have long been listed in important products ofspace weather service.

Several methods are applied to solar flare forecasting.Based mainly on McIntosh classification, an expert systemcalled ‘Theophrastus’ was developed and adopted in 1987

0273-1177/$34.00 � 2008 COSPAR. Published by Elsevier Ltd. All rights rese

doi:10.1016/j.asr.2007.12.015

* Corresponding author. Tel.: +86 10 13581911395.E-mail address: rongli501@hotmail.com (R. Li).

as a tool in the daily operations of the Space EnvironmentCenter (SEC) (McIntosh, 1990). At Beijing AstronomicalObservatory, Zhang and Wang (1994) developed a multi-discrimination method based on observations of sunspots,10 cm radio flux and longitudinal magnetic fields. Galla-gher et al. (2002) at Big Bear Solar Observatory proposeda flare prediction system which estimated the probabilityfor each active region to produce C-, M-, or X-class flaresusing historical averages of flare numbers by McIntochclassifications. For proton event forecasting, SEC devel-oped a model that can provide proton event probability,peak flux, and rise time. Neural network methods havebeen proposed and presented a good performance (Wang,2000; Patrick et al., 2002; Gong et al., 2004).

The support vector machine (SVM), as a good alterna-tive method of neural network, has been successfullyapplied to different fields because of its ability to toleratehigh-dimension and/or incomplete data. In the fields of

rved.

1470 R. Li et al. / Advances in Space Research 42 (2008) 1469–1474

space weather, the SVM has been applied to the geomag-netic substorm forecast (Gavrishchaka and Ganguli,2001). Despite the impressive general performance of theSVM, a number of problems remains to be solved, suchas low classifying accuracy for complicated applications.In order to solve these problems, a new SVM classifyingalgorithm (SVM-KNN method) has been proposed, whichcombines SVM with K-nearest neighbor (KNN) and dem-onstrates excellent performance (Li et al., 2002).

In this paper, the SVM-KNN method is applied to fore-casting models for solar flares and proton events. In follow-ing sections, we described the SVM-KNN classifyingmethod in Section 2, the data and predictions in Section3, the models in Section 4, the testing results in Section 5,and the summary and discussion in Section 6.

2. Methods description

Support vector machine, proposed by Vapnic in 1995, isa new promising pattern classification technique based onthe structural risk minimization principle. For a givenlearning task, with a given finite amount of training data,the best generalization performance will be achieved ifthe right balance is struck between the accuracy attainedon that particular training set, and the ability of themachine to learn any training set without error (Burges,1998).

SVM aims at minimizing an upper bound of the gener-alization error through maximizing the margin between theseparating hyperplane and the data (Amari and Wu, 2001).Then supposing in the case when the data are nonlinearlyseparable, for training set ðx1; y1Þ � � � ðxl; ylÞ belonging totwo different classes, the problem of searching the optimalhyperplane amounts to finding the adjustable coefficient ai

maximizing the Lagrangian function with constraints(Vapnik, 1995):

W ðaÞ ¼Xl

i¼1

ai �1

2

Xl

i;j¼1

aiajyiyjkðxi; xjÞ; ð1Þ

0 6 ai 6 C; i ¼ 1; � � � ; l;Xl

i¼1

aiyi ¼ 0: ð2Þ

The variable C in Eq. (2) is a given value which need to beset in the training algorithm. kðx; x0Þ ¼

Pi/ðxÞ/ðx0Þ is the

kernel functions and represents the dot products betweenmapped pairs of input points. /ðxÞ is a nonlinear mappingfrom a input space to a higher-dimensional feature space.Those sample points having ai > 0 are called support vec-tor locating near the hyperplane.

The separating rule is the following discriminantfunction:

f ðxÞ ¼ sgnXi2SV

yiaikðxi; xÞ � b

!: ð3Þ

where xi is support vector, ai is Lagrangian coefficient and b

is a constant (threshold).

The 1-nearest neighbor (1NN) classifier is an importantpattern recognizing method based on representative points(Bian and Zhang, 2000). In the 1NN algorithm, whole trainsamples are taken as representative points and the dis-tances from the test samples to each representative pointare computed. The test samples have the same class labelas the representative point nearest to them. The KNN isan extension of 1NN, which determines the test samplesthrough finding the K-nearest neighbors.

The SVM-KNN method is based on a theoretic relationbetween the SVM and the KNN, which is expatiated on atheorem.

Theorem 1. SVM classifier is equal to a 1NN classifier

which chooses one representative point for support vectors in

each class (A detailed proof can be found in Li et al., 2002).

Guided by this theorem, SVM and KNN classifyingalgorithm is applied, respectively, to testing sample accord-ing to its distribution in a feature space. For test samplesfar from separating hyperplane, the SVM classifying algo-rithm is available; while for samples close to hyperplane,the KNN classifying algorithm is suitable. Main steps ofnew classified algorithm are given as follows:

Step1: if test set T test is not empty, get x from T test; if T test

is empty, stop;Step2: calculate the distance difference g(x) from test sam-

ple x to representative points of two class,gðxÞ ¼

Piyiaikðxi; xÞ � b;

Step3: if jgðxÞj larger than a given constant e, put test sam-ple x into SVM classifying algorithm; if jgðxÞj smal-ler than e, put it into KNN algorithm to classify;

Step4: take out x from T test, go to Step 1.

3. Data and predictors

The data for X-ray flares are from GOES satellite, whichcan be downloaded at http://www.ngdc.noaa.gov/stp/solar/ftpsolarflares.html. In our study, a proton event isdefined as occurring if proton flux with energyEp > 10 MeV is greater or equal to 1 particle per squarecentimeter per second per steradian(1 pfu) up to the back-ground level near 1 AU. One-hundred and twenty-fourproton evens are identified occurring by searching plotsof GOES integrated proton relying on the definition ofsolar proton events. The data of solar active regions usedin our forecast model span from January of 1996 toDecember of 2004, which are downloaded from SEC website: http://sec.noaa.gov/ftpmenu/forecasts/SRS.html. Weselect each of active regions observed as a sample. Then,we have 19,817 samples in total.

Predictors of the flare prediction model include the areaof sunspot groups, the magnetic classification of sunspotgroups, the McIntosh classification of sunspot groupsand the 10 cm solar radio flux. Considering the initial valueof predictors is unfitted for the inputs of forecasting models

( )1222 ..10 −−−zHmw

).( 1.2 −− smw

Fig. 1. Solar flare productivity with the area of sunspots and the solarradio flux; proton productivity with the soft X-ray.

R. Li et al. / Advances in Space Research 42 (2008) 1469–1474 1471

directly, We make a statistical on the correlation of thesepredictors with solar flare by calculating their flare produc-tivities. For the area of sunspot and the solar radio flux,their flare productivities are defined as

P ðX Þ ¼ SaðX Þ=StðX Þ; ð4Þwhere SaðX Þ and StðX Þ are the number of the flare occur-rence samples and that of the total samples when valuesof predictors are in the range ½X ;1� (Cui et al., 2006). AGaussian function is used to fit the data curve in Fig. 1,which is given as follows

Y ¼ A1 þA2ffiffiffiffiffiffi2pp

W� exp�ðX � X 0Þ2

2W 2: ð5Þ

For the magnetic class and the McIntosh class of sun-spot group, their flare productivities can be estimated withthe ratio between the number of flare bursts and that oftotal samples in a corresponding class. The figures of flareproductivities are shown in Fig. 2.

Similar to the process of the flare productivity, the pro-ton event productivities with the longitude of active regionsand the soft X-ray flux are calculated. We divide the wholelongitude into several small sections and calculate the ratiobetween the number of proton event occurrence samplesand that of the total in each section. Fig. 2 shows thatthe most of proton events are related to active regionslocated around 50�W. The correlation of the proton pro-ductivity with the soft X-ray flux is fitted with Boltzmannfunction

Y ¼ A2 þA1 � A2

1þ expðX � X 0Þ=W: ð6Þ

The plots of the fitting function are presented in Fig. 1.The parameters of all fitting function are listed in Table 1.

4. Models

Applying SVM-KNN algorithm into our problem isbased on the understanding that the forecasting modelsfor solar flares and proton events can be formalized to apattern recognition problem. Input variable of flare predic-tion model is xi ¼ ðxi1; xi2; xi3; xi4Þ, which corresponds to thepredictors. Specifically, xi1 means the area of sunspotgroups; xi2 means the magnetic classification of sunspotgroups; xi3 means the McIntosh classification of sunspotgroups; xi4 means 10 cm solar radio flux. The predictorsare valued according to their flare productivities. The out-put yi refers to a classification divided by the importance ofsolar flares occurring within the coming 48 h, which con-tains two cases: greater or equal to M which correspondsto yi ¼ þ1, and lower than M which corresponds toyi ¼ �1.

The forecasting model of proton events is constructedon the assumption that there is a relationship between solarflare emissions and proton events. The inputs of the modelare represented as a variable xi ¼ ðxi1; xi2; xi3; xi4; xi5; xi6Þ.xi1; xi2; xi3; and xi4 are the same as those in the flare predic-

tion model. The xi5 is the longitude of the active region andthe xi6 is the soft X-ray flux, both of which are valued bytheir proton event productivities. Each input variable cor-responds to a output variable yi 2 ð�1;þ1Þ. The output,+1, means that the active region charactered by inputs

Fig. 2. Solar flare productivity with the magnetic class and the McIntosh class; proton productivity with longitude.

1472 R. Li et al. / Advances in Space Research 42 (2008) 1469–1474

Table 1Fitting function parameters table

Predictors Function A1 A2 X 0 W

Area Gauss �0.039 1708.06 1631.63 1597.52Solar radio flux Gauss 0.024 77.32 248.31 138.38Soft X-ray flux Boltzmann �15.57 0.98 �0.46 0.16

R. Li et al. / Advances in Space Research 42 (2008) 1469–1474 1473

produces a flare in a looking forward period of 24 h andthis flare results in a proton event hours later, and the out-put, �1, means that there are no proton events observedduring the looking forward period.

The data of training set contain input xi and output yi,where i 2 ð1; 2; � � � ; lÞ and l is the total number of trainset. In the training process, xi and yi are taken into Eq.(1) to obtain the coefficient ai, which is the input of classi-fying algorithm.

5. Testing results

Two testing sets are chosen to verify flare and protonforecasting models. The first one includes all of data con-cerning about the inputs and outputs of the model in2002, and the second in 2004. The two testing sets have atraining set respectively that begins in January of 1996and ends in December of the year before the testing set.

The model provide a prediction for each of activeregions. In order to have a same format with the daily rou-tine prediction, a flare and proton event prediction is needto give once a day. we define the event occurrence status ofa day as occurrence when there exists at least one activeregion predicted to be related to event occurrence.

For testing set, SVM and SVM-KNN method areapplied and compared here. The SVM-KNN algorithmmodified from LIBSVM described by Chang et al. (2001),in which the distance threshold is set to be 1 and the num-ber of nearest neighbors is set to be 5. Gaussian Radial

Basis function, which is given by Kðx; xiÞ ¼ exp � jx�xij2r2

� �,

is used as kernel function. The training parameters r andC are adjusted for optimal result in the SVM and SVM-KNN model respectively. The adjustment method is thatwe try the parameter ‘C’ from 1 to 5000 to get a high accu-racy, then fix the parameter C and try the ‘r’ to get thehighest accuracy Test results are showed in Table 2–5.

In above tables, column of ‘Predic.’ refer to the totalnumber of predictions respectively. ‘Equal’ is the numberof correct predictions, which include the correct predictionnumber for the flare and the proton event occurrence andnot occurrence. The ‘High’ means that prediction shows

Table 2Solar flare testing results of various methods for 2002

Methods Predic. Equal High

SVM 363 207 138SVM-KNN 363 232 76

that solar flares or proton events will occur and observa-tion shows that they do not occur, and conversely the‘Low’. The last three columns are rates of ‘equal’, ‘High’and ‘Low’ in the total.

As demonstrated in these tables, the SVM-KNNmethod presents a higher rate of ‘Equal’ and a lower rateof ‘High’ than the SVM method in both flare and protonevent prediction models. That is because the SVM-KNNmethod can give a correct classification for samples whichare close to hyperplane. In the flare prediction model, someactive regions with large area and complicated structure arenon-flare occurrence cases, which are misclassified as flareoccurrence cases by the SVM method. For example, theactive region 9767 in January 01 in 2002 did not burst aM flare in the coming 24 h, which has a area of 510, themagnetic class of ‘c’, the McIntosh class of ‘eki’, and thesolar radio flux of 246. The active region is located nearhyperplane and predicted correctly using the SVM-KNNmethod as non-flare occurrence active region.

It is noted that the rate of ‘Low’ is slightly increasedfrom the SVM-KNN in comparison with that fromSVM. This fact can be explained as follows. The numberand value range of non-flaring samples is larger than thatof flaring samples, which means the non-flaring samplesare more spread-out in the feature space than the flaringsamples. According to the SVM-KNN algorithm, samplesnear the separating hyperplane take part in the classifica-tion, the more spread-out distribution makes the non-flar-ing samples in the training set slightly more attractive tothe samples in the testing set, which results in a slightlyincrease of Low predictions.

6. Summary and discussion

The SVM-KNN method is a new classifying algorithmby combining the SVM with the KNN algorithm accordingto the distribution of the test samples in a high dimensionspace. This method is applied to forecasting models forflares and proton events, in which training set constitutedby inputs and outputs are put into the SVM training algo-rithm to obtain prediction models. The predictors includ-ing the sunspot area, the sunspot magnetic class, and theMcIntosh class of sunspot group and 10 cm solar radio fluxare taken as inputs of the models. The occurrence and non-occurrence of solar flares and proton events can be pre-dicted with the these models. Verified by two years of test-ing data, these Models show a higher forecasting accuracyusing the proposed new method than that only supportedby the SVM. At the same time, however, it also gives an

Low Equal (%) High (%) Low (%)

18 57.02 38.02 4.9655 63.91 20.94 15.16

Table 3Solar proton events testing results of various methods for 2002

Methods Predic. Equal High Low Equal (%) High (%) Low (%)

SVM 339 314 6 19 92.63 1.76 5.6SVM-KNN 339 316 0 23 93.21 0 6.78

Table 4Solar flare testing results of various methods for 2004

Methods Predic. Equal High Low Equal (%) High (%) Low (%)

SVM 360 242 97 21 67.22 26.94 5.83SVM-KNN 360 269 53 38 74.72 14.72 10.56

Table 5Solar proton events testing results of various methods for 2004

Methods Predic. Equal High Low Equal (%) High (%) Low (%)

SVM 354 318 27 9 89.83 7.63 2.54SVM-KNN 354 339 2 13 95.16 0.78 3.67

1474 R. Li et al. / Advances in Space Research 42 (2008) 1469–1474

increased rate of ‘Low’ prediction, which is not alwaysdesirable. The testing results demonstrate that the SVM-KNN is a promising method for forecasting models ofsolar flare and proton events.

In the current models, we only used the conventionalpredictors to forecast the solar flares and proton events.Some new predictors need to be adopted. For example,solar photospheric magnetic parameters can be employedto be new predictors. Cui et al. (2006) proved that this theseparameters are closely related to the solar flare productiv-ity. For the forecast model of proton events, we shouldconsider interplanetary magnetic fields and CMEs, whichare well known factors causing proton events in the nearearth space.

Acknowledgements

This work is supported by National Natural ScienceFoundation of China (NSFC) under Grants 10233050and 10673017, by Chinese Academy of Sciences GrantKGCX3-SYW-403-10 and by National Ministry of Scienceand Technology under Grant 2006CB806307.

References

Amari, S., Wu, S. Improving Support Vector Machine Classifiers byModifying Kernel Functions. The Institute for Physical and ChemicalResearch, Hirosawa, Japan, 2001.

Bian, Z.Q., Zhang, X.G. Pattern Recognition. Tsinghua University Press,Beijing, 2000.

Burges, C.J.C. A tutorial on support vector machines for patternrecognition. Data Min. Knowl. Disc. 2 (2), 1998.

Chang, C.C., Lin, C.J. LIBSVM: A Library for Support Vector Machines(Version 2.3.1). Available from: <http://citeseer.ist.psu.edu/chang01libsvm.html/>, 2001.

Cui, Y.M., Li, R., Zhang, L.Y., He, Y.L., Wang, H.N. Correlationbetween solar flare productivity and photospheric magnetic fieldproperties. Sol. Phys. 237, 45–59, 2006.

Gallagher, P.T., Moon, Y.-J., Wang, H.M. Active-region monitoring andflare forecasting. Sol. Phys. 209, 171, 2002.

Gavrishchaka, Valeriy V., Ganguli, S.B. Support vector machine as anefficient tool for high-dimensional data processing: application tosubstorm forecasting. J. Geophys. Res. 106, 29911–29914, 2001.

Gong, J.C., Xue, B.S., Liu, S.Q., Zou, Z.M., Juan, M., Wang, J.L. Short-term prediction of solar proton events by neural network method.Chinese Astron. Astrophys. 28, 174–182, 2004.

Li, R., Ye, S.W., Shi, Z.Z. SVM-KNN classifier – a new method ofimproving the accuracy of SVM classifier. Chinese J. Electron. 30 (5),745–748, 2002.

McIntosh, P.S. The classification of sunspot groups. Sol. Phys. 125, 251–267, 1990.

Patrick, P., Gabriel, S., Rodgers, D., Clucas, S. Neural network predictionof solar proton events with long lead times, in: Huguette, Sawaya-Lacoste (Ed.), Proceedings of the Second Solar Cycle and SpaceWeather Euroconference, 24–29 September 2001, Vico Equense, Italy,ESA SP-477. ESA Publications Division, Noordwijk, pp. 517–520,ISBN 92-9092-749-6, 2002.

Schwenn, R. Space weather: the solar perspective. Living Rev. Solar Phys.3, 2, 2006.

Wang, J.L. A two-pass neural network model for solar proton event alert.Chinese Astron. Astrophys. 24, 10–12, 2000.

Vapnik, V. The Nature of statistical Learning Theory. Springer-Verlag,New York, 1995.

Zhang, G.Q., Wang, J.L. Extended compendium about now scheme ofshort-term prediction of X-ray flares. Prog. Geophys. 9, 54, 1994.