8/12/2019 2237-7487-1-SM

1/16

Defence S cience Joum al, Vol51 ,No 3, July 2001, pp 263-278001, DESlDOC

Neural Network Parameters Affecting Image Classification

K.C. TiwariArmy Headquarters Kashmir House New Delhi

ABSTRACTThis studv is to assessthe behaviour and immof m io us neural network varame ters and their effects othe classificati& ccur cy of remotely sensed irk ge s which resulted in sueeessii~l lassification of an I R S lB

LISS 11 image of ROO&& and its su&unding &using eural netw ork classification technique s. The methodc nbem l i d orvariousdefence awlicat ions. suchas for the identificationofenemvtmor cuncenbationsand n.logistical planning in desertsby identification of suitable areas for vehicularmovem ent. Five p arameten, namelytraining sample size, nllmberofhid den layers, nu m bm of hidden nodes, learning rate and momentum factor wereselected. In each wse sets of values were decided based on earlier works ryorted Neural network-basedclassificationswerec m i e d o u t f o r a s m y as 45 0c omb ina t i on s o f t h e s e~ e r s . inally, agm phical analysisof the results obtained w s canied out to understand the relationship among these parameters. A table ofrecommended values for these parameters or achieving 90 per cent and higher classification accuracy wasgenerated andused inc lassificationofan IRS-1B LlSS I1 image. Theannlysis suggeststheexisten ceofan intricaterelationship among these parameters and calls for a wider series of classification experiments as also a moreintrica te analysis of the relationships..Keywords Rem ote sensing, digital image classification, artificial neural netwo rk technique, knowledge-basedclassification techniques, fuzzy techniques, m ulti-band remote sensing data

1. INTRODUCTIONOur planet, Earth is endowed with rich natural

classification algorithms are based on certainstatistical distribution assumptions, i.e. normal or

8/12/2019 2237-7487-1-SM

2/16

DEF SCI J, VOL 51, NO 3, JULY 2 1

T h e l i m i t a t i o n s o f t h e c o n v e n t i o n a lclassification techniques have heralded an era ofnewer and better techniques capable of handlingcomplex, multi-band data'. Some of these mostp o p u l a r t e c h n i q u e s a r e k n o w l e d g e - b a s e dclassification techniques2.', fuzzy techniques4, andartificial neural network technique^^ ^ Unlikeconventional classifiers, these are non-parametericin nature, as they do not depend on statisticalpre-assumptions about the distribution of data. Inaddition, these classifiers have show n the capabilityof handling multi-source data and so have foundinc reas ing accep tance in c l a ss i f i ca t ion o fmulti-band remote sensing data, particularly whendata from other ancillary souices are also used togenerate quali tat ive classificat ions. Variouspara'meters involved in one such new techniquecurrently under active research, namely theartificial neural network technique, has beenstudied.2 STUDY AREA DATA

An IRS-IB LISS I1 512 512 image of19 January 1992 in four spectral bands (three invisible and one in near-infrared region) of Roorkeeand its surrounding areas has been selected for thisstudy (Fig. 1). The area is largely dominated by

vegetation of various types with a river m eanderingin its vicinity. The river has produced large patchesof sandy areas. The centre of the city is largelybuilt-up area and spreads right up to the peripheriesof the city.

Survey of India map of Roorkee area at scale1:50,000 (map sheet No 53-Gl13) and aerialphotograp hs at scale 1:25,000 of thesam e area wereused as reference data for selecting training andtesting data. The same was also confirmed throughground visits. Selection of training and testing datawas done displaying the image on a computermonitor using intergraph1s image processingsystem. F ive different classes dominant in the area,namely built-up, water, sand, vegetation andmiscellaneous were extracted from the image usingthe reference data. The miscellaneous classconstituted pixels, which did not fall in any of theclasses selected. Extraction of data for variousclasses was done by generating a number ofpolygons well spread over the entire image. Theidentity of a particular class was determined usingthe reference data. A total numb& of pixelscontained in the polygons for different classesvaried from '600 to 1,000.The data thus generated was segregatedrandomly into training and testing data. While thetesting sample size was fixed for each class, thetraining sam ple size- as varied since it was one ofthe various parameters under investigation. A totalof 300 pixels were selected in each class as testing

8/12/2019 2237-7487-1-SM

3/16

TIWARI NEURAL NETWORK PARAMETERS FFECTING IMAGE CLASSlFlCAnON

the effect of various neural network parameters andtheir interdependency to make a suitablerecommendation for their judicious selection.Although, there are several parameters which affectthe classification performance of ANN asclassifiers, the effect of only the followingparameters have been investigated in the presentstudy:2.1.1 Training Sample Size

Initially 300 pixels per class (viz., built-up,water, sand, vegetation and miscellaneous class)were selected as testing pixels and then from therest of the data, five different tEaining sample slzeswere created varying in size' from as low as 10pixels per class to as hlgh as 200pixels per class anddesignated asTraining sample size TSS 10 10 pixels per classTraining sample size TSS 50 50 pixels per classTraining sample size TSS 100 100 pixels per classTrain~ngample slze TSS 150 150 pixels per classTralnlng sample size TSS 200 200 pixels per class

The data presented a case of nonlinearclassification and the miscellaneous class furtherintroduced sufficient confusion because of thepresence of many different classes. This was doneso that the selected data could act as a true litmustest for the ANNs capability as a classifier.2.1.2 Number of Hidden Layers Hidden o es

In remote sensing literature, studies using both

hidden layers. The architecture for two hiddenlayers was kept as 212, 414 818, 12/12 and 16/16(number of hidden nodes in first and second hiddenlayers, respectively).2.1.3 Learning Rate Momentum Factor

Being faster and supenor, adaptlve learningrate approach of back propagation learning rule wasadopted for training the nodes. This required inputof only the initial value of the learning rate.Thereafter, at the end of each epoch, if the learningrate produced error more than a pre-defined ratio, itwas discarded and a lower value o learning ratewas selected, otherwise learning rate value wasincreased to speed up learning. Three differentvalues of starting learning rates were tested. Theupper and lower range of these learning rate valueswere determined using the empirical formula givenby Heerman i d Khazenie in 1992. The formula isLr Co/P.N where Lr is learning rate, P is numberof samples (or patterns), N is total number of nodesin the network and Co is a factor experimentallydeterminedq which is equal to 10. The learningrates, thus determined and selected for thc s.t,udyiy : ..were 0.04, 0.004 and 0.0004. Three rri entumfactor values chosen in the range 0 to 1 were 0.15,0.55 and 0.95.3 METHODOLOGY

The main objectlve of this study was to assessthe behaviour and impact of various ANN

8/12/2019 2237-7487-1-SM

4/16

DEE SCI J VOL 51 O 3 N L Y 2 1

3.2 Inpu t Data EncodingDifferent data files for various training and

testing sample sizes were created. To ensu re that theinput and output data matched well with each other,input data was normalised in the range 0 to 1. Thenormalisation of input data was done separately foreach training sample size at the time o f training bymultiplying the input data by an encoding factor.This encoding factor (EF) was determined using thefollowing formula:DN, DNmimEF = DN rn, DN miwhere DN, denotes the individual pixel value andDN,, and DN,,, are the maximum and min imumpixel values, respectively.3 3 Target Data EncodingAlthough in remote sensing literature, variousschemes of target data encoding have been reported,however, for the present study, it was decided toconstitute the target data set using only twonum bers, 1 and 0. The code consisted of five digits(one for each class). The leftmost digit representedfirst class and the second leftmost digit representedsecond class and so on. A target value of 1 signifiedidentification with a particular class and a targetvalue of 0 indicated otherwise. Thus, a value of 1 asfirst leftmost digit-meant identification with firstclass. It was not poss ible to have value of 1 at anytwo locations in the five digit code. For exam ple, anoutput node representing class 3 was encoded as

method with a fixed starting learning rate of0.000267 and mom entum factor of 0.95. Separatetraining was canied out for different hidden nodecategories. These weights were then used as fixedstarting weights in all subsequent classificationexperiments.3 5 Other Pa ramete rs

Separate experimen ts using all the three valuesof learning rate and m omentum factors were carriedout. All the training were comm enced by randd nlypegging the error goal at 0.01. Sum squared errorand last iteration error were noted for eachclassification. In all the experiments, training wascarried out for a fixed number of epochs(2500 epochs) and no attempt was made to seekconvergence to the error goal fixed.3 6 Classification

In the remote sensing literature, dependingupon the type of input and output data 'encoding,several classif ication strategies have beenproposed9.

In the present study, the target outputs fortraining were constituted using two num bers, i.e. 0and 1 only. How ever, the values of classified outputdata ranged between 0 and I( in decimals) for eachclass. Therefore, a pixel was assigned to a classwhich had the highest output value.3 7 Accuracy Assessment

The accuracy of classifications were assessed

8/12/2019 2237-7487-1-SM

5/16

TIWARI: NEURAL NETWORK PARAMETERS AFFECI DJG IMAGE CLASSIFICATION

4. RESULTS DISCUSSIONThe objective of the study was to invest~gate

the effects of various neural network param eters onthe accuracy of image classification. Although,there are several parameters which affect theclassification accuracy, only five parameters wereinvestigated in this study . These parameters and therange in which these parameters w ere studied werearrived at after a thorough literature surv ey' ofneural network applications in the field of remotesens ing . A to ta l o f 450 neura l ne tworkclassifications were performed (225 classificationseach in one and two hidden layer categories). Theovera l l accu racy 'was esse ssed fo r eac hclassification with a testing sample size of 300pixels per class. The results were analysed' usingsimple gaph ica l analysis carried out separately forone and two hidden layer catego ries. When effect oftraining sample size was stu died, experiments for agiven number of nodes (in case of two layers, thenumber of nodes were same in first and secondlayer) were grouped together. Since 'there werethree values each of learning rate and momentumfactor, there were nine different experiments. Th esewere differentiated using three different coloursand three different line types (Figs 2 and 3). Asimilar approach was adopted when effect ofnumber of nodes was studied except that hereexperiments for a given training sample size weregrouped together and plotted (Figs 4 and 5).However, when learning rate and m omentum factor

to 3(e)]. Th e following observations regarding thebehaviour of training sample size were made:In general for one hidden layer category,overall accuracy shows improvement with the

increase in training sample size. However, thisimprovement is not gradual and continuous.Between tw o sam ple sizes, it decreases temporarilyat times but the frequency of decrease reduces athigher number of hidden nodes where theimprovement in overall accuracy is continuousthroughout with minor fluctuations. Hence, thereexis ts a threshold value for sample size for differenthid de n. node categories below which ovetallaccuracy cannot give good results.

A sim ilar trend is noticed m case of two hiddenlayer category also. However, the curves in thiscase are relatively sm oother than in the ease of oneh ~ d d e n ayer category. In other words, there is anear-continuous improvement in overall classl-fication accuracy w ~ t hhe increase in sample size.4.2 Effect of Number of Hidden Nodes

The number of h~dden odes In one hlddenlayer category were selected as mult~plesof thenumher of Input nodes. Thus, 2, 4, 8 12 and16 h~ dd en ddes (112, 1 2, 3, 4 mu lt~pl es)wereselected for one h~ dd enayer category. The numberof h~ dd en odes were kept same In both the first andsecond h~ dd en ayer for the two hldden layercategory. Thus, the number of h~ dd en odes were2,4,8,12 and 16m each of the h~ dd enayers.

8/12/2019 2237-7487-1-SM

6/16

-DEF SCI J, VOL 51, NO 3 JULY 2Mll

68 NODES 2

0 40 80 I20 160 200T RA I N I N O ' SA M PL E lZ E

'1 NODES 4

0 40 80 120 I60 200TRAINING SAMPLE SrZE

9 0 b)

0 40 X I20 160 200TRAINING SAMPLE SlZ EC )

NODES I6

J? -0 40 80 120 160 200

TRAINING SAMPLE SIZE(11)

8/12/2019 2237-7487-1-SM

7/16

TIWARI. NEURAL NETWORK PARAMETERS AFFECTMG IMAGE CLASSIFICATION

] NODES 818

22

; 21