a pixel shape index coupled with spectral information for … · 2011-12-26 · resolution remotely...

2950 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 44, NO. 10, OCTOBER 2006

A Pixel Shape Index Coupled With SpectralInformation for Classification of High Spatial

Resolution Remotely Sensed ImageryLiangpei Zhang, Xin Huang, Bo Huang, and Pingxiang Li

Abstract—Shape and spectra are both important features ofhigh spatial resolution remotely sensed (HSRRS) imagery, andthey are concrete manifestation of textures on such imagery. Thispaper presents a spatial feature index, pixel shape index (PSI), todescribe the shape feature in a local area surrounding a pixel. PSIis a pixel-based feature which measures the gray similarity dis-tance in every direction. As merely the shape feature is inadequatefor classifying HSRRS imagery, a transformed spectral featureextracted by independent component analysis is added to the inputvectors of our classifier, and this replaces the original multispectralbands. Meanwhile, a fast fusion algorithm that integrates bothshape and spectral features using the support vector machinehas been developed to interpret the complex input vectors. Theresults by PSI are compared with some spatial features extractedusing wavelet transform, gray level co-occurrence matrix, and thelength–width extraction algorithm to test its effectiveness. Theexperiments demonstrate that PSI is capable of describing shapefeatures effectively and result in more accurate classifications thanother methods. While it is found that spectral and shape featurescan complement each other and their integration can improveclassification accuracy, the transformed spectral components arealso found to be more suitable for classification.

Index Terms—Independent components analysis (ICA),integration of shape and spectra, shape feature, support vectormachine (SVM).

I. INTRODUCTION

H IGH spatial resolution remotely sensed (HSRRS) imageswith multispectral bands such as QuickBird and IKONOS

provide a large amount of information, thus opening up avenuesfor new remote sensing applications. However, their availabilityposes challenges to image classification. Due to the complexspatial arrangement and spectral heterogeneity even withinthe same class, conventional spectral classification methodsare grossly inadequate for HSRRS imagery [1]. In order toovercome this inadequacy, spectral features must evidently becomplemented by one or the other means. It is by and largeagreeable to not only use the spectral information, but also to

Manuscript received October 26, 2005; revised April 11, 2006. This workwas supported by the National Natural Science Foundation of China underGrant 40471088, 40523005, the 973 Program of the People’s Republic of Chinaunder Grant 2006CB701302, and the Foundation of State Key Laboratory ofInformation Engineering in Surveying, Mapping, and Remote Sensing underGrant 904151695.

L. Zhang, X. Huang, and P. Li are with the State Key Laboratory ofInformation Engineering in Surveying, Mapping, and Remote Sensing, WuhanUniversity, Wuhan 430079, China (e-mail: [email protected]).

B. Huang is with the Department of Geography and Resource Management,The Chinese University of Hong Kong, Shatin, N.T., Hong Kong.

Digital Object Identifier 10.1109/TGRS.2006.876704

exploit spatial analysis [2]. Spatial analytical approaches cansimply be categorized into spatial features extracted by movingwindows or elements and a spatial classifier based on contextualdecision criteria with consideration of neighboring pixels insidethe classifier [2]. This paper focuses on the spatial features tostrengthen the feature space for better classification accuracies.

Several effective spatial features that have been proposed,e.g., wavelet transform (WT) methods, are used to extract tex-ture features [1], [3]. WT allows extraction of different texturefeatures at different scales, thus providing a useful alternativefor the spatial features analysis of HSRRS images. The graylevel co-occurrence matrix (GLCM) method can introduce spa-tial information into a spectral classification, in which an imagegray value is transformed into the co-occurrence matrix spaceand various output images are calculated adopting differentspatial measures [4], [5]. The Gaussian Markov random field(GMRF) model, which is an accurate and compact versionof the Markov random field (MRF) [6], [7], is yet anotherrepresentative method for extracting spatial information. No-tably, Shackelford and Davis [8] proposed an effective algo-rithm, length–width extraction algorithm (LWEA), to extractthe length and width of spectrally similar connected groupsof pixels. The notion of LWEA is akin to GLCM, whichalso measures the spatial features within a spectrally similarneighborhood along a certain direction. In addition to theaforementioned methods, a series of spatial features extractedby morphological profiles (MPs) and differential morphologicalprofiles (DMPs) was proposed [9]–[11]. These morphologicalfeatures have proven to be effective in extracting multiscalestructural information from HSRRS images and, hence, com-plement spectral information.

Based on the aforementioned work, this paper investigatesthe following three issues pertaining to spectral and spatialfeature extraction and their fusion in the classifier.

1) In addition to textural features, HSRRS imagery alsoincludes ample shape features. The complex raster datastructure of remotely sensed images makes the shapedescription of objects a nontrivial task. Especially, asthe spatial resolution increases, sharper spatial featuresmust be interpreted and captured effectively. This paper,therefore, puts forward the pixel shape index (PSI), animproved version of LWEA, to describe the spatial infor-mation around the central pixel in its entirety.

2) Quite frequently, multispectral classification utilizes theoriginal spectral bands as the input for classifiers.

0196-2892/$20.00 © 2006 IEEE

ZHANG et al.: PSI COUPLED WITH SPECTRAL INFORMATION FOR CLASSIFICATION OF HSRRS IMAGERY 2951

Nevertheless, it is very difficult to achieve good classi-fication results this way, as most of the land cover classesin an HSRRS image contain a number of spectrally dif-ferent pixels or objects [1]. Thus, it would be interestingto investigate if some transformed spectral features canachieve higher accuracy than the original multispectralHSRRS images. For this purpose, the feature extractionmethod of independent component analysis (ICA) is em-ployed. ICA is an effective approach for blind sourceseparation and has drawn a great deal of attention owingto its tremendous potential in image processing.

3) High-resolution imagery reveals objects in greater detail,depicting their size, shape, structure, and color. Unlikemedium-resolution images, a single feature is not suffi-cient for classifying the high-resolution imagery. Thus,interpretation of such images entails multiple effectivefeatures. An appropriate classifier and better decisionrules are inevitable for identifying the multiple inputfeatures. Unlike the multispectral classification method[2] and the classifiers fusion approach [12]–[14], themultiple spectral and spatial features in this paper areinput into the support vector machine (SVM), a relativelynew method of machine learning. The notable advantagesof SVM include self-adaptability, swift learning pace,and high-dimensional property in feature space. SVMcan also reduce the dominance effects of the spectralinformation if the input vector contains both spectral andspatial features. This SVM approach of a single classifierwith multiple features is chosen because of its fast andeffective processing.

The rest of this paper is organized as follows. Section IIdescribes the PSI algorithm. Section III discusses ICA trans-formation for the original spectral bands. Section IV describesthe fusion of spectral and spatial features based on SVM, andSection V details the two experiments with two multispectralHSRRS datasets. The first experiment detects the influence ofthree parameters on the PSI algorithm, and the second exploresfusion of the shape and spectral features. Finally, Section VIconcludes the paper.

II. PIXEL SHAPE INDEX

Shackelford and Davis [8] proposed LWEA to examine thecontext of each pixel and measure the spatial dimensions ofgroups of spectrally similar connected pixels. LWEA calculateda length and width value for each pixel in the image. Thesevalues are found by searching along a predetermined numberof equally spaced lines radiating from the central pixel.

PSI adds an extension to LWEA. The shape features areextracted in a pixel-by-pixel manner, wherein each pixel has afeature value. The shape represents the contextual feature alongall the directions to some predefined limit around the centralpixel. Like LWEA and GLCM, PSI is also based on the spectralsimilarity of its neighboring pixels.

PSI shape feature extraction consists of three steps.1) extension of direction lines based on gray level similarity;2) measurement of the length of each direction line;3) calculation of PSI.

Fig. 1. Direction lines of PSI algorithm.

A. Extension of Direction Lines

The extended direction lines used to detect the object’soverall contour are symmetric around the centric pixel. In onechosen direction, the spectral difference is measured between apixel and its centric pixel in order to decide if this pixel lies inthe homogeneous area around the centric pixel. Some exampledirection lines are shown in Fig. 1, where different gray levelssymbolize different direction lines.

The pixel homogeneity is defined using the followingmethod:

PHi =n∑

s=1

|pcens − psur

s | (1)

where PHi represents the spectral homogeneity of the ithdirection between the centric pixel and its surrounding pixel,n denotes the number of spectral bands, pcen

s is the spectralvalue of the centric pixel, as shown in black in Fig. 1, and psur

s

denotes the spectral value of the surrounding pixel, which is onthe direction line in different gray levels (Fig. 1).

The ith direction line is extended if the following conditionsare met.

1) PHi is less than a predefined threshold T1.2) The total number of pixels in this direction line is less

than another predefined threshold T2.

The extension of each direction line will cease if either of theabove conditions is not met. In this case, the extension of theith direction line will be terminated, and the algorithm will skipto the (i+ 1)th direction line.

B. Length of Direction Line

After determining all the direction lines, the length of eachand every direction line is calculated. In the ith direction, thelength is measured as follows:

di = max{|me1 −me2|, |ne1 − ne2|} (2)

where (me1, ne1) denotes the row and column number of thepixel in one end of the direction line, and (me2, ne2) denotesthe row and column numbers of the pixel in the other end. Themaximum city-block distance is adopted here because: 1) it cansmooth the spatial feature values in a homogeneous area and letthe pixels in the same shape area possess the same or close PSIvalues and 2) a large amount of computation can be saved withthe use of integers instead of decimals in (2).


Fig. 2. Flowchart of the PSI algorithm.

Upon completion of this step, the lengths of all the directionlines are obtained. Thus, the feature sets of the centric pixelcan be formulated as (d1, d2, d3, . . . , dD), where D denotes thetotal number of all the directions.

C. Calculation of PSI

The shape index of the centric pixel is the sum total length ofall the direction lines, which is defined as

PSIm,n =D∑

i=1

di (3)

where PSIm,n represents the shape index of the centric pixel(m,n). The flowchart of PSI is shown in Fig. 2.

The spectral similarity is used to construct spectrally con-nected groups of pixels to describe the shape features. Thereare three parameters in PSI: the total number of directions D,and the direction line extension thresholds T1 and T2.

The parameterD indicates the total number of direction linesof PSI. The value of D can be raised in order to describe theshape features in detail. The values, T1 and T2 ought to bedetermined through experiments as these two parameters arerelated to the shape and spatial arrangement of objects in theimage. T1 is the threshold for homogeneity, and pertains to thespectral variability in the same shape area. T2 is the maximumlength for one direction line, and relates to the size of one shapearea. More details about the parameters of PSI will be discussedlater under the experiments section.

It should be noted that PSI differs from LWEA in thefollowing aspects.

1) LWEA extracts length–width features and stores the max-imum value as the length and the minimum value as thewidth after searching all direction lines. However, PSIsums the length of all the direction lines instead of just themaximum length and width for describing the contextualstructure around the centric pixel. The features of lengthand width in LWEA only represent two directions aroundthe centric pixel, which is inadequate to provide sufficientcontextual information along all directions. The aim ofPSI is to describe the overall neighborhood features and

Fig. 3. PSI feature values for four classes (wide street, narrow street, building,tree, and grass) from a multispectral image of Beijing, China. The PSI parame-ters used in this figure are D = 20, T1 = 110, and T2 = 50. The verticalaxis of the histogram represents the occurrence frequency of the PSI value in alocal area.

let all the pixels in the same shape area have the samefeature value.

2) In LWEA, the spatial feature values are determined bysearching along the predetermined number of equallyspaced lines radiating from the centric pixel; however,these values in PSI are calculated along the directionlines throughout the centric pixel. Hence, PSI providesan orientation-independent property, which, in turn, hasan advantage to provide more stable spatial features.Moreover, the azimuth sampling in PSI is enhanced from10 of LWEA (D = 18) to 9 (D = 20), which improvesthe descriptive ability of spatial features.

To illustrate the strength of PSI, an example of PSI featuresfor four classes in a multispectral QuickBird image of Beijing,China, is given in Fig. 3. It can be found that different PSIvalues are achieved within the spectrally similar classes suchas building versus road and grass versus tree.

III. INDEPENDENT COMPONENT ANALYSIS

ICA is a multivariate data analysis method for blind sourceseparation of signals. ICA seeks to render the components asstatistically independent as possible, and is a useful tool for datamining of remotely sensed data. The use of ICA features forunsupervised signature extraction of IKONOS images has beeninvestigated previously [15]. In this paper, the ICA features ofthe QuickBird RGB bands are, however, extracted to strengthenmultispectral information to reduce the spectral complexity andcomplement the shape features extracted using the PSI-basedalgorithm.

The basic model of ICA is

Y = A X (4)

whereY vector of observed signals;A mixing matrix to be estimated;X mutually independent components.The goal of ICA is to calculate the matrix B such that the

sources X = B Y can be estimated from the observed vectorY by optimizing the statistical independence criterion.


Fig. 4. Fusion of transformed spectral information and PSI shape featuresusing SVM.

Fig. 5. QuickBird image of typical buildings and roads in Beijing.

TABLE INUMBER OF TRAINING AND TESTING PIXELS IN FIG. 5

Our paper adopts the fast and robust ICA (Fast-ICA) algo-rithm proposed by Hyvarinen [16]. It is a fixed-point algorithmbased on an optimization of entropy function called negativeentropy. Initially, this algorithm was introduced using kurtosisas a function, and was subsequently extended for general con-trast functions such as

JG(w) =[E

{G(wTx)

} − E {G(v)}]2 (5)

where w is an m-dimensional vector constrained such thatE{(wTx)2} = 1. v is a Gaussian variable of zero mean andunit variance.G(y) is practically any nonquadratic function andis used in its derivation form g(y) in the Fast-ICA algorithm,which can be chosen as follows:

G(y) =14y4 g(y) = y3

G(y) =1a

log (cosh(ay)) g(y) = tanh(ay)

G(y) = − 1ae−

ay2

2 g(y) = y e−ay2

2 . (6)

TABLE IICLASSIFICATION ACCURACIES OF DIFFERENT VALUES

FOR D WITH T1 = 110 AND T2 = 100

Fig. 6. Relationship between Kappa coefficient and D.

Fig. 7. Relationship between Kappa coefficient and T1.


One by one, the Fast-ICA algorithm estimates each inde-pendent component, and the independent component can beobtained subsequent to the optimization process denoted by (5).The RGB bands of the QuickBird are viewed as the inputobserved vectors, and Fast-ICA is employed to obtain threespectral independent components.


Fig. 9. (a) Classification result of spectral features. (b) and (c) The classification results with different parameter values of PSI. The parameters setting of are,respectively, D = 20, T1 = 100, and T2 = 30 in (b), and D = 20, T1 = 100, and T2 = 5 in (c).

Fig. 10. (a) RGB image, (b) the classification map of original spectral fea-tures, (c) the classification result with T2 = 5, and (d) the classification resultwith T2 = 40. The other two PSI parameters are set to the same T1 = 110and D = 20 in this test.

IV. INTEGRATION OF SHAPE AND SPECTRAL

FEATURES USING SVM

This paper examines the feasibility of fusion using PSI in-tegrated with spectral features based on SVM. Primarily, threeproblems need to be resolved in the fusion: the strategy of in-tegration, normalization prior to classification, and parametersselection of SVM.

Various methods have been proposed to integrate spectral andspatial features. One appropriate method is that of decision-level fusion [2], [12]–[14]. This way, spectral and spatial featurebands can be processed, respectively, in different classifiers,and the final classification map can be obtained by the ap-


Fig. 12. QuickBird image for the second experimental area in RGB true color.

proaches based on decision-level fusion. A notable hierarchicalfuzzy classification approach has been developed [8], whereindifferent features were employed for classes with differentcharacteristics. However, as HSRRS data require a significantamount of memory for storage and analysis, computationallylight algorithms are of our particular interest [2].

In this paper, the single classifier with multiple features isutilized, which requires less CPU time for classification thanthe multiple-classifiers method. Owing to the complexity ofinput features, a nonparametric model needs to be chosenfor the classification [17]. Conventional classifiers like the


maximum–likelihood classifier (MLC) are not capable ofachieving a satisfactory accuracy. This is because the estimateddistribution function usually employs the normal distribution,which may not represent the actual distribution of the data. Inrecognition of this, fuzzy ARTMAP was tested for the contex-tual features extracted by the spatial reclassification algorithm[2]. Similarly in [10] and [11], the morphological featuresafter feature selection and the spectrally transformed bands byprincipal component analysis (PCA) were input to the neuralnetwork classifier at the same time. This paper investigatesSVM owing to its computational simplicity and superior accu-racy when compared to other classifiers.

The second problem is to normalize the spatial and spectralfeatures into [0, 1] in such a way that they can then be inputinto the classifier. The normalization method of spectral inputsdiffers from that of spatial ones because of the dissimilarrange distribution of spatial feature values. The normalizationmethods can be formulated as follows:

Step1 :

{d′ij = dij−dmin

dmax−dmin· 255 spectral features

histogram − equalization spatial features

(7)

Step2 : d′′ij =d′ij − d′max

d′max − d′min

· 1 (8)

where dij denotes the original value of the pixel (i, j), andaccordingly dmax, dmin represent the maximum and minimumvalue in that band, d′′ij is the feature value of the pixel (i, j) afternormalization.

The third problem relates to the parameters of SVM. SVMclassifiers [18], [19] of the form f(x) = w · Φ(x) + b arelearned from the data {(xi, yi), i = 1, 2, 3, . . . , N}, where xi

is an n-dimensional feature vector, f(x) denotes a hyperplane,which separates samples label yi = ±1 on each side, and wand b are the parameters of the hyperplane. The hyperplanecalculation can be formulated into a constrained optimizationproblem as follows:

minw,b,ξ

12‖w‖2 + C

N∑i=1

ξi (9)

where C is a regularization parameter and ξi is slack-variablessubject to the constraints

yi (w · Φ(xi) + b) ≥ 1 − ξiξi ≥ 0. (10)

To simplify the learning of nonlinear SVMs, the objectivefunction is typically expressed in its dual form

min0≤αi≤C

W =12

N∑i,j=1

αiQijαj −N∑

i=1

αi + b

N∑i=1

yiαi. (11)

Subject to 0 ≤ αi ≤ C and∑N

i=1 yiαi = 0, where b isLagrange multiplier and Qii = yiyiΦ(xi)Φ(xi). For more

TABLE IIINUMBER OF TRAINING AND TESTING PIXELS

Fig. 13. Available reference data.

details about the SVM, see [18]–[20]. The design of SVM for aspecific application needs to handle several issues.

1) Multiclass classification: While SVM was originally de-signed for binary classification, most remote sensingapplications involved multiple classes. Two main ap-proaches have been suggested for extending SVM tomulticlass classifications [21]. The first is “one againstall” (OAA), which consists of a set of binary SVMs.Each classifier is trained to separate one class from therest and the pixel is allocated to the class for whichthe largest decision value is determined. The second is“one against one” (OAO), where a series of classifiers areapplied to each pair of classes, with the most commonlycomputed class label reserved for each pixel [21]. In ourexperiments, the OAO method is employed.

2) Selection of kernel function: The commonly used kernelfunctions are Gaussian radial basis function (RBF) andpolynomial function (POLY):

{K(xi, x) = (xi · x+ 1)p POLY

K(xi, x) = exp(−γ‖xi − x‖2

)RBF.

(12)

For pattern recognition of HSRRS images, the POLY kernelwas found to be better than RBF, because the POLY kernel isa type of function with overall influence, whereas RBF onlyhas great respondence to local points around the central value[17]. Considering that the feature space of HSRRS images isdispersedly distributed without an obvious clustering center, thePOLY kernel is used in this paper.

In addition, the kernel-based implementation of SVM in-volves problems pertaining to the selection of multiple parame-ters, including the kernel parameters (p) and the regularizationparameter C. Some standard ways exist, which can facilitate


Fig. 14. Classification results of (a) original RGB spectral features, (b) ICA signals, (c) PSI and RGB features, and (d) ICA and PSI features fused by SVM.

TABLE IVCONFUSION MATRIX OF THE CLASSIFICATION RESULT FOR FIG. 14(a)

the selection of parameters in SVM classifier design such asthe well-known leave-one-out (LOO) procedure [21]. In thispaper, these parameters are selected based on our experienceconsidering that different parameters setting can be comparedin that way. The flowchart of the whole processing is shownin Fig. 4.

V. EXPERIMENTS

Two experiments were performed to test the effectivenessof the proposed methodologies. The first was designed to detectthe influence of the PSI parameters. The second was to comparethe classification effect of different spatial features. Both theexperiments employed QuickBird images of Beijing.

A. Experiment 1—Parameters of PSI

Typically, an urban infrastructure consists of buildings androads. However, these two types of objects usually present Fig. 15. Histograms of spectral features.


TABLE VCLASSIFICATION ACCURACIES OF SPECTRAL FEATURES

analogous spectral features in the HSRRS image. Hence, otherfeatures are needed to discriminate them. In Chinese cities,buildings and roads have typical shape characteristics. Fig. 5illustrates a sample QuickBird image of Beijing city. Thisexperiment tested different values for the three parameters ofPSI: T1, T2, and D, in order to detect their influence on thealgorithm. There are three object types in Fig. 5: buildings,roads, and trees. Although buildings and roads are spectrallysimilar, they vary in their shapes. The numbers of training andtesting pixels of the three types of objects are listed in Table I.

The parameter D indicating the total number of directionsof PSI was detected first. As mentioned in Section II, whenthe value of D increases, the difference of PSI values forpixels in the same shape area decreases. In this experiment,the PSI features are regarded as an additional band of thethree spectral RGB bands, the dimension of the input featurevectors is four, and the classifier used herein is the MLC asthe performance of feature space is concentrated here. Allthe input features were preprocessed employing the two stepsof normalization. The classification results with different Dvalues are shown in Table II. As D increases, the classificationaccuracies of buildings and roads are also found to increase.Nonetheless, the accuracy of trees remains nearly at the samelevel as they have no apparent shape features; the classificationresults have not been influenced with the input of PSI. This testindicates that D is a parameter representing the capability ofPSI for shape feature description, which can be demonstratedin Fig. 6.

Next, the parameter T1 indicating the maximum spectraldistance between the centric pixel and its surrounding pixelswas detected. The value of T1 should be adjusted in accordancewith the different image features. If the homogeneous area orthe same shape area displayed varying spectral information,the value of T1 should be set to a larger number. As shownin Fig. 5, the building’s roof facing to the sunlight presents alarger spectral value than the other sides with less sunlight, andthe vehicles on the roads also influence T1. This signifies thata lager value of T1 may result in better classification accuracy.Fig. 7 proves this. Similar to the situation of D, when the valueof T1 ranges from 10 to 130, the classification accuracies ofbuildings and roads increase clearly, and the accuracy of treesremain almost the same.

Finally, the parameter T2 indicating the maximum spatialdistance between the centric pixel and its surrounding pixelswas detected. T2 value is critical in discriminating objects ofdifferent shapes. The relationship between the Kappa coeffi-cient and T2 is shown in Fig. 8.

When the value of T2 ranges from 110 down to 30, the Kappacoefficients slowly escalate to the maximum value, and thenKappa decreases sharply to the minimum with the value of T2

Fig. 16. Orientation-independent WT feature.

being from 30 to 5. It is noted that at 30, T2 is approximatelythe average of the width and length of the buildings in Fig. 5.If T2 is small to some extent, PSI features are not capableof discriminating the objects with different shapes. As Fig. 8shows, the classification accuracies decrease sharply when thevalue of T2 ranges from 30 to 5.

Fig. 9 shows three classification maps of Fig. 5. Fig. 9(b)displays the result using spectral and PSI features withD = 20,T1 = 100, and T2 = 30, and Fig. 9(c) the result using spectraland PSI features with D = 20, T1 = 100, and T2 = 5.

Fig. 9(a) demonstrates the obvious misclassification of spec-tral features, whereas the results in Fig. 9(b) and (c) showa considerable improvement over the spectral classification.Comparing Fig. 9(b) and (c), it is apparent that the parameterT2 is also a scale factor of PSI. A larger value of T2 [Fig. 9(b)]can identify larger objects but overlooks some smaller objects.A smaller value of T2 [Fig. 9(c)] can identify smaller objectssuch as the narrow roads between buildings, whose widthapproximately equals T2, about five pixels. Another datasetwas added here to discuss the dependence of PSI to the localstructural pattern and its multiscale effect. Fig. 10(a) shows atypical residential area in Beijing, consisting of low and densehouses in the old city area, and some communities with highbuildings and green covers in the new developing region. Theimage size is 545 × 535 pixels, and it displays multiscale shapefeatures of objects. The features employed here also include theRGB and PSI. The classification maps with different values ofT2 are listed in Fig. 10(b)–(d). T2 was given considerable atten-tion because it was the scale factor of PSI critical to multiscaleobjects. The relationship between the Kappa coefficient and T2

is charted in Fig. 11, where similar results are observed: thereare apparently misclassifications with only spectral signals asinput; when T2 increases from 3 to 10, the Kappa curve growsto the zenith, and after that the curve fluctuates to another sum-


TABLE VICLASSIFICATION ACCURACIES OF SPATIAL FEATURES WITH RGB BANDS

TABLE VIICONFUSION MATRIX OF THE CLASSIFICATION RESULT FOR FIG. 14(d)

mit with T2 = 40. Subsequently the curve descends graduallyto a stable value, which remains above 0.56 even though T2

ranges greatly from 70 to 200. Note the two T2 values (10and 40) corresponding to the two summits in the curve, whichare just the sizes of low and high buildings, respectively, in theimage. Accordingly, it can be found that PSI with an invariableT2 cannot describe multiscale characteristics of objects at thesame time; however, some multiscale profiles [10], [11] can beconstructed to overcome this deficiency. This will be exploredin our future work.

In the aforementioned curve, the maximum value is about0.61, and the minimum is 0.56. Considering the Kappa co-efficient of spectral features classification that is 0.48, it canbe found that the appropriate parameter range to achieve anapparent accuracy increase is wide comparing to the results ofthe spectral classification.

B. Experiment2—Fusion of PSI and ICA

The purpose of this experiment is twofold. The first is tocompare the classification results using different spatial featuresto demonstrate the effectiveness of PSI features. Second, itillustrates that the combination of the shape features extractedby PSI and the spectral features extracted by ICA can facilitateHSRRS data classification.

Another QuickBird image of the Beijing city was used.Fig. 12 shows the image (495 × 317 pixels), replete withavailable ground truth data. This image presents a typicalurbanized area in Beijing, which includes a water body, long,and wide roads, low and dense buildings, bare soil, grass, andhigh trees. The key challenge in classifying this image lies inthe confusion among roads and buildings, water and shadow,grass and trees, bare soil and buildings, and bare soil and roadsand the uncertainties arising thereof.

In this experiment, the training samples were selected at ran-dom. The accuracy estimation was based on the reference pixels

Fig. 17. Increase of accuracy from original RGB spectral bands, to RGB andthe spatial information, PCA and the spatial information, and ICA and thespatial information using WT, GLCM, and PSI, respectively.

independent of the training data. The image was classified intoseven classes, namely water, tree, grass, building, bare soil,road, and shadow. The numbers of training and testing pixelsin each class are listed in Table III, and the available referencedata are shown in Fig. 13.

The classification maps using different shape and spectralfeatures are shown in Fig. 14. Fig. 14(a) shows many uncer-tainties in the classification map that employs only spectralfeatures. These significant misclassifications are between ob-jects of similar spectral signatures such as building–road, andtree–grass. This is also illustrated by the confusion matrix inTable IV. Fig. 14(b) shows the classification map with ICAreplacing the original spectral bands. PCA and ICA can beregarded as transformed spectral features, which are utilizedto reduce the complexity distribution and heterogeneity oforiginal spectral bands. Three histograms of RGB, PCA, andICA are compared in Fig. 15, where clear, stable, and lessvariable spectral distribution are observed from PCA and ICAimages. Their classification accuracies are listed in Table V.Upon comparing the results of RGB and ICA, it is noted thatthe classification accuracies of grasses and trees are improved,hence contributing to an increased accuracy by 17%.


TABLE VIIICLASSIFICATION ACCURACIES COMPARISON BETWEEN MLC AND SVM

Both ICA and PCA can reduce the correlation among thedifferent spectral bands, but the accuracy increase from PCAto ICA is only 1.3%. This is not apparent because only threespectral bands are available, and there is no obvious meaningof statistical independence in this case.

Due to the inadequacy of spectral information for discrimi-nating different objects, spatial features were added to the clas-sifier. For purposes of comparison, the spatial features extractedby GLCM and WT were also employed. The feature imagesof GLCM were acquired using a 7 × 7 window displacementof one pixel, and multiple texture measures such as variance,dissimilarity, mean, and contrast [1] were calculated based onthe first primary component (PC1) of RGB bands. Four co-occurrence matrices in four directions were averaged, and sojust one additional feature band of GLCM can be obtained. TheWT feature band was also acquired based on PC1. Here, an8 × 8 moving window was utilized, and the entropy measurewas employed to extract the features of four subchannels [1].For the purpose of fair comparison, an orientation-independentWT texture feature was constructed with the parameters illus-trated in Fig. 16

F (x, y) =E(AP )

E(D) + E(H) + E(V )(13)

where E is the entropy measure, F (x, y) represents the WTfeature value of pixel (x, y), and E(AP ), E(V ), E(H), E(D)are the entropy values of the four subimages obtained by theWT at level 1. The spatial features extracted using LWEA werethen compared with PSI. The two-band length–width featureimages obtained using LWEA were added into spectral bandsas the input of the classifier [8].

All the spatial features were normalized using the (7) and(8), and these features were fused with RGB bands based onSVM. Their classification accuracies are listed in Table VI,which shows that the features of GLCM, LWEA, and PSI cansupplement the spectral information and describe the structureof objects effectively, whereas WT is unable to.

The highest accuracy was achieved by PSI as its feature bandcan enhance the homogeneity of neighborhood using the con-textual information and describe the shape feature effectively.The classification map of PSI and RGB based on SVM is shownin Fig. 14(c).

The high accuracy of classifying buildings indicates thatPSI is a good shape descriptor; however, features based onPSI could not completely exploit the spectral information si-multaneously. When PSI and original RGB bands were usedtogether as input, the road–building classification of showed apromising improvement. Nevertheless, the results of tree–grassbecame worse. Consequently, some transformed spectral bands

TABLE IXCLASSIFICATION ACCURACIES FOR SPECTRAL FEATURES OF FIG. 10(a)

obtained by PCA or ICA were introduced to attain better spec-tral classification accuracy. Additionally, this offers strength-ened spectral information to reduce the complex distributionand heterogeneity in the original RGB bands and enlarge thecontrast between objects and background. The classificationmap in Fig. 14(d) and the confusion matrix in Table VII revealan improvement which maintains the accuracy of buildingswhile increasing those of bare soil and grass.

In this case, the accuracy is enhanced in two stages. In thefirst stage, the increase is due to the input of spatial features, andin the second stage, the increase is caused by the introduction oftransformed spectral information such as PCA and ICA. Theseenhancements are shown in Fig. 17 with different combinationsof spatial features.

For comparisons, MLC was employed as the fusion classifierof PSI and ICA features, and all the inputs were also normalizedin advance. The accuracy statistics of Table VIII shows thatSVM has a better capability of interpreting complex inputfeatures than MLC. This is due to the fact that the distributionof multiple features in the feature space may not be normal andSVM overcomes the defect of MLC where the dominant effectsof spectral information may occur.

In order to verify that the integration algorithm based uponboth PSI and ICA works stably, the image of Fig. 10(a) wasalso tested. First, the classification results for spectral featuresincluding RGB, PCA, and ICA are shown in Table IX, in whichICA and PCA really acquire the same accuracy. In this test,the accuracy improvement is not very apparent. The Kappacoefficients of PCA and ICA are higher than that of RGB byjust 0.05. Subsequently, the spatial features were investigated.In this test, the classification results were obtained based onthe respective spatial measures and the RGB bands fused bySVM. Comparable spatial features included the variance, mean,dissimilarity, and contrast measures of GLCM; the two bandsLWEA features; the length and width bands that are separatefeatures of LWEA [8]; and also, the orientation-independentWT feature. All the feature bands had to be normalized inadvance. The classification statistics are listed in Table X,where the PSI obtains the best result. The accuracy increasefrom LWEA to PSI verifies that more direction lines can extractthe contextual features more effectively. Considering that PSIhas only one feature band but LWEA has two, so the proposedPSI is a meaningful extension of the latter.

At last, the integration of PSI and the transformed spec-tral bands based on SVM were tested. Results are shown


TABLE XCLASSIFICATION ACCURACIES FOR SPATIAL FEATURES OF FIG. 10(a)

TABLE XICLASSIFICATION ACCURACIES FOR THE INTEGRATION

BETWEEN PSI AND PCA OR ICA

in Table XI. Similar conclusion is drawn that the Kappa co-efficient is slightly improved when ICA is employed.

VI. CONCLUSION

A shape feature index PSI, which presents a novel extensionto LWEA, was proposed in this paper. PSI can effectivelydescribe the contextual information through a simple method.This is especially true considering the huge quantity of HSRRSdata. The proposed PSI leads to a notable increase of classifica-tion accuracy as compared to other spatial measures. The threeparameters of PSI were also calibrated.

PCA and ICA have been utilized to extract spectral featuresin order to replace the original spectral signals in the classifi-cations. In the experiments, ICA performed just slightly betterthan PCA because only a few spectral bands are available. How-ever, it is exciting to find that the transformed spectral featurescan improve the classification accuracy with original bands forthe multispectral images with the high spatial resolution.

In addition, an efficient algorithm capable of swiftly inte-grating spatial and spectral features using SVM has been alsodeveloped. This is based on the recognition of the fact thatonly spectral or spatial features are inadequate for classifyingHSRRS data. The approach of a single classifier with multiplefeatures was attempted and in this way, the input features shouldbe normalized beforehand, and then SVM is employed to avoidthe spectral dominance effect in the feature space. Our exper-iments showed that the combination of transformed spectralbands such as PCA or ICA and PSI based on SVM performedwell. A single classifier cannot fully exploit the spectral andspatial features, yet it is a good approach providing that bothcomputation time and accuracy are considered. While the useof PSI in conjunction with PCA or ICA and SVM improvesthe classification accuracy considerably, PSI also has someweakness. For example, it provides contextual information witha single scale; however, it could be overcome by the method ofmultiple profiles [10], [11] with different values of T2. This willbe investigated in our future work.

ACKNOWLEDGMENT

The authors would like to thank the anonymous reviewers fortheir comments. Their insightful suggestions have significantlyimproved the paper.

REFERENCES

[1] S. W. Myint, N. S. N. Lam, and J. M. Tylor, “Wavelets for urban spatialfeature discrimination: Comparisons with fractal, spatial autocorrelation,and spatial co-occurrence approaches,” Photogramm. Eng. Remote Sens.,vol. 70, no. 7, pp. 803–812, Jul. 2004.

[2] F. Dell’Acqua, P. Gamba, A. Ferari, J. A. Palmason, J. A. Benediktsson,and K. Arnason, “Exploiting spectral and spatial information in hyper-spectral urban data with high resolution,” IEEE Geosci. Remote Sens.Lett., vol. 1, no. 4, pp. 322–326, Oct. 2004.

[3] C. Zhu and X. Yang, “Study of remote sensing image texture analysisand classification using wavelet,” Int. J. Remote Sens., vol. 19, no. 16,pp. 3197–3203, Nov. 1998.

[4] D. G. Barber and E. F. LeDrew, “SAR sea ice discrimination using tex-ture statistic: A multivariate approach,” Photogramm. Eng. Remote Sens.,vol. 57, no. 62, pp. 949–958, 1991.

[5] Y. Zhang, “Optimisation of building detection in satellite images bycombining multispectral classification and texture filtering,” ISPRSJ. Photogramm. Remote Sens., vol. 54, no. 1, pp. 50–60, 1999.

[6] Y. Dong, B. C. Forster, and A. K. Milne, “Segmentation of radar imageryusing Gaussian Markov random field model,” Int. J. Remote Sens., vol. 20,no. 8, pp. 1617–1639, Oct. 1999.

[7] Y. Dong, A. K. Milne, and B. C. Forster, “Segmentation and classificationof vegetated areas using polarimetric SAR image data,” IEEE Trans.Geosci. Remote Sens., vol. 39, no. 2, pp. 321–329, Feb. 2001.

[8] A. K. Shackelford and C. H. Davis, “A hierarchical fuzzy classificationapproach for high-resolution multispectral data over urban areas,” IEEETrans. Geosci. Remote Sens., vol. 41, no. 9, pp. 1920–1932, Sep. 2003.

[9] M. Pesaresi and J. A. Benediktsson, “A new approach for the morpho-logical segmentation of high-resolution satellite imagery,” IEEE Trans.Geosci. Remote Sens., vol. 39, no. 2, pp. 309–320, Feb. 2001.

[10] J. A. Benediktsson, M. Pesaresi, and K. Arnason, “Classification andfeature extraction for remote sensing images from urban areas basedon morphological transformations,” IEEE Trans. Geosci. Remote Sens.,vol. 41, no. 9, pp. 1940–1949, Sep. 2003.

[11] J. A. Benediktsson, J. A. Palmason, and J. R. Sveinsson, “Classificationof hyperspectral data from urban areas based on extended morphologicalprofiles,” IEEE Trans. Geosci. Remote Sens., vol. 43, no. 3, pp. 480–491,Mar. 2005.

[12] J. A. Benediktsson and I. Kanellopoulos, “Classification of multisourceand hyperspectral data based on decision fusion,” IEEE Trans. Geosci.Remote Sens., vol. 37, no. 3, pp. 1367–1377, May 1999.

[13] G. J. Briem, J. A. Benediktsson, and J. R. Sveinsson, “Multiple classifiersapplied to multisource remote sensing data,” IEEE Trans. Geosci. RemoteSens., vol. 40, no. 10, pp. 2291–2299, Oct. 2002.

[14] M. Petrakos, J. A. Benediktsson, and I. Kanellopoulos, “The effect ofclassifier agreement on the accuracy of the combined classifier in deci-sion level fusion,” IEEE Trans. Geosci. Remote Sens., vol. 39, no. 11,pp. 2539–2546, Nov. 2001.

[15] P. S. Huang and T. M. Tu, “A target fusion-based approach for classifyinghigh spatial resolution imagery,” in Proc. IEEE Int. Geosci. Remote Sens.Symp., 2004, pp. 175–180.

[16] A. Hyvarinen, “Fast and robust fixed-point algorithms for indepen-dent component analysis,” IEEE Trans. Neural Netw., vol. 10, no. 3,pp. 626–634, Mar. 1999.

[17] J. Li and R. M. Narayanan, “Integrated spectral and spatial informationmining in remote sensing imagery,” IEEE Trans. Geosci. Remote Sens.,vol. 42, no. 3, pp. 673–685, Mar. 2004.

[18] G. H. Halldorsson, J. A. Benediktsson, and J. R. Sveinsson, “Source basedfeature extraction for support vector machines in hyperspectral classifica-tion,” in Proc. IEEE Int. Geosci. Remote Sens. Symp., 2004, pp. 536–539.

[19] F. Melgani and L. Bruzzone, “Classification of hyperspectral remote sens-ing images with support vector machines,” IEEE Trans. Geosci. RemoteSens., vol. 42, no. 8, pp. 1778–1790, Aug. 2004.

[20] G. Camps-Valls and L. Bruzzone, “Kernel-based methods for hyperspec-tral image classification,” IEEE Trans. Geosci. Remote Sens., vol. 43,no. 6, pp. 1351–1362, Jun. 2005.

[21] G. M. Foody and A. Mathur, “A relative evaluation of multiclass imageclassification by support vector machines,” IEEE Trans. Geosci. RemoteSens., vol. 42, no. 6, pp. 1335–1343, Jun. 2004.


Liangpei Zhang received the B.S. degree in physicsfrom Hunan Normal University, ChangSha, China,in 1982, the M.S. degree in optics from the Xi’an In-stitute of Optics and Precision Mechanics of ChineseAcademy of Sciences, Xi’an, China, in 1988, and thePh.D. degree in photogrammetry and remote sensingfrom Wuhan University, Wuhan, China, in 1998.

From 1997 to 2000, he was a Professor in theSchool of the Land Sciences, in Wuhan University.In August 2000, he joined the State Key Laboratoryof Information Engineering in Surveying, Mapping,

and Remote Sensing, Wuhan University, as a Professor and Head of the RemoteSensing section. His research interests include hyperspectral remote sensing,high-resolution remote sensing, image processing, and artificial intelligence.He is also an Associate Editor of Geospatial Information Science Journal.

Prof. Zhang has served as Cochair of the International Society for OpticalEngineers (SPIE) Series Conferences on Multispectral Image Processing andPattern Recognition (MIPPR), the Conference on Asia Remote Sensing in1999, Editor of the MIPPR01, MIPPR05 Symposium, and Chinese NationalCommittee for the International Geosphere–Biosphere Programme.

Xin Huang received the B.S. and the M.S. degreesfrom Wuhan University, Wuhan, China, in 2004 and2006, respectively. He is currently working towardthe Ph.D. degree at Wuhan University.

His current research interests include multi/hyperspectral and high-resolution remotely sensedimage processing, spatial feature extraction, neuralnetworks, pattern recognition, and remote sensingapplications.

Bo Huang is currently an Associate Professor inthe Department of Geography and Resource Man-agement, The Chinese University of Hong Kong.He was previously an Associate Professor in theSchulich School of Engineering, University ofCalgary. His work focuses on the intersection ofgeospatial information systems (GIS), artificial intel-ligence, operations research, and spatial databases.He has authored and coauthored over 30 papers inrefereed international journals and developed sev-eral GIS, spatial analysis, and optimization software

packages. At present, he serves as a Guest Professor of the State Key Laboratoryof Information Engineering in Surveying, Mapping, and Remote Sensing,Wuhan University, Wuhan, China and the Institute of Remote Sensing Appli-cations, Chinese Academy of Sciences, Beijing, China. His current researchinterests include spatial optimization, spatial statistics for change analysis, andimage processing.

Pingxiang Li received the B.S., M.S., and Ph.D.degrees in photogrammetry and remote sensing fromWuhan University, Wuhan, China, in 1986, 1994, and2003, respectively.

Since 2002, he has been a Professor in theState Key Laboratory of Information Engineering inSurveying, Mapping, and Remote Sensing, WuhanUniversity. His research interests include photogram-metry and synthetic aperture radar image processing.

a pixel shape index coupled with spectral information for … · 2011-12-26 · resolution remotely...

Documents