AbstractThis paper benchmarks the simple Local Binary Pattern

(LBP) approach in the supervised texture segmentationproblems of the recent comparative study of Randen andHusøy. A multi-predicate version of LBP is proposed, mak-ing our approach even more powerfulfor images contain-ing textures at multiple scales.

1. Introduction

In their recent paper, Randen and Husøyreviewed the fil-tering approaches to texture feature extraction and per-formed an extensive comparative study [5]. For comparison,the co-occurrence and multi-resolution autoregressive (AR)features were also included in the study. In their experi-ments, various filtering approaches yielded different resultsfor different images. No single approach performed best orvery close to the best for all images. The computationalcomplexity of the methods was also considered to be toohigh. Randen andHusøy concluded that a very useful direc-tion for future research would be the development of pow-erful texture measures that can be extracted and classifiedwith low computational complexity.

In this paper, we will use the image set ofRanden andHusøy to benchmark the Local Binary Pattern approach,which has been very powerful in various classification andsegmentation problems and is of low computational com-plexity [3,4]. The 3x3 neighborhood of the basic LBP maybe inadequate for images containing textures at largerscales. To solve this problem, we propose a simple multi-scale extension for LBP.

Section 2 describes texture discrimination using LBP-type operators. First, the principle of the basic LBP is de-scribed, and then multi-scale extensions of LBP are pro-posed. Section 3 presents experimental results for the basicand multiscale LBP and compares them to the best resultsobtained by Randen and Husøy [5]. Section 4 contains dis-cussion and conclusions.

2. LBP operators

2.1. Basic LBP

Ojala et al. [4] introduced the Local Binary Pattern(LBP) texture operator shown in Fig. 1. The original 3x3neighborhood is thresholded by the value of the center pix-el. The values of the pixels in the thresholded neighborhoodare multiplied by the binomial weights given to the corre-sponding pixels. Finally, the values of the eight pixels aresummed to obtain the LBP number for this neighborhood.

The 256-bin LBP histogram computed over a region isused for texture description. LBP is invariant against anymonotonic gray scale transformation and provides informa-tion about the spatial structure of image texture. LBP canbe easily combined with a local contrast measure to make iteven more powerful in certain applications. There are manydifferent ways of measuring the dissimilarity between sam-ple and model histograms. In our experiments, we used thelog-likelihood measure:

(1)

whereN is the total number of bins in the histogram, andSnandMn correspond to the probabilities of binn in the sampleand model histogram, respectively.

2.2. Multiscale LBP

Some may find the performance of LBP surprisinglygood, given the small support of 3x3 pixels. One may arguethat this operator size is by no means adequate, in compari-

Figure 1. Calculation of the LBP

6 5 2

7 6 1

9 8 7

1

1

1 11

0

00 1 2 4

8

163264

128

example thresholded weights

LBP = 1 + 16 +32 + 64 + 128 =241

L S M,( ) Sn Mnlnn 1=

N

∑–=

Texture Classification by Multi-Predicate Local Binary Pattern Operators

Mäenpää Topi, Pietikäinen Matti, Ojala TimoMachine Vision and Media Processing Unit

Infotech Oulu and Department of Electrical EngineeringP.O.Box 4500, FIN-90014 University of Oulu, Finland

{topiolli, mkp, skidi}@ee.oulu.fi

son to the much larger Gabor filter masks, for example, thatare often used. Actually, the ‘built-in’ support of the opera-tor is inherently larger than 3x3 pixels, as only a specificlimited set of binary patterns can occur next to a particularbinary pattern. Further, the histogram of local operator re-sponses incorporates larger scale texture properties.

However, if larger scale analysis is required, it can be ac-complished simply by increasing the predicate (i.e. neigh-borhood size) of the operator. This means that we choosethe eight neighbors of the center pixel from the correspond-ing positions in different neighborhoods (3x3, 5x5, 7x7,etc.). A resembling approach has been used, for example,with the gray level co-occurrence method to extract multi-scale information.

Another way would be to scale the source image and ap-ply the operation to different scales. Here scaling refers tosubsampling with interpolation or low-pass filtering. Theoperation is applied to one or many low-pass filtered anddown-sampled versions of the source image. As opposed tothe local structures caught by the standard LBP, everydown-sampled scale gives more global information aboutthe structure of the texture. However, as the texture shrinks,the area over which the LBP operation is calculated be-comes smaller, which may make the LBP histograms statis-tically unstable. Scaling can therefore only be used if thesamples are big enough. In our experiments, the multipred-icate approach performed more robustly than the scaling ap-proaches, and therefore it was chosen for the experimentspresented in Section 3.

When a texture is processed using LBP operators withdifferent support areas, one histogram for each operator isobtained. To be useful in classification problems, the infor-mation provided by the histograms must be combinedsomehow. Yet more importantly, we must assume that us-ing another support region really gives us more informationabout the texture. If this is not the case, the new informationcan cause the classification accuracy to deteriorate.

In our experiments where more than one support regionwas used, the dissimilarity between training and testingsamples was calculated using three different approaches:

(2)

(3)

(4)

whereH is the number of support regions, i.e. the numberof histograms for each sample, andN is the number of bins

in each of the histograms.Shn andMhn correspond to theprobabilities of binn in the hth sample and model histo-gram, respectively.Ths andThm denote the total number ofentries used in producing sample and model histogramh, re-spectively. Eq. 2 sums up all dissimilarity measures be-tween corresponding histograms, whereas Eq. 4 considersonly the histogram pair with the smallest dissimilarity. InEq. 3 all histograms are concatenated into a single histo-gram.

We have discovered that the two approaches presentedby equations 2 and 3 give almost identical results, whereasthe third one (Eq. 4), fails chiefly because of the noisinessof the measurement. The probability of one histogram tomisclassify a sample is larger than that of the joined histo-gram. Joining the histograms reduces noise effects, whichexplains the better performance. Due to computational sav-ings, we prefer Eq. 3 to Eq. 2, and select it as the dissimilar-ity measure for the experiments presented in Section 3.

3. Experiments

As a test bench for the approach, we employed the su-pervised segmentation problems used in the recent compar-ative study of Randen and Husøy [5], shown in Fig. 2. Inorder to get comparable results we followed the experimen-tal setup of Randen and Husøy as closely as possible.

The experiment involves images from three differentsources: the Brodatz album [1], the MIT Vision Texture da-tabase [2], and the MeasTex database [6]. Consequently,images captured with different equipment and under differ-ent conditions are used. For each texture present in a testmosaic there is a 256x256 training image that is extractedfrom a different area in the source image so that an unbiasederror estimate is obtained. Since the source images wereglobally histogram equalized prior to being used, the graylevel mean and deviation of a training image, and the corre-sponding texture in the test mosaic are roughly equal.However, since the training and testing portions were ex-tracted from different locations in the large source image,there are few notable, even visible, differences in the graylevel properties between the training and testing portions ofsome texture classes in certain mosaics. A different meangray level is not a problem, if a texture feature is invariantagainst changes in the mean, but changes in gray scale canresult in serious performance loss.

In the experiments, texture classes were known a priori,and an independent sample of each of them was used intraining. The segmentation was made on a pixel-by-pixelbasis. Each pixel was classified into one of the trainingclasses by placing a disk centered on the pixel, calculatingthe LBP histogram over it, calculating the dissimilaritymeasure against all classes and placing the pixel into the

L S M,( ) Shn Mhnlnn 1=

N

∑h 1=

H

∑–=

L S M,( )ThsShn

Thsh∑-----------------

ThmMhn

Thmh∑

-------------------lnn 1=

N

∑h 1=

H

∑–=

L S M,( ) minh 1 … H, ,{ }∈

Shn Mhnlnn 1=

N

∑–=

most similar one.The size of the disk has a great influence on the segmen-

tation error. Whereas a big disk more reliably captures thetextural structure and reduces the classification error in themiddle of the texture regions, it also covers a large part ofboth of the textures on the boundaries. This results in a largenumber of classification errors on texture boundaries, whichincreases the overall error rate. Our experiments were car-ried out without any optimization of the parameter combi-nations, i.e. the same parameters were used for everyproblem. Since a disk radius of 20 pixels seemed to serve agood compromise between classification accuracy andboundary errors, it was chosen as the “universal” disk size.Better results would have been achieved if the disk size hadbeen optimized for each problem individually. The best re-sults using this disk size were obtained by combining infor-mation obtained by LBP operators with predicates 1 and 3,corresponding to neighborhood sizes of 3x3 and 7x7 pixels,respectively.

Table 1 shows the achieved error rates in addition to er-ror rates obtained by the basic LBP operator without an ad-ditional predicate. We can see that in all but two cases, thesegmentation result was better than the best result obtainedby Randen and Husøy. It is important to note that Randenand Husøy’s best results were selected from among manydifferent texture operators, and LBP can still perform betterin all but two cases. If average error percentages are com-pared, the multi-predicate LBP (MP-LBP) with predicatesone and three (1,3) is 7.5 percentage units better than the op-timized set of operators used in Randen and Husøy’s test. Itis also 1.4 percentage units better than the basic LBP. The

effect of the neighborhood size on the segmentation errorcan also be seen in Table 1. Large predicates result in asparse sampling of the neighborhood, which decreases theclassification performance. An additional predicate of 5pixels does not actually enhance the segmentation accuracy.Adding a third predicate is not beneficial, either. This can beseen in the last column of Table 1, where the segmentationerror rates for an MP-LBP with three different predicates(one, two and three) are shown. The average error rate isslightly larger than that obtained without predicate two.

4. Discussion and conclusions

We see that the LBP approach, despite its simplicity,provides the lowest error rate of all operators in ten of the12 cases, and in most cases by a clear margin. This impres-sive result is largely attributable to the gray scale invarianceof the LBP operator. It is understandably a very useful prop-erty when the gray scale properties of the unknown testsample differ from the training data, which is the case inmost of the 12 mosaics used in this study. A similar conclu-sion was drawn in our recent study, in which the perform-ance of the LBP operator was compared to that ofmultidimensional distributions of signed gray level differ-ences [7].

The results indicate that, in most cases, it is advanta-geous to use more than one predicate for the LBP operator.There are only three cases where the basic LBP performsbetter than the one extended with another predicate. Themulti-predicate approach shows its usefulness especially inproblem #12, where the segmented textures have differentlyscaled structures: the error percentage drops from 9.9 to 5.3.Furthermore, the multi-predicate approach should be pre-ferred to scaling because scaling decreases the amount ofinformation in an image. In Randen and Husøy’s segmenta-tion problems, the best results were obtained using an addi-tional LBP histogram with three pixels as the predicatevalue.

When the log-likelihood dissimilarity measure is used, itusually makes no significant difference whether the dis-tance is calculated by concatenating histograms or by sum-ming up the distances between corresponding histograms.Taking the minimum distance is, however, much more inef-ficient. Since concatenating the histograms is the simplestmethod of the three, it was preferred in our experiments.

5. Acknowledgements

The financial support provided by the Academy of Fin-land and the Graduate School in Electronics, Telecommuni-cations and Automation are gratefully acknowledged.

Table 1. Error rates using 20 pixels as the diskradius for the LBP operators.

Image

Best ofRanden

andHusøy

Basic LBP

MP-LBP(1,3)

MP-LBP(1,5)

MP-LBP

(1,2,3)

#1 7.2 6.2 6.7 7.3 7.0#2 18.9 18.1 14.3 14.5 13.6#3 20.6 12.1 10.2 13.7 11.1#4 16.8 10.0 9.1 10.5 8.8#5 17.2 10.9 8.0 9.4 8.3#6 34.7 16.8 15.3 15.5 15.5#7 41.7 20.8 20.7 23.3 21.9#8 32.3 22.8 18.1 20.3 17.6#9 27.8 19.2 21.4 23.3 23.2#10 0.7 0.3 0.4 0.6 0.4#11 0.2 1.0 0.8 0.7 0.8#12 2.5 9.9 5.3 6.8 5.3

Mean 18.4 12.3 10.9 12.2 11.1

6. References

[1] P. Brodatz (1966),Textures: A Photographic Album for Artistsand Designers. New York: Dover.

[2] MIT Vision and Modelling Group (1998).http://www.me-dia.mit.edu/vismod.

[3] T. Ojala and M. Pietikäinen (1999), “Unsupervised TextureSegmentation Using Feature Distributions”,Pattern Recogni-tion, vol. 32, pp. 477-486.

[4] T. Ojala, M. Pietikäinen, and D. Harwood (1996), “A Compar-ative Study of Texture Measures with Classification Based onFeature Distributions”,Pattern Recognition, vol. 29, pp. 51-59.

[5] T. Randen and J.H. Husøy (1999), “Filtering for Texture Clas-sification: A Comparative Study”,IEEE Trans. Pattern Anal-ysis and Machine Intelligence 21, 291-310. http://www.ux.his.no/~tranden/.

[6] G. Smith and I. Burns (1997), “Measuring Texture Classifica-tion Algorithms”, Pattern Recognition Letters, vol. 18, pp.1495-1501. http://www.cssip.elec.uq.edu.au/~guy/meastex/meastex.html.

[7] Ojala T, Valkealahti K, Oja E & Pietikäinen M (2000), “Tex-ture discrimination with multidimensional distributions ofsigned gray leveldifferences”,Pattern Recognition, in press.

Figure 2. Texture mosaics used in segmentation experiments.

mosaic #1 mosaic #2 mosaic #3 mosaic #4 mosaic #5

mosaic #6 mosaic #7 mosaic #10

mosaic #8

mosaic #9

mosaic #11

mosaic #12