Texture Segmentation Based on Voting of Blocks, Bayesian Flooding and Region Merging

C. Panagiotakis(1), I. Grinias(2) and G. Tziritas(3)

07-07-200922th International Conference on Pattern Recognition

(2): Geoinformatics and Surveying Dep., TEI of Serres, Serres, Greece(3): Computer Science Department, University Of Crete, Greece

Presenter: Dr. Costas Panagiotakis, Assistant Professor, (1) Business Administrator Administration Dep., TEI of Crete, Agios Nikolaos, Greece

2

Introduction: Related Work

• Segmentation of images is quite important for many applications, such as content based image retrieval and object recognition.

• In our previous work [1], we proposed a framework that performs automatic segmentation of images, knowing only the number of regions, which involves feature extraction and classification in feature space, followed by flooding (PMCFA) and merging in spatial domain.

• PMCFA has been also successfully applied on interactive image segmentation [2], where the goal is to classify the image pixels into foreground and background classes, when some foreground and background markers are given.

[1] C. Panagiotakis, I. Grinias and G. Tziritas, Natural Image Segmentation based on Tree Equipartition, Bayesian Flooding and Region Merging, IEEE Transactions on Image Processing, Vol. 20, No. 8, pp. 2276 - 2287, Aug. 2011.

[2] C. Panagiotakis, H. Papadakis, E. Grinias, N. Komodakis, P. Fragopoulou and G. Tziritas,  Interactive Image Segmentation Based on Synthetic Graph Coordinates, Pattern Recognition, vol. 46, no. 11, pp. 2940-2952, Nov. 2013.

3

Introduction: Contribution

1.The proposed method uses features that are optimized and tested for textured images.2.We solve the problem to find subset of blocks that represent well the whole dataset of blocks by a new framework that takes into account the blocks’ similarity and topology. The representative blocks are used to extract the features for each class. 3.The proposed method automatically computes the number of classes regions by a new criterion that takes into account the average likelihood per pixel of the classification map and penalizes the complexity of the regions boundaries. In [1-2] the number of classes were given.

[1] C. Panagiotakis, I. Grinias and G. Tziritas, Natural Image Segmentation based on Tree Equipartition, Bayesian Flooding and Region Merging, IEEE Transactions on Image Processing, Vol. 20, No. 8, pp. 2276 - 2287, Aug. 2011.

[2] C. Panagiotakis, H. Papadakis, E. Grinias, N. Komodakis, P. Fragopoulou and G. Tziritas,  Interactive Image Segmentation Based on Synthetic Graph Coordinates, Pattern Recognition, vol. 46, no. 11, pp. 2940-2952, Nov. 2013.

4

System Overview

• The proposed framework can operate completely unsupervised.• In this work, MINR = 3 and MAXR = 15

Main Steps:

5

Methodology: Feature Selection

• The image is divided into overlapping blocks (50% overlapping). • 64 × 64 block for a frame of 512 × 512 pixels is used.• We use the three components of Lab color space to represent the color

• The last component is the energy of horizontal and vertical components from wavelet transform using the fourth-order binomial filter [1 4 6 4 1]/16. We show that these components of wavelet transform suffice to represent well the texture information.

50 100 150 200 250 300 350 400 450 500

50

100

150

200

250

300

350

400

450

50050 100 150 200 250 300 350 400 450 500

50

100

150

200

250

300

350

400

450

50050 100 150 200 250 300 350 400 450 500

50

100

150

200

250

300

350

400

450

500

50 100 150 200 250 300 350 400 450 500

50

100

150

200

250

300

350

400

450

500

50 100 150 200 250 300 350 400 450 500

50

100

150

200

250

300

350

400

450

500

6

Methodology: MAXR BLOCKS SELECTION

Goal: Select the MAXR most representative image blocks taking into account the blocks similarity and topology. Main Steps:• The M image blocks are represented by a graph G, whose weights are given by the Mallows distance of three color components and of the texture component of the corresponding blocks (4-connections neighborhood). • Next, we find the MxM matrix of all shortest paths in graph G – taking into account similarity and topology. • Similar results are also obtained and by using the MST of G instead of G.

20 40 60 80 100 120 140 160 180 200 220

20

40

60

80

100

120

140

160

180

200

220

20

40

60

80

100

120

140

160

180

200

50 100 150 200 250 300 350 400 450 500

50

100

150

200

250

300

350

400

450

500

MxM matrix of all shortest paths in graph G

7

Methodology: MAXR BLOCKS SELECTION

The proposed MAXR BLOCKS SELECTION is inspired from [1]. Main Steps:• The first block is given by the block of minimum mean distance from others (centroid). • Next, we repeat MAXR-1 times the following procedure:

o The next block is given taking into account the current selected blocks.

o We get the block that has low distances from others (non selected blocks) and high distance from the selected blocks.

MAXR Selected Blocks

1

2

4

5

3

6

[1] C. Panagiotakis, Clustering via Voting Maximization, Journal of Classification, 2014 (accepted).

8

• Definition of a topographic map for each class k using the computed conditional probabilities:– Height of pixel s represents the dissimilarity of s from class k, defined as

–ln P{k|ξ(s)}where P{k|ξ(s)} is the a-posteriori probability of class k given the feature vector ξ(s).

Flooding Process for Class Propagation PMCFA (1/3)

Class Class

[1] C. Panagiotakis, I. Grinias and G. Tziritas, Natural Image Segmentation based on Tree Equipartition, Bayesian Flooding and Region Merging, IEEE Transactions on Image Processing, Vol. 20, No. 8, pp. 2276 - 2287, Aug. 2011.

Flooding Process for Class Propagation PMCFA (2/3)

Path cost Ci(s0,s) between pixels s and s0: the maximum height of pixels in that path.

Topographic distance δk(s) between s and s0: the minimum cost of paths between s and s0.

[1] C. Panagiotakis, I. Grinias and G. Tziritas, Natural Image Segmentation based on Tree Equipartition, Bayesian Flooding and Region Merging, IEEE Transactions on Image Processing, Vol. 20, No. 8, pp. 2276 - 2287, Aug. 2011.

Flooding Process for Class Propagation PMCFA (3/3)

¨ Input• Topographic map and• initial regions of high

confidence per class.

Priority Multi-Class Flooding Algorithm:

¨ Competitive growing for both the computation of topographic map and pixel labeling.

¨ Flooding stops when all image pixels are labeled.

PMCFA Result

4[1] C. Panagiotakis, I. Grinias and G. Tziritas, Natural Image Segmentation based on Tree Equipartition, Bayesian Flooding and Region Merging, IEEE Transactions on Image Processing, Vol. 20, No. 8, pp. 2276 - 2287, Aug. 2011.

Topographic map Original image

Class

Merging Process

• Usually, the number of computed regions is greater than the real number of classes.

• A merging state solves this over-segmentation problem.

• We have used a greedy algorithm that iteratively merges the regions taking into account the dissimilarity in appearance of the segments and the gradient on region boundaries.

11

Selection of the appropriate segmentation Map

We select the segmentation that minimizes a criterion C(k) = FS(K) + λ PC(k) taking into account

– the average likelihood per pixel of the classification map (FS(K)) and

– penalizes the complexity of the regions boundaries (PC(K)) that is computed from the points with curvature higher than 0.5 multiplied by a normalization factor.

12

Experimental Results on Prague Texture Segmentation Benchmark

4

14

Conclusions

An unsupervised segmentation algorithm is proposed which combines•color and texture features, •region features and •topology.yielding high performance results.

Results on Prague Texture Segmentation Benchmark: