1

s1160123 Tomoyuki Soeta

Supervised by Prof. Qiangfu Zhao

System Intelligence Lab

Information Retrieved

- Image based search

2

Outline

Introduction

Information Retrieval

VQ (Vector Quantization)

Divide into the 8x8 block

Making of Code book

K-means algorithm

Extract each image’s feature vector

Result

Image and feature vector

Distance of feature vector

Conclusion

Future work

3

Introduction

I want to aim at the improvement of information retrieval system to search it even if the input data are documents or images.

I have charge of a research on information retrieval based on a image.

To search images using a search engine, we may use the index attached to the image, the file name, etc. as the key-words. We may also use "the contents of an image themselves."

I study a new image search technique based on the code book information.

4

Information Retrieved

Text Image

Word Filtering

Morphological

Analysis

Divide into the block

(1 block 8x8)

Feature Vector Feature Vector

NNTree or SVM

Code book

Code of each block

5

VQ (Vector Quantization)

Compression coding of images

Image compression technology

In my study, I use VQ to translate an image into a bag-of-blocks (BOB)

feature vector

image

Vector

Quantization

(VQ)

same way as document search

6

Divide into the 8x8 block (1)

I used 10 facial images with the size 256x256.

images are converted to gray scale images.

Divided into the block (one-block 8x8 size).

Each image obtains the block of 32×32

pieces severally.

1 block 8x8

32 blocks

32

b

l

o

c

k

s

7

Divide into the 8x8 block (2)

Block’s pixel value is read.

Pixel read value is stored in the array of 1x64.

One image can be divided into 1024 blocks,

and an array of 1024 rows can be obtained.

1 block 8x8

2 3 4 6 8 3 7 2

8 2 8 2 8 2

2 3 4 6 8 3 7 2 8 2 8 2

・・・・

1x64

8x8 1024 rows

8

10 images

Image 0 Image 4 Image 3 Image 2 Image 1

Image 5 Image 6 Image 7 Image 8 Image 9

9

Making of Code book

The array of 10240 that can be done

by reading 10 images is made

The code book is made by using the

k-means method.

Making Code book (size 256)

10

K-means algorithm

Step 1) k initial "means" are randomly selected from

the data set .

Step 2) k clusters are created by associating every

observation with the nearest mean.

Step 3) The centroid of each of the k clusters

becomes the new means.

Step 4) Steps 2 and 3 are repeated until

convergence has been reached.

Step 1 Step 2 Step 3 Step 4

Extract each image’s feature vector (1)

The feature vector are extracted by using code book.

There is arrangement 1024 per one image.

Arranging an individual distance of the array each one and code book is measured

The number of the nearest code is returned.

Which code how many times came out is preserved as an array.

11

・・・

1x64

1024 rows

2 3 4 7 8 9 2 # # # # # # # # # #

Code book

Code 7 Code 38 Code 72 Code 200 Code 7

Code 7

Code 38

Code 72

Code 200

Code 7 0

1

2

3

4

5

1 2567 38 72 200

Result – image and feature vector(1)

12

Image 0 Image 1

Result – image and feature vector(2)

13

Image 2 Image 3

Result – image and feature vector(3)

14

Image 4 Image 5

Result – image and feature vector(4)

15

Image 6 Image 7

Result – image and feature vector(5)

16

Image 8 Image 9

Euclidean distance between feature vectors is measured, and the accuracy of the code book is seen.

17

Result - Distance of feature vector(1)

P and Q are assumed to be two feature vectors.

Data : x = (x1, x2, ..., xn) and y = (y1, y2, ..., yn)

n : size of the feature vector

The distance of P and Q is below.

:minimum distance

18

Result - Distance of feature vector(2)

256 feature0 feature1 feature2 feature3 feature4 feature5 feature6 feature7 feature8 feature9

feature0 0 0.279945 0.280761 0.226158 0.291376 0.322875 0.300502 0.2307 0.23509 0.228708

feature1 0.279945 0 0.19849 0.271927 0.318353 0.352126 0.324807 0.272823 0.269333 0.30847

feature2 0.280761 0.19849 0 0.308124 0.352732 0.378846 0.359333 0.310492 0.316141 0.324054

feature3 0.226158 0.271927 0.308124 0 0.221109 0.276269 0.240734 0.09959 0.086469 0.136439

feature4 0.291376 0.318353 0.352732 0.221109 0 0.222279 0.17478 0.202749 0.210865 0.248531

feature5 0.322875 0.352126 0.378846 0.276269 0.222279 0 0.084866 0.282603 0.270858 0.306136

feature6 0.300502 0.324807 0.359333 0.240734 0.17478 0.084866 0 0.245255 0.232873 0.276931

feature7 0.2307 0.272823 0.310492 0.09959 0.202749 0.282603 0.245255 0 0.105974 0.155957

feature8 0.23509 0.269333 0.316141 0.086469 0.210865 0.270858 0.232873 0.105974 0 0.152093

feature9 0.228708 0.30847 0.324054 0.136439 0.248531 0.306136 0.276931 0.155957 0.152093 0

Image 6 Image 5

The image5 and image6 is the same persons, image5 doesn't wear glasses, and image6 wears glasses.

Between feature5 and feature6 is minimum distance.

Conclusion

In my research, I study a new image search technique based on the code book information. The code book is obtained using the VQ method.

It is thought that an accurate feature vector was able to be extracted about the accuracy of the feature vector because the distance between Feature5 and 6 was short.

19

Information retrieval based on

"the contents of a image themselves."

20

Future work

The background is nullified.

The feature vector is extracted in the block of a different size like the block of not the block of 8x8 size but 16x16 size etc.

Multimedia retrieval that uses SVM.

21

Thank you for your attention!