1

Faculty of Information TechnologyFaculty of Information Technology

Image Segmentation andSearching

Gour Karmakar, Laurence Dooley, M. Manzur MurshedDengsheng Zhang, Guojun Lu

Gippsland School of Computing & Info TechMonash UniversityChurchill, VIC 3842

http://www.gscit.monash.edu.au/

2

Faculty of Information TechnologyFaculty of Information Technology

Image Segmentation

Definition – What is it?vThe process of separating out mutually

exclusive homogeneous regions ofinterest.vThere is no standard formal definition

Definition – What is it?vThe process of separating out mutually

exclusive homogeneous regions ofinterest.vThere is no standard formal definition

Cloud

Urban Scene

3

Faculty of Information TechnologyFaculty of Information Technology

Industry Applications

v Automatic car assembly in robotic visionv Airport security systemsv Object recognitionv Criminal investigationv Computer graphicsvMedical ImagingvMPEG-4 video object (VO) segmentationvMEPG-7 description of multimedia content

4

Faculty of Information TechnologyFaculty of Information Technology

Image

Any SegmentationAlgorithm: 5 Refinement

(FRIS): 2

IncorporatingTexture

(FRIST): 3

Colour

Segmentation

(FRCIS): 4

Evaluation: 6

Numerical

Results

Segmented

Results

Reference

ImageGray Level

Segmentation

(GFRIS): 1

Integrated Fuzzy Rule Based Framework

5

Faculty of Information TechnologyFaculty of Information Technology

Example Segmentation Results (2 regions)Using Different Techniques

a) Cloud Image b) Ref. Regions (c) FCM (d) PCM

e) GFRIS, r=1 f) GFRIS, r=2 g) GFRIS, r=4

R1

R2

6

Faculty of Information TechnologyFaculty of Information Technology

Segmented Results of Our FRIS Algorithm

(a) Food (b) Ref. Regions (c) FCM (d) PCM

(e) GFRIS, r=1 (f) GFRIS, r=2 (g) GFRIS, r=4

R1 R2

R3

R4R5

7

Faculty of Information TechnologyFaculty of Information Technology

Segmented Results of Our FRIS Algorithm

(a) FCM (b) PCM (C) GFRIS r=1 (d) GFRIS r=2 (e) GFRIS r=4

(a) Rocket (b) Ref: (C) FCM (d) PCM (e) GFRIS r=1

(a) GFRIS r=2 (b) FCM (C) PCM (d) GFRIS r=1 (e) GFRIS r=2

R1R2

R3

8

Faculty of Information TechnologyFaculty of Information Technology

Segmented Results of FRIST Algorithm

(a) GFRIS, r=1 (c) FRIST, r=1

(b) GFRIS, r=2 (d) FRIST, r=2

(a) GFRIS,r=1

(b) FRIST,r=1

(c) GFRIS, r=2

(d) FRIST,r=2

R1

R2

9

Faculty of Information TechnologyFaculty of Information Technology

Fuzzy Rules for Colour Segmentation (FRCIS)

Considers the following color models: HSV, RGB,XYZ, YUV, YIQ, YCBCR , CIELAB and OHTA

(a) Original Cloud (b) FRCIS ( HSV r=1) (c) FRCIS (HSV r=2) (d) FRCIS HSV r=4

(e) FCM (HSV) (f) PCM (HSV)

10

Faculty of Information TechnologyFaculty of Information Technology

Color Image Segmentation Using the HSV Model

(a) Original Cloud (b) Reference Image (b) FRCIS r=1 (c) FRCIS r=2

(d) FRCIS r=4 (e) FCM (f) PCM

R1

R2R3

11

Faculty of Information TechnologyFaculty of Information Technology

Image Searching• How do we find an image or object

from a remote database of images?

• Potential applications– Online Shopping– Internet– Digital Library– Designing applications– Education– Medical diagnoses

• How do we find an image or objectfrom a remote database of images?

• Potential applications– Online Shopping– Internet– Digital Library– Designing applications– Education– Medical diagnoses

12

Faculty of Information TechnologyFaculty of Information Technology

Finding similar images from a database.Finding similar images from a database.Finding similar images from a database.

13

Faculty of Information TechnologyFaculty of Information Technology

How does it work?

• Polar Raster Sampling• Polar Raster Sampling

Polar Grid

Polar image Polar raster sampled image in Cartesian space

14

Faculty of Information TechnologyFaculty of Information Technology

• Binary polar raster sampled shapeimages

• Binary polar raster sampled shapeimages

Polar raster sampling

Polar raster sampling

15

Faculty of Information TechnologyFaculty of Information Technology

Feature Extraction

• 2-D Fourier transform on the polar rastersampled image f(r,θ)

• The N normalized Fourier coefficients areused to represent the shape in database.

• 2-D Fourier transform on the polar rastersampled image f(r,θ)

• The N normalized Fourier coefficients areused to represent the shape in database.

feature={f1, f2, …, fN}

16

Faculty of Information TechnologyFaculty of Information Technology

• Each object in the database isrepresented by the extracted features.

• Each object in the database isrepresented by the extracted features.

Indexing

f1: (a0, a2, a3, …, aN)

f2: (b0,, b1, b2, …, bN)

f3: (c0, c1, c2, …, cN).

.

.

fm: (z0, z1, z2, …, zN)

objectsobjectsobjects

featuresfeaturesfeatures

indexing

indexing

indexing

indexing

.

.

.

17

Faculty of Information TechnologyFaculty of Information Technology

Retrieval

• User submits a query object, the systemcompares it with all the indexed objects in thedatabase, the most similar objects are sent tothe user.

• User submits a query object, the systemcompares it with all the indexed objects in thedatabase, the most similar objects are sent tothe user.

UserUserUser

Query object comparing

comparing

comparing

comparing

Objects

inthe

database

similar

.

.

.

18

Faculty of Information TechnologyFaculty of Information Technology

Query

Sea

rchi

ngR

esul

t

Real Searching

19

Faculty of Information TechnologyFaculty of Information Technology

Searching for Pictures of Sydney Opera House

20

Faculty of Information TechnologyFaculty of Information Technology

Searching for a flower