relevance feedback-based image retrieval interface incorporating region and feature saliency...

Relevance Feedback-Based Image Retrieval InterfaceIncorporating Region and Feature Saliency Patterns

as Visualizable Image Similarity Criteria

Speaker： Kun Hsiang

Outline

RFSP Image Similarity Model

• GA-Based Relevance Feedback Mechanism Using RFSP

• Experimental Evaluation of RFSP Method

RFSP Image Similarity Model

• Set of images I represents the image database.

• The area of each image is partitioned into nR regions, defined by the set of regions R .

• From each region, nF features are extracted, based on the set of features F.

• The set of weights W contains nW real-valued region and feature weights.

RFSP Image Similarity Model

RFSP Image Similarity Model

• Region is defined as a rectangular array of image pixels, and all image regions are of equal size, obtained by uniformly partitioning the image area into NxN (=nR) blocks.

• Feature denotes an arbitrary image feature , e.g., color or texture ,based on which the similarity of a pair of image regions is computed.

• Weight denotes the relative importance of a region or a feature.

RFSP Image Similarity Model

RFSP Image Similarity Model

RFSP Image Similarity Model

RFSP Image Similarity Model

RFSP Image Similarity Model

• Thanks to the relevance feedback, all the user has to do is to specify the query image and a couple of relevant images, without worrying about the region and feature weights, or at all being aware of the existence of the RFSP structure.

RFSP Image Similarity Model

RFSP Image Similarity Model

GA-Based Relevance Feedback Mechanism Using RFSP

• GA for inferring RFSP：– We employed five weight levels：minimal (0),

low(0.25), medium(0.5), high(0.75), maximal (1)

– W={w1,w2,w3,w4,w5}

GA-Based Relevance Feedback Mechanism Using RFSP

• GA for inferring RFSP：– An important difference between the proposed

RFSP model and all of the surveyed relevance feedback methods is that the proposed model uses a discrete set of (region and feature) weights, rather than arbitrary weights from interval [0, 1].

GA-Based Relevance Feedback Mechanism Using RFSP

• GA for inferring RFSP：– Using a discrete set of weights：

• While theoretically decreasing the expressive power of the model

• Also contributes to the faster convergence of the GA, and does not have practical implications on the retrieval performance, as confirmed through the preliminary experiments.

• Increasing the number of weight levels only made the convergence slower, without improving the retrieval performance.

• In the preliminary experiments, five weight levels resulted in the optimal balance between the speed of GA convergence and the retrieval performance.

GA-Based Relevance Feedback Mechanism Using RFSP

• Chromosome coding：– Gene

GA-Based Relevance Feedback Mechanism Using RFSP

• Chromosome coding：– Chromosome

GA-Based Relevance Feedback Mechanism Using RFSP

• Fitness measure

GA-Based Relevance Feedback Mechanism Using RFSP

GA-Based Relevance Feedback Mechanism Using RFSP

• Evolution parameters：– Regarding the parameters of the GA evolution,

selection is a standard proportional selection incorporating elitist model.

– Crossover probability is 0.6– Mutation probability is 0.1– Population size is 50 chromosomes– Evolution length is 250 generations.

Experimental Evaluation of RFSP Method

• Test Databases：– Vistex-60 database– Vistex-167 database– Brodatz-208 database– Corel-1000-A database– Corel-1000-B database

Experimental Evaluation of RFSP Method

Experimental Evaluation of RFSP Method

Experimental Evaluation of RFSP Method

Experimental Evaluation of RFSP Method

Experimental Evaluation of RFSP Method

GA

Experimental Evaluation of RFSP Method

LSP

Experimental Evaluation of RFSP Method

RFSP