image mining – intricacies and innovations

38
Image Mining – Intricacies and Innovations SUNDARAM R M D M.Tech Computer Vision & Image Processing REcognition And Learning Lab Amrita School of Engineering

Upload: nerita

Post on 10-Feb-2016

64 views

Category:

Documents


0 download

DESCRIPTION

Image Mining – Intricacies and Innovations. SUNDARAM R M D M.Tech Computer Vision & Image Processing RE cognition A nd L earning Lab Amrita School of Engineering. Agenda. Introduction 5-min Recap Problem Definition – CBIR Need for CBIR Flow of work Feature Extraction – A brief view - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Image Mining – Intricacies and Innovations

Image Mining – Intricacies and Innovations

SUNDARAM R M D M.Tech Computer Vision & Image Processing

REcognition And Learning LabAmrita School of Engineering

Page 2: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 2Saturday, April 22, 2023

Agenda

Introduction5-min RecapProblem Definition – CBIRNeed for CBIRFlow of workFeature Extraction – A brief viewPromising directions and open issuesPerformance EvaluationConclusion

Page 3: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 3Saturday, April 22, 2023

Introduction

Page 4: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 4Saturday, April 22, 2023

5-min Recap

Why is Image Information Retrieval more important?

“A Picture is worth thousand words” Alternative form of communication Not everything can be described in text Popular medium of information on the Internet Well known search engines like “Google” fails to

retrieve images based on their contents.

Page 5: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 5Saturday, April 22, 2023

5-min Recap

Page 6: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 6Saturday, April 22, 2023

5-min Recap

Page 7: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 7Saturday, April 22, 2023

5-min Recap

Page 8: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 8Saturday, April 22, 2023

Problem Definition

Given a query image, with single / multiple object present in it; mission of this work is to retrieve similar kind of images from the database based on the features extracted from the query image.

- Content based Image Retrieval (CBIR)

Page 9: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 9Saturday, April 22, 2023

Need for CBIR

Early work on image retrieval can be tracked back to the late 1970s. Techniques used were not generally based on visual features but on the textual annotation of the images.

Through text descriptions, images can be organized by topical or semantical hierarchies to facilitate easy navigation and browsing based on standard Boolean queries.

But automatically generating texts for a wide spectrum of images is not feasible. Also, annotating images manually is a cumbersome and expensive task for large image databases, and is often subjective, context-sensitive and incomplete.

Hence it is widely recognized that a more efficient and intuitive way to represent and index visual information is needed. This gave birth to a new concept called CBIR.

Page 10: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 10Saturday, April 22, 2023

Applications of CBIR

Crime prevention: Automatic face recognition systems, used by police forces.

Security Check: Finger print or retina scanning for access privileges.

Medical Diagnosis: Using CBIR in a medical database of medical images to aid diagnosis by identifying similar past cases.

Intellectual Property: Trademark image registration, where a new candidate mark is compared with existing marks to ensure no risk of confusing property ownership.

Page 11: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 11Saturday, April 22, 2023

Flow of Work

Flow Chart

Page 12: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 12Saturday, April 22, 2023

Feature Extraction [1]

Query Image

Extract Features(Primitives)

Image Database

Features Database

SimilarityMeasure

MatchedResults

RelevanceFeedbackAlgorithm

Page 13: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 13Saturday, April 22, 2023

Feature Extraction [2] -- Color

Color is the most extensively used visual content for image retrieval.

Its three-dimensional values make its discrimination potentiality superior to the single dimensional gray values of images.

The Various parameters that can be extracted from color information are as follows:

1) Color Moments2) Color Histogram

Page 14: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 14Saturday, April 22, 2023

Feature Extraction [3] -- Color

Color is the most extensively used visual content for image retrievalFinding the Color moments

1) μi = (1/N) Σ fij …Mean2) σi = { (1/N) Σ (fij - μi )2 } ½ …Variance3) Si = { (1/N) Σ (fij - μi )3 } 1/3 …Skew nessWhere fij is the value of the ith color component of the image pixel j, and N is the number of pixels in the image.

Through Color HistogramSince any pixel in the image can be described by three components in a certain color space, a histogram, i.e., the distribution of the number of pixels for each quantized bin, can be defined for each component. Clearly, the more bins a color histogram contains, the more discrimination power it has.

Page 15: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 15Saturday, April 22, 2023

Feature Extraction [4] -- Color

Page 16: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 16Saturday, April 22, 2023

Why Fuzzy Color Histogram(FCH)?

Existing Algorithm:Given a color space containing n color bins, the color histogram of image containing N pixels is represented as, H(I) = [h1, h2, h3, ….hn] where hi = Ni / N is the probability of a pixel in the image belonging to the ith color bin, and Ni is the total number of pixels in the ith color bin.

Fuzzy Color Histogram:Instead of using the normal probability values, we consider each of the N pixels in the image I being related to all the n color bins via Fuzzy – set membership functions such that the degree of belongingness or association of the jth pixel to the ith color bin is determined by distributing the membership value of the jth pixel,μij to the ith color bin.

Page 18: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 18Saturday, April 22, 2023

Feature Extraction [6] -- Texture

Texture is another important property of images. Basically, texture representation methods can be classified into two categories: structural and statistical.

Structural methods, including morphological operator and adjacency graph, describe texture by identifying structural primitives and their placement rules. They tend to be most effective when applied to textures that are very regular.

Statistical methods include, for example, in areas with smooth texture, the range of values in the neighborhood around a pixel will be a small value; in areas of rough texture, the range will be larger. Similarly, calculating the standard deviation of pixels in a neighborhood can indicate the degree of variability of pixel values in that region.

Page 19: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 19Saturday, April 22, 2023

Feature Extraction [7] -- Texture

Grey – level co-occurrence matrix

Page 20: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 20Saturday, April 22, 2023

Feature Extraction [8] -- Texture

Tamura Features – Three main features including Coarseness, contrast and directionality are extracted.

1) Coarseness is the measure of granularity of an image, or average size of regions that have the same intensity.

2) Contrast is the measure of brightness of the texture pattern. Therefore, the bigger the blocks that makes up the image, the higher the contrast.

3) Directionality is the measure of directions of the gray values within the image.

Page 21: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 21Saturday, April 22, 2023

Why Tamura Features ? [1]

Following are the features calculated from the normalized co-occurrence matrix P(i,j):

Page 22: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 22Saturday, April 22, 2023

Why Tamura Features ? [2]

An image will contain textures at several scales; coarseness aims to identify the largest size at which a texture exists, even where a smaller micro texture exists.

Here we first take averages at every point over neighborhoods, the linear size of which are powers of 2. The average over the neighborhood of size 2k * 2k at the point (x,y) is

Page 23: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 23Saturday, April 22, 2023

Why Tamura Features ? [3]

Then at each point one takes differences between pairs of averages corresponding to non-overlapping neighborhoods on opposite sides of the point in both horizontal vertical orientations. In the horizontal case this is

At each point, one then picks the best size which gives the highest output value, where k maximizes E in either direction. The Coarseness measure is then the average of Sopt (x,y) = 2kopt over the picture.

Page 24: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 24Saturday, April 22, 2023

Why Tamura Features ? [4]

Contrast aims to capture the dynamic range of gray levels in an image, together with the polarization of the distribution of black and white.

Here μ4 is the fourth moment about the mean and σ2 is the variance.

Directionality is a global property over a region. Detect the edges in an image. At each pixel the angle and magnitude are calculated. A histogram of edge probabilities is then built up by counting all points with magnitudes greater than a threshold and quantizing by the edge angle. The histogram will reflect the degree of directionality.

Page 26: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 26Saturday, April 22, 2023

Feature Extraction [10] -- Shape

Compared with Color and Texture features, shape features are usually described after images have been segmented into regions or objects.

Since robust and accurate image segmentation is difficult to achieve, the use of shape features for image retrieval has been limited to special applications where objects or regions are readily available.

The state-of-art methods for shape description can be categorized into either boundary-based polygonal approximation, finite element models and Fourier-based shape descriptors or region-based methods (statistical moments.)

A good shape representation feature for an object should be invariant to translation, rotation and scaling. Phase Congruency is one such feature.

Page 27: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 27Saturday, April 22, 2023

Feature Extraction [11] -- Shape

Algorithm phase_congruency:Input: Amplitude An(x)Phase angle n(x)Weighted mean phase angle (x)Weights for frequency spread w(x)To avoid division by zero, add a small

ComputePC1 (x) = E (x) / n An(x)

WhereE (x) = n An (cos ((x) - (x))To produce more localized response,PC1 (x) = w (x)E (x) -T / n An(x) + Compute distance measurement between query image and image in the database.Output:Retrieve similar images.

Page 28: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 28Saturday, April 22, 2023

Similarity / Distance Measure

Instead of exact matching, content-based image retrieval calculates visual similarities between a query image and images in a database. Accordingly, the retrieval result is not a single image but a list of images ranked by their similarities with the query image.

If each dimension of image feature vector is independent of each other and is of equal importance, the Minkowski-form distance is appropriate for calculating the distance between two images. This distance is defined as:

D( i, j ) = Σ fi (I) - fi (J) p 1/p

Page 29: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 29Saturday, April 22, 2023

Highlights and Novelty in the work

Using Rodriguez formula to normalize the images before taking the color Histogram.

Analysis of Statistical methods, including Fourier power spectra, co-occurrence matrices, Tamura feature, and multi-resolution filtering techniques such as Gabor and wavelet transform, for characterizing texture.

Phase Congruency measurement for Shape Information.

Page 30: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 30Saturday, April 22, 2023

Promising Directions and Open Issues

Relevance feedback is proposed as a technique for overcoming many of the problems faced by fully automatic systems by allowing the user to interact with the computer to improve retrieval performance.

More interestingly, the semantic gap is need to be bridged. Semantic gap here refers to “the large disparity between the low-level features or content descriptors that can be computed automatically by current machines and algorithms, and the richness and subjectivity of semantics in user queries and high-level human interpretations of audiovisual media”

Page 31: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 31Saturday, April 22, 2023

Performance Evaluation

The proposed technique is tested on two types of data sets, first one consisting of different animals and the second dataset consisting of birds, flowers and buildings.

The retrieval accuracy is found to be 96.4% and 92.2% for a database size of 55 each.

Page 32: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 32Saturday, April 22, 2023

Database (Partial Set)

Page 33: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 33Saturday, April 22, 2023

Results [1]

QUERY IMAGE

Page 34: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 34Saturday, April 22, 2023

Results [2] -- Color

Page 35: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 35Saturday, April 22, 2023

Results [3] – Color + Texture

Page 36: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 36Saturday, April 22, 2023

References M. Flickner, H. Sawhney, W. Niblack, and P. Yanker, “Querying by image

and video content: The QBIC system,” IEEE Trans. Comput., vol. 28, pp. 23–32, 1995.

James Z. Wang, Jia Li and Gio, “SIMPLIcity: Semantics – sensitive Integrated Matching for picture Libraries,” IEEE Transactions on pattern Analysis and Machine Intelligence, Vol.23,no.9,pp.947-963,2001.

Smeulders, Senior Member, IEEE, Santini, Member, IEEE, “Content-Based Image Retrieval at the End of the Early Years”, IEEE transactions on pattern analysis and machine intelligence, vol. 22, no. 12, december 2000

K.Satya Sai Prakash, RMD. Sundaram, “Shape Information from Phase Congruency and its application in Content based Image Retrieval”, Proceedings of the 3rd Workshop on Computer Vision, Graphics and Image Processing – WCVGIP 2006, Hyderabad PP.88-93, 2006.

Page 37: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 37Saturday, April 22, 2023

References J. van de Weijer and J. M. Geusebroek. Color edge and

corner detection by photometric quasi-invariants. IEEE Trans. Pattern Anal. Machine Intel. 27(4):625-630, 2005.

Barbeau Jerome, Vignes-Lebbe Regine, and Stamon Georges, “A Signature based on Delaney Graph and Co-occurrence Matrix,” Laboratories Informatique et Systematique, University of Paris, Paris, France, July 2002, Found at: http://www.math-info.univ-paris5.fr/sip-lab/barbeque/barbeque.pdf

Page 38: Image Mining – Intricacies and Innovations

Content Based Image Retrieval 38Saturday, April 22, 2023

THANK YOU

[email protected]

Discussions????