content based image retrieval techniques · 2020-05-07 · content based image recognition to use...
TRANSCRIPT
![Page 1: Content Based Image Retrieval Techniques · 2020-05-07 · Content based Image Recognition To use local features for image retrieval, three different methods are available: Direct](https://reader034.vdocuments.site/reader034/viewer/2022050106/5f44ad35955311791a6b2543/html5/thumbnails/1.jpg)
Content Based Image Retrieval Techniques
Ambrose Tuscano ([email protected])University of Maryland Baltimore County,
CMSC 676 Information Retrieval
![Page 2: Content Based Image Retrieval Techniques · 2020-05-07 · Content based Image Recognition To use local features for image retrieval, three different methods are available: Direct](https://reader034.vdocuments.site/reader034/viewer/2022050106/5f44ad35955311791a6b2543/html5/thumbnails/2.jpg)
Introduction
Image retrieval systems aim to find similar images to a query image among an image dataset.
![Page 3: Content Based Image Retrieval Techniques · 2020-05-07 · Content based Image Recognition To use local features for image retrieval, three different methods are available: Direct](https://reader034.vdocuments.site/reader034/viewer/2022050106/5f44ad35955311791a6b2543/html5/thumbnails/3.jpg)
Represented as
• Pixels (Also called Rasters)
• Vectors
![Page 4: Content Based Image Retrieval Techniques · 2020-05-07 · Content based Image Recognition To use local features for image retrieval, three different methods are available: Direct](https://reader034.vdocuments.site/reader034/viewer/2022050106/5f44ad35955311791a6b2543/html5/thumbnails/4.jpg)
● By annotation (manual) • Text retrieval • Semantic level (good for picture with people, architectures)
● By the content (automatic) • Color, texture, shape • Vague description of picture (good for pictures of scenery and with pattern and texture)
![Page 5: Content Based Image Retrieval Techniques · 2020-05-07 · Content based Image Recognition To use local features for image retrieval, three different methods are available: Direct](https://reader034.vdocuments.site/reader034/viewer/2022050106/5f44ad35955311791a6b2543/html5/thumbnails/5.jpg)
Features in an Image
● Color : Low level, Can't specify context.
● Texture : Produce a mathematical characterisation of a repeating pattern in the image.
● Shape: Region based and Contour(outline) based.
● Local Image Features : small parts of a big image. ○ extracted from the images at salient points and dimensionality
reduced using Principal Component Analysis (PCA) transformation○ SIFT using Harris interest points
![Page 6: Content Based Image Retrieval Techniques · 2020-05-07 · Content based Image Recognition To use local features for image retrieval, three different methods are available: Direct](https://reader034.vdocuments.site/reader034/viewer/2022050106/5f44ad35955311791a6b2543/html5/thumbnails/6.jpg)
Structure
![Page 7: Content Based Image Retrieval Techniques · 2020-05-07 · Content based Image Recognition To use local features for image retrieval, three different methods are available: Direct](https://reader034.vdocuments.site/reader034/viewer/2022050106/5f44ad35955311791a6b2543/html5/thumbnails/7.jpg)
IMAGE RETRIEVAL METHODS
![Page 8: Content Based Image Retrieval Techniques · 2020-05-07 · Content based Image Recognition To use local features for image retrieval, three different methods are available: Direct](https://reader034.vdocuments.site/reader034/viewer/2022050106/5f44ad35955311791a6b2543/html5/thumbnails/8.jpg)
Text based Image RetrievalFirst annotated the images by text and then used text-based database management systems to perform image retrieval.
![Page 9: Content Based Image Retrieval Techniques · 2020-05-07 · Content based Image Recognition To use local features for image retrieval, three different methods are available: Direct](https://reader034.vdocuments.site/reader034/viewer/2022050106/5f44ad35955311791a6b2543/html5/thumbnails/9.jpg)
Text based Image Retrieval
Three Ways to go● Manually Assign Keywords to each image
● Use text associated with the images (captions, web pages)
● Analyse the image content to automatically assign keywords(Computer Vision?)
![Page 10: Content Based Image Retrieval Techniques · 2020-05-07 · Content based Image Recognition To use local features for image retrieval, three different methods are available: Direct](https://reader034.vdocuments.site/reader034/viewer/2022050106/5f44ad35955311791a6b2543/html5/thumbnails/10.jpg)
Content Based Image Recognition
● A technique which uses visual contents to search images from large scale image databases according to users' interests.
● CBIR research is mainly contributed by the computer vision community
![Page 11: Content Based Image Retrieval Techniques · 2020-05-07 · Content based Image Recognition To use local features for image retrieval, three different methods are available: Direct](https://reader034.vdocuments.site/reader034/viewer/2022050106/5f44ad35955311791a6b2543/html5/thumbnails/11.jpg)
Content based Image Recognition
To use local features for image retrieval, three different methods are available:
● Direct transfer: nearest neighbors for each of the local features of the query searched and the database images containing most of these neighbors returned.
● Local feature image distortion model (LFIDM): Compares the distances between local features from the query image to the local features of each image of the database . The images with the lowest total distances are returned.
● Histograms of local features: A reasonably large amount of local features from the database is clustered and then each database image represented by a histogram of indices of these clusters.
![Page 12: Content Based Image Retrieval Techniques · 2020-05-07 · Content based Image Recognition To use local features for image retrieval, three different methods are available: Direct](https://reader034.vdocuments.site/reader034/viewer/2022050106/5f44ad35955311791a6b2543/html5/thumbnails/12.jpg)
● Color Histogram● Color Correlogram● Color AutoCorrelogram● Color Coherence vector● Dominant Color Descriptors
![Page 13: Content Based Image Retrieval Techniques · 2020-05-07 · Content based Image Recognition To use local features for image retrieval, three different methods are available: Direct](https://reader034.vdocuments.site/reader034/viewer/2022050106/5f44ad35955311791a6b2543/html5/thumbnails/13.jpg)
● A shape is the form of an object or its external boundary, outline,or external surface, as opposed to other properties like color, texture or composition.
● Fourier Descriptors● Canny Algorithm● SIFT Descriptors● Moment Invariants● Eccentric and Axis Oriented
![Page 14: Content Based Image Retrieval Techniques · 2020-05-07 · Content based Image Recognition To use local features for image retrieval, three different methods are available: Direct](https://reader034.vdocuments.site/reader034/viewer/2022050106/5f44ad35955311791a6b2543/html5/thumbnails/14.jpg)
●
●●●●
![Page 15: Content Based Image Retrieval Techniques · 2020-05-07 · Content based Image Recognition To use local features for image retrieval, three different methods are available: Direct](https://reader034.vdocuments.site/reader034/viewer/2022050106/5f44ad35955311791a6b2543/html5/thumbnails/15.jpg)
●●●
○
●○○○
![Page 16: Content Based Image Retrieval Techniques · 2020-05-07 · Content based Image Recognition To use local features for image retrieval, three different methods are available: Direct](https://reader034.vdocuments.site/reader034/viewer/2022050106/5f44ad35955311791a6b2543/html5/thumbnails/16.jpg)
● Smoothing: Blur image to remove Noise
● FInd Gradients : Edges are marked where gradients of image have large magnitudes.
● Non-Max Suppression: Only local Maxima is marked for edges.
● Double Thresholding: Potential Edges are determined
● Hysteresis : Finally Edges which are not connected/near to many other potential edges are removed.
![Page 17: Content Based Image Retrieval Techniques · 2020-05-07 · Content based Image Recognition To use local features for image retrieval, three different methods are available: Direct](https://reader034.vdocuments.site/reader034/viewer/2022050106/5f44ad35955311791a6b2543/html5/thumbnails/17.jpg)
Texture Extraction- Motif Co-Occurrence Matric● MCM is used to represent transveral
of adjacent pixel color difference in an image.
● Each Pixel corresponds to four adjacent pixel colors
● Each image can be presented by four images of motifs of scan pattern, which can be further constructed into four two dimensional matrices of the image size.
● The attribute of the image will be computed with motifs of scan pattern and a color motif cooccurence matrix(CMCM) will be obtained
![Page 18: Content Based Image Retrieval Techniques · 2020-05-07 · Content based Image Recognition To use local features for image retrieval, three different methods are available: Direct](https://reader034.vdocuments.site/reader034/viewer/2022050106/5f44ad35955311791a6b2543/html5/thumbnails/18.jpg)
● Euclidean DIstance● Mahalanobis Distance● MInkowski Distance● Histogram Intersection Distance● Quadratic Form Distance
![Page 19: Content Based Image Retrieval Techniques · 2020-05-07 · Content based Image Recognition To use local features for image retrieval, three different methods are available: Direct](https://reader034.vdocuments.site/reader034/viewer/2022050106/5f44ad35955311791a6b2543/html5/thumbnails/19.jpg)
Techniques used by CBIR
● K-MeansK-means clustering algorithm is proposed as it improves the scalability.
● Wavelet TransformFeature vectors of images are be constructed from wavelet transformations,
which can also be utilized to distinguish images through measuring distances between feature vectors.
● Support Vector Machine: SVM classifier can be trained using training data of images marked by users .
● Neural Network:A CNN doesn’t need complex work like feature extraction to work. Having a
proper labelled data, we can train the system to learn the data features using complex layer structure.
![Page 20: Content Based Image Retrieval Techniques · 2020-05-07 · Content based Image Recognition To use local features for image retrieval, three different methods are available: Direct](https://reader034.vdocuments.site/reader034/viewer/2022050106/5f44ad35955311791a6b2543/html5/thumbnails/20.jpg)
CBIR + TBIR◦CBIR can be costly in the fact that it needs a lot of complex computations.
◦TBIR can be comparatively fast but has low precision.◦A hybrid model is currently being implemented.
◦a text-based image meta-search engine retrieves images from the Web using the text information from the Query.
◦ Techniques like matching term frequency-inverse document frequency (tf-idf) weightings and cosine similarity are used.
◦use the CBIR approach to re-filter the search results.
![Page 21: Content Based Image Retrieval Techniques · 2020-05-07 · Content based Image Recognition To use local features for image retrieval, three different methods are available: Direct](https://reader034.vdocuments.site/reader034/viewer/2022050106/5f44ad35955311791a6b2543/html5/thumbnails/21.jpg)
● X.Y. Wang,Y.J. Hong and H.Y.Yang,”An effective image retrieval scheme using color,
texture and shape features”
● Nidhi Singh ,Kanchan Singh and Ashok Sinha “A Novel Approach for Content Based Image
Retrieval”
● Yogita Mistry,and D. T. Ingole “Survey on Content Based Image Retrieval Systems”
● John Canny, “A Computational Approach to Edge Detection”
● Mussarat Yasmin, Muhammad Sharif and Sajjad Mohsin, “Use of Low Level Features for
Content Based Image Retrieval: Survey”
● N. Jhanwar, Subhasis Chaudhuri, et.al. Content based image retrieval using motif
cooccurrence matrix