![Page 1: Content Based Image Retrieval Romit Das Ryan Scotka](https://reader036.vdocuments.site/reader036/viewer/2022062223/5a4d1b927f8b9ab0599c2189/html5/thumbnails/1.jpg)
Content Based Image Retrieval
Romit Das · Ryan Scotka
![Page 3: Content Based Image Retrieval Romit Das Ryan Scotka](https://reader036.vdocuments.site/reader036/viewer/2022062223/5a4d1b927f8b9ab0599c2189/html5/thumbnails/3.jpg)
GIS Problems
• Search based on filename– Verbatim match– Noun replacement
• Potential for Abuse (Google Hack)
![Page 4: Content Based Image Retrieval Romit Das Ryan Scotka](https://reader036.vdocuments.site/reader036/viewer/2022062223/5a4d1b927f8b9ab0599c2189/html5/thumbnails/4.jpg)
Possible Solutions
• Metadata– Standards– Re-index existing images
• Manual Classification– Time
• Content-based Classification
![Page 5: Content Based Image Retrieval Romit Das Ryan Scotka](https://reader036.vdocuments.site/reader036/viewer/2022062223/5a4d1b927f8b9ab0599c2189/html5/thumbnails/5.jpg)
CBIR – Training
1. Choose features to distinguish images.2. Extract said features.3. Apply statistical method to model
features.4. Categorize based on textual description.
![Page 6: Content Based Image Retrieval Romit Das Ryan Scotka](https://reader036.vdocuments.site/reader036/viewer/2022062223/5a4d1b927f8b9ab0599c2189/html5/thumbnails/6.jpg)
ExampleDimensions
Color Frequencies
Spatial Distribution
200 x 200 + Mostly flesh tones + Flesh tones concentrated in the center =
baby
![Page 7: Content Based Image Retrieval Romit Das Ryan Scotka](https://reader036.vdocuments.site/reader036/viewer/2022062223/5a4d1b927f8b9ab0599c2189/html5/thumbnails/7.jpg)
Author’s Feature Set
• Feature Set (6 dimensions):– Color averages (LUV)– High-frequency energy bands
• “Effectively discern local texture”• Wavelet transform on 4x4 blocks• Use HL, LH, and HH “high energy bands”• Use the LL for lower resolution analysis
![Page 8: Content Based Image Retrieval Romit Das Ryan Scotka](https://reader036.vdocuments.site/reader036/viewer/2022062223/5a4d1b927f8b9ab0599c2189/html5/thumbnails/8.jpg)
Author’s Implementation
• Statistical Modeling– Use machine learning to build concepts
Concept = Paris
Training Set =
![Page 9: Content Based Image Retrieval Romit Das Ryan Scotka](https://reader036.vdocuments.site/reader036/viewer/2022062223/5a4d1b927f8b9ab0599c2189/html5/thumbnails/9.jpg)
Markov Models
• Take known facts• Deduce hidden/unknown data
![Page 10: Content Based Image Retrieval Romit Das Ryan Scotka](https://reader036.vdocuments.site/reader036/viewer/2022062223/5a4d1b927f8b9ab0599c2189/html5/thumbnails/10.jpg)
Markov Model Example
• Given:– Queues of people, shelves, price labels,
disgruntled workers• Possible Results:
– Post office– Supermarket– Record Store
![Page 11: Content Based Image Retrieval Romit Das Ryan Scotka](https://reader036.vdocuments.site/reader036/viewer/2022062223/5a4d1b927f8b9ab0599c2189/html5/thumbnails/11.jpg)
Markov Model Example
• Given:– Queues of people, shelves, price labels,
disgruntled workers, food products• Possible Results:
– Post office– Supermarket– Record Store
![Page 12: Content Based Image Retrieval Romit Das Ryan Scotka](https://reader036.vdocuments.site/reader036/viewer/2022062223/5a4d1b927f8b9ab0599c2189/html5/thumbnails/12.jpg)
Ninja ModelPerson, outdoors
![Page 13: Content Based Image Retrieval Romit Das Ryan Scotka](https://reader036.vdocuments.site/reader036/viewer/2022062223/5a4d1b927f8b9ab0599c2189/html5/thumbnails/13.jpg)
Ninja ModelPeople, ninjas, outdoor
![Page 14: Content Based Image Retrieval Romit Das Ryan Scotka](https://reader036.vdocuments.site/reader036/viewer/2022062223/5a4d1b927f8b9ab0599c2189/html5/thumbnails/14.jpg)
Ninja ModelPeople, ninjas, weapons, outdoors
![Page 15: Content Based Image Retrieval Romit Das Ryan Scotka](https://reader036.vdocuments.site/reader036/viewer/2022062223/5a4d1b927f8b9ab0599c2189/html5/thumbnails/15.jpg)
Ninja Markov Model
Person, outdoors
People, ninjas, outdoors
People, ninjas, outdoors
weapons, class photo
![Page 16: Content Based Image Retrieval Romit Das Ryan Scotka](https://reader036.vdocuments.site/reader036/viewer/2022062223/5a4d1b927f8b9ab0599c2189/html5/thumbnails/16.jpg)
Creating Concepts
• Training Concept– Created from hand-picked images– Must choose statistically significant training
size• Resulting Concept
– Used in automatic cataloging of future images
![Page 17: Content Based Image Retrieval Romit Das Ryan Scotka](https://reader036.vdocuments.site/reader036/viewer/2022062223/5a4d1b927f8b9ab0599c2189/html5/thumbnails/17.jpg)
![Page 18: Content Based Image Retrieval Romit Das Ryan Scotka](https://reader036.vdocuments.site/reader036/viewer/2022062223/5a4d1b927f8b9ab0599c2189/html5/thumbnails/18.jpg)
Observations
• Images are associated with multiple concepts.
• Not foolproof• Example:
People, ninjas, outdoors
weapons, class photo
![Page 19: Content Based Image Retrieval Romit Das Ryan Scotka](https://reader036.vdocuments.site/reader036/viewer/2022062223/5a4d1b927f8b9ab0599c2189/html5/thumbnails/19.jpg)
Advantages
• Automatic categorization
![Page 20: Content Based Image Retrieval Romit Das Ryan Scotka](https://reader036.vdocuments.site/reader036/viewer/2022062223/5a4d1b927f8b9ab0599c2189/html5/thumbnails/20.jpg)
Disadvantages
• False positives– Concepts may require a vast amount of
images• Increases training time
• Dissimilar images needed for training of a concept
![Page 21: Content Based Image Retrieval Romit Das Ryan Scotka](https://reader036.vdocuments.site/reader036/viewer/2022062223/5a4d1b927f8b9ab0599c2189/html5/thumbnails/21.jpg)
Future Additions
• Further refinement of conflicting semantics• Weights assigned to classifications
![Page 22: Content Based Image Retrieval Romit Das Ryan Scotka](https://reader036.vdocuments.site/reader036/viewer/2022062223/5a4d1b927f8b9ab0599c2189/html5/thumbnails/22.jpg)
Our Implementation
• Perform classification with alternate learners (Weka)