a sample recognition problem
DESCRIPTION
A Sample Recognition Problem. Joseph Tighe University of North Carolina at Chapel Hill. What is recognition?. What is recognition?. What is recognition?. What is recognition?. Figure from Shotton et al. (2009). - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: A Sample Recognition Problem](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681619e550346895dd15482/html5/thumbnails/1.jpg)
A SAMPLE RECOGNITION PROBLEM
Joseph Tighe University of North Carolina at Chapel Hill
![Page 2: A Sample Recognition Problem](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681619e550346895dd15482/html5/thumbnails/2.jpg)
What is recognition?
![Page 3: A Sample Recognition Problem](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681619e550346895dd15482/html5/thumbnails/3.jpg)
What is recognition?
![Page 4: A Sample Recognition Problem](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681619e550346895dd15482/html5/thumbnails/4.jpg)
What is recognition?
![Page 5: A Sample Recognition Problem](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681619e550346895dd15482/html5/thumbnails/5.jpg)
What is recognition?
He et al. (2004), Hoiem et al. (2005), Shotton et al. (2006, 2008, 2009), Verbeek and Triggs (2007), Rabinovich et al. (2007), Galleguillos et al. (2008), Gould et al. (2009), etc.
Figure from Shotton et al. (2009)
![Page 6: A Sample Recognition Problem](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681619e550346895dd15482/html5/thumbnails/6.jpg)
What is recognition? Have some entity
photo bounding box in photo pixel
Assign semantic meaning scene type, tags, sentence object label, action semantic class, material, geometric
orientation
![Page 7: A Sample Recognition Problem](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681619e550346895dd15482/html5/thumbnails/7.jpg)
Our Problem Entity: Images Goal: Assign 1 of 4 labels (airplane, car,
face, motorbike
![Page 8: A Sample Recognition Problem](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681619e550346895dd15482/html5/thumbnails/8.jpg)
Finding Similar Images
![Page 9: A Sample Recognition Problem](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681619e550346895dd15482/html5/thumbnails/9.jpg)
Ocean
Open Field
Highway
Street
Forest
Mountain
Inner City
Tall BuildingWhat is depicted in this image?
Which image is most similar?
Then assign the label from the most similar image
![Page 10: A Sample Recognition Problem](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681619e550346895dd15482/html5/thumbnails/10.jpg)
Pixels are a bad measure of similarity
Most similar according to pixel distance Most similar according to “Bag of Words”
![Page 11: A Sample Recognition Problem](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681619e550346895dd15482/html5/thumbnails/11.jpg)
Origin of the Bag of Words model
Orderless document representation: frequencies of words from a dictionary Salton & McGill (1983)
US Presidential Speeches Tag Cloudhttp://chir.ag/phernalia/preztags/
![Page 12: A Sample Recognition Problem](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681619e550346895dd15482/html5/thumbnails/12.jpg)
What are words for an image?
![Page 13: A Sample Recognition Problem](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681619e550346895dd15482/html5/thumbnails/13.jpg)
![Page 14: A Sample Recognition Problem](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681619e550346895dd15482/html5/thumbnails/14.jpg)
![Page 15: A Sample Recognition Problem](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681619e550346895dd15482/html5/thumbnails/15.jpg)
![Page 16: A Sample Recognition Problem](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681619e550346895dd15482/html5/thumbnails/16.jpg)
![Page 17: A Sample Recognition Problem](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681619e550346895dd15482/html5/thumbnails/17.jpg)
Wing Tail
WheelBuildingPropeller
![Page 18: A Sample Recognition Problem](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681619e550346895dd15482/html5/thumbnails/18.jpg)
Wing TailWheelBuildingPropeller Jet Engine
![Page 19: A Sample Recognition Problem](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681619e550346895dd15482/html5/thumbnails/19.jpg)
Wing TailWheelBuildingPropeller Jet Engine
![Page 20: A Sample Recognition Problem](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681619e550346895dd15482/html5/thumbnails/20.jpg)
Wing TailWheelBuildingPropeller Jet Engine
![Page 21: A Sample Recognition Problem](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681619e550346895dd15482/html5/thumbnails/21.jpg)
But where do the words come from?
![Page 22: A Sample Recognition Problem](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681619e550346895dd15482/html5/thumbnails/22.jpg)
![Page 23: A Sample Recognition Problem](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681619e550346895dd15482/html5/thumbnails/23.jpg)
Then where does the dictionary come from?
![Page 24: A Sample Recognition Problem](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681619e550346895dd15482/html5/thumbnails/24.jpg)
Example Dictionary
Source: B. Leibe
![Page 25: A Sample Recognition Problem](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681619e550346895dd15482/html5/thumbnails/25.jpg)
Another dictionary
………
…
Source: B. Leibe
![Page 26: A Sample Recognition Problem](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681619e550346895dd15482/html5/thumbnails/26.jpg)
Fei-Fei et al. 2005
![Page 27: A Sample Recognition Problem](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681619e550346895dd15482/html5/thumbnails/27.jpg)
Outline of the Bag of Words method
Divide the image into patches Assign a “word” for each patch Count the number of occurrences of
each “word” in the image
![Page 28: A Sample Recognition Problem](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681619e550346895dd15482/html5/thumbnails/28.jpg)
Matlab Demo