richard russellcbcl.mit.edu/people/sinha-lab.pdf · 2001. 3. 29. · - dirty deeds, done dirt cheap...
TRANSCRIPT
![Page 1: Richard Russellcbcl.mit.edu/people/sinha-lab.pdf · 2001. 3. 29. · - Dirty Deeds, Done Dirt Cheap - The Cambridgeport Saloon - Community-Based Policing What I like least about MIT/Cambridge:-WHAT](https://reader035.vdocuments.site/reader035/viewer/2022071516/613876730ad5d2067649459f/html5/thumbnails/1.jpg)
Richard RussellGraduate [email protected]
![Page 2: Richard Russellcbcl.mit.edu/people/sinha-lab.pdf · 2001. 3. 29. · - Dirty Deeds, Done Dirt Cheap - The Cambridgeport Saloon - Community-Based Policing What I like least about MIT/Cambridge:-WHAT](https://reader035.vdocuments.site/reader035/viewer/2022071516/613876730ad5d2067649459f/html5/thumbnails/2.jpg)
The Problems:
Which computational image similarity metrics correspond most closely to human impressions of image similarity?
How can image compression be optimized?
What makes images look similar or dissimilar?
How do humans perform comparisons of images?
![Page 3: Richard Russellcbcl.mit.edu/people/sinha-lab.pdf · 2001. 3. 29. · - Dirty Deeds, Done Dirt Cheap - The Cambridgeport Saloon - Community-Based Policing What I like least about MIT/Cambridge:-WHAT](https://reader035.vdocuments.site/reader035/viewer/2022071516/613876730ad5d2067649459f/html5/thumbnails/3.jpg)
Comparisons of different image similarity metrics
L1 matchΣ | xi-yi |
L2 matchΣ (xi-yi)2
Original Image
![Page 4: Richard Russellcbcl.mit.edu/people/sinha-lab.pdf · 2001. 3. 29. · - Dirty Deeds, Done Dirt Cheap - The Cambridgeport Saloon - Community-Based Policing What I like least about MIT/Cambridge:-WHAT](https://reader035.vdocuments.site/reader035/viewer/2022071516/613876730ad5d2067649459f/html5/thumbnails/4.jpg)
Vector quantization style image compression using different similarity metrics
L1 Metric L2 MetricOriginal
![Page 5: Richard Russellcbcl.mit.edu/people/sinha-lab.pdf · 2001. 3. 29. · - Dirty Deeds, Done Dirt Cheap - The Cambridgeport Saloon - Community-Based Policing What I like least about MIT/Cambridge:-WHAT](https://reader035.vdocuments.site/reader035/viewer/2022071516/613876730ad5d2067649459f/html5/thumbnails/5.jpg)
What I like most about MIT/Cambridge:- ‘All under one roof’- living in Boston - free espresso
What I like least about MIT/Cambridge:- expensive - winter - can’t buy booze on Sunday
![Page 7: Richard Russellcbcl.mit.edu/people/sinha-lab.pdf · 2001. 3. 29. · - Dirty Deeds, Done Dirt Cheap - The Cambridgeport Saloon - Community-Based Policing What I like least about MIT/Cambridge:-WHAT](https://reader035.vdocuments.site/reader035/viewer/2022071516/613876730ad5d2067649459f/html5/thumbnails/7.jpg)
What is the role of 3D in image processing?
Does 3D scene information facilitate object search?
How are different 3D scene attributes, such as illumination,represented by the visual system?
Does object recognition influence 3D form perception?
The Problems:
![Page 8: Richard Russellcbcl.mit.edu/people/sinha-lab.pdf · 2001. 3. 29. · - Dirty Deeds, Done Dirt Cheap - The Cambridgeport Saloon - Community-Based Policing What I like least about MIT/Cambridge:-WHAT](https://reader035.vdocuments.site/reader035/viewer/2022071516/613876730ad5d2067649459f/html5/thumbnails/8.jpg)
Exploring the influence of 3D scene layout on visual search
Good3D
Bad3D
1000
1200
1400
1600
1800
2000
2200
2400
2600
2800
3000
Rea
ctio
n T
ime
(ms)
1 2 3 4 5 6 7Trial Number for a Given Word
Good 3D
Bad 3D
Instant Good3D advantage
Amount of training
How fast do people find objects in Good3D vs. Bad3D?
![Page 9: Richard Russellcbcl.mit.edu/people/sinha-lab.pdf · 2001. 3. 29. · - Dirty Deeds, Done Dirt Cheap - The Cambridgeport Saloon - Community-Based Policing What I like least about MIT/Cambridge:-WHAT](https://reader035.vdocuments.site/reader035/viewer/2022071516/613876730ad5d2067649459f/html5/thumbnails/9.jpg)
Exploring the representation of 3D scene attributes such as illumination
![Page 10: Richard Russellcbcl.mit.edu/people/sinha-lab.pdf · 2001. 3. 29. · - Dirty Deeds, Done Dirt Cheap - The Cambridgeport Saloon - Community-Based Policing What I like least about MIT/Cambridge:-WHAT](https://reader035.vdocuments.site/reader035/viewer/2022071516/613876730ad5d2067649459f/html5/thumbnails/10.jpg)
What I like most about MIT/Cambridge:- The inexhaustible supply of nerdy rhetoric- Those charming long-time residents that scream at you for riding your bike on the sidewalk
- Parades, parades, parades
What I like least about MIT/Cambridge:- Too much sunshine
![Page 12: Richard Russellcbcl.mit.edu/people/sinha-lab.pdf · 2001. 3. 29. · - Dirty Deeds, Done Dirt Cheap - The Cambridgeport Saloon - Community-Based Policing What I like least about MIT/Cambridge:-WHAT](https://reader035.vdocuments.site/reader035/viewer/2022071516/613876730ad5d2067649459f/html5/thumbnails/12.jpg)
Can we develop a unified experimental paradigm for exploring different issues in object perception?
What are the neural correlates of conscious object percepts?
Can we develop novel, quantitative measures of objectagnosias, development, and priming?
How are object concepts learned, especially in the absence of supervision and image normalization?
The Problems:
![Page 13: Richard Russellcbcl.mit.edu/people/sinha-lab.pdf · 2001. 3. 29. · - Dirty Deeds, Done Dirt Cheap - The Cambridgeport Saloon - Community-Based Policing What I like least about MIT/Cambridge:-WHAT](https://reader035.vdocuments.site/reader035/viewer/2022071516/613876730ad5d2067649459f/html5/thumbnails/13.jpg)
The RISE Paradigm for Exploring Issues in Object Perception
Idea of Object-Image Space andRandom Image Structure Evolution....
... Bit-Flip RISE
... Phase RISE
manipulatephase
![Page 14: Richard Russellcbcl.mit.edu/people/sinha-lab.pdf · 2001. 3. 29. · - Dirty Deeds, Done Dirt Cheap - The Cambridgeport Saloon - Community-Based Policing What I like least about MIT/Cambridge:-WHAT](https://reader035.vdocuments.site/reader035/viewer/2022071516/613876730ad5d2067649459f/html5/thumbnails/14.jpg)
A sample RISE sequence
![Page 15: Richard Russellcbcl.mit.edu/people/sinha-lab.pdf · 2001. 3. 29. · - Dirty Deeds, Done Dirt Cheap - The Cambridgeport Saloon - Community-Based Policing What I like least about MIT/Cambridge:-WHAT](https://reader035.vdocuments.site/reader035/viewer/2022071516/613876730ad5d2067649459f/html5/thumbnails/15.jpg)
What I like most about MIT/Cambridge:- the Rev. Sayan Mukherjee, Ph.D.- Top-Notch Local Weather Coverage- Dirty Deeds, Done Dirt Cheap- The Cambridgeport Saloon- Community-Based Policing
What I like least about MIT/Cambridge:- WHAT !- SOMEONE SET UP US THE BOMB !
![Page 17: Richard Russellcbcl.mit.edu/people/sinha-lab.pdf · 2001. 3. 29. · - Dirty Deeds, Done Dirt Cheap - The Cambridgeport Saloon - Community-Based Policing What I like least about MIT/Cambridge:-WHAT](https://reader035.vdocuments.site/reader035/viewer/2022071516/613876730ad5d2067649459f/html5/thumbnails/17.jpg)
How do we recognize faces?
What aspects of a face do caricaturists capture to make super-recognizable portraits?
What features are important for recognizing faces that are blurred or far away?
How are objects represented in the brain?
The Problems:
![Page 18: Richard Russellcbcl.mit.edu/people/sinha-lab.pdf · 2001. 3. 29. · - Dirty Deeds, Done Dirt Cheap - The Cambridgeport Saloon - Community-Based Policing What I like least about MIT/Cambridge:-WHAT](https://reader035.vdocuments.site/reader035/viewer/2022071516/613876730ad5d2067649459f/html5/thumbnails/18.jpg)
Caricaturists have an intuitive sense of what features make a face recognizable as a unique individual.
By creating a database of caricatures and real faces and developing computational schemes for extracting consistencies across the images of specific individuals, we hope to determine what aspects of a face define its recognizability.
![Page 19: Richard Russellcbcl.mit.edu/people/sinha-lab.pdf · 2001. 3. 29. · - Dirty Deeds, Done Dirt Cheap - The Cambridgeport Saloon - Community-Based Policing What I like least about MIT/Cambridge:-WHAT](https://reader035.vdocuments.site/reader035/viewer/2022071516/613876730ad5d2067649459f/html5/thumbnails/19.jpg)
Even when faces are far away or blurred and individual features (such as eyes, nose, lips) are difficult to discern, people are still able to recognize specific individuals.
One goal is to define the heuristics used by human observers by determining the capabilities and limitations in face recognitionwith degraded images.
![Page 20: Richard Russellcbcl.mit.edu/people/sinha-lab.pdf · 2001. 3. 29. · - Dirty Deeds, Done Dirt Cheap - The Cambridgeport Saloon - Community-Based Policing What I like least about MIT/Cambridge:-WHAT](https://reader035.vdocuments.site/reader035/viewer/2022071516/613876730ad5d2067649459f/html5/thumbnails/20.jpg)
What I like most about MIT/Cambridge:• Students and Faculty with diverse interests.• Academic resources and talks.• Art loan program.
What I like least about MIT/Cambridge:• Tow trucks.• Lack of late night coffee-shops.• Slush.
![Page 22: Richard Russellcbcl.mit.edu/people/sinha-lab.pdf · 2001. 3. 29. · - Dirty Deeds, Done Dirt Cheap - The Cambridgeport Saloon - Community-Based Policing What I like least about MIT/Cambridge:-WHAT](https://reader035.vdocuments.site/reader035/viewer/2022071516/613876730ad5d2067649459f/html5/thumbnails/22.jpg)
How can one extract automatically the invariant features of an arbitrary class of objects?
Can a reliable object invariant be learned from a handful of examples?
What kind of object model could be used to encode low resolution images to facilitate detection at a distance?
The Problems:
![Page 23: Richard Russellcbcl.mit.edu/people/sinha-lab.pdf · 2001. 3. 29. · - Dirty Deeds, Done Dirt Cheap - The Cambridgeport Saloon - Community-Based Policing What I like least about MIT/Cambridge:-WHAT](https://reader035.vdocuments.site/reader035/viewer/2022071516/613876730ad5d2067649459f/html5/thumbnails/23.jpg)
A qualitative invariant object representation.
The ratio template is a compact object representation, robust to changes in illumination, noise and image degradation.
Emphasizes low-frequency (low-res) spectral components.
![Page 24: Richard Russellcbcl.mit.edu/people/sinha-lab.pdf · 2001. 3. 29. · - Dirty Deeds, Done Dirt Cheap - The Cambridgeport Saloon - Community-Based Policing What I like least about MIT/Cambridge:-WHAT](https://reader035.vdocuments.site/reader035/viewer/2022071516/613876730ad5d2067649459f/html5/thumbnails/24.jpg)
Detection results on faces.
Note the range of spatial resolution and illumination differences.
![Page 25: Richard Russellcbcl.mit.edu/people/sinha-lab.pdf · 2001. 3. 29. · - Dirty Deeds, Done Dirt Cheap - The Cambridgeport Saloon - Community-Based Policing What I like least about MIT/Cambridge:-WHAT](https://reader035.vdocuments.site/reader035/viewer/2022071516/613876730ad5d2067649459f/html5/thumbnails/25.jpg)
What I like most about MIT/Cambridge:- New England sports (SCUBA, snowboarding)- Student-sponsored activities- Variety of lectures and talks institute-wide- Academic freedom and resources
What I like least about MIT/Cambridge:- No late-night food options- Classes
![Page 27: Richard Russellcbcl.mit.edu/people/sinha-lab.pdf · 2001. 3. 29. · - Dirty Deeds, Done Dirt Cheap - The Cambridgeport Saloon - Community-Based Policing What I like least about MIT/Cambridge:-WHAT](https://reader035.vdocuments.site/reader035/viewer/2022071516/613876730ad5d2067649459f/html5/thumbnails/27.jpg)
What is the role of contextual information inobject recognition?
How can we efficiently represent context structure?
How can we computationally model contextual influenceson object recognition?
What are the neural correlates of contextual processing?
The Problems:
![Page 28: Richard Russellcbcl.mit.edu/people/sinha-lab.pdf · 2001. 3. 29. · - Dirty Deeds, Done Dirt Cheap - The Cambridgeport Saloon - Community-Based Policing What I like least about MIT/Cambridge:-WHAT](https://reader035.vdocuments.site/reader035/viewer/2022071516/613876730ad5d2067649459f/html5/thumbnails/28.jpg)
Contextual influences in object recognitionObject recognitionperformances
6 10 32
6 10 32
6 10 32
Object intrinsic features
Object background features
Object in context
c/i
c/i
c/i
%
%
%
![Page 29: Richard Russellcbcl.mit.edu/people/sinha-lab.pdf · 2001. 3. 29. · - Dirty Deeds, Done Dirt Cheap - The Cambridgeport Saloon - Community-Based Policing What I like least about MIT/Cambridge:-WHAT](https://reader035.vdocuments.site/reader035/viewer/2022071516/613876730ad5d2067649459f/html5/thumbnails/29.jpg)
A statistical framework for modeling contextual influenceson object recognition
Outdoor scene.Urban environment.Street.50-100 meters.
Object priming: Car Person1
0
FurnitureTraffic signs
Context driven focus of attention
categories
Location of Pedestrian area derived by context-based processing.
Context driven scale selection
Likely pedestriansize derived viacontextual cues
…
![Page 30: Richard Russellcbcl.mit.edu/people/sinha-lab.pdf · 2001. 3. 29. · - Dirty Deeds, Done Dirt Cheap - The Cambridgeport Saloon - Community-Based Policing What I like least about MIT/Cambridge:-WHAT](https://reader035.vdocuments.site/reader035/viewer/2022071516/613876730ad5d2067649459f/html5/thumbnails/30.jpg)
What I like most about MIT/Cambridge:
What I like least about MIT/Cambridge:- this guy