lecture 5: image interpolation and features cs4670: computer vision kavita bala
TRANSCRIPT
![Page 1: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/1.jpg)
Lecture 5: Image Interpolation and Features
CS4670: Computer VisionKavita Bala
![Page 2: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/2.jpg)
Announcements
• Tell us if conflicts on PA 1 grading
• iclickerGo– Tell us if you don’t have it
![Page 3: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/3.jpg)
Upsampling
• This image is too small for this screen:• How can we make it 10 times as big?
• Simplest approach: repeat each row and column 10 times
![Page 4: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/4.jpg)
Image interpolation
Original image: x 10
![Page 5: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/5.jpg)
Image interpolation
Also used for resampling
![Page 6: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/6.jpg)
Upsampling
• This image is too small for this screen:• How can we make it 10 times as big?• Simplest approach:
repeat each row and column 10 times
• (“Nearest neighbor interpolation”)
![Page 7: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/7.jpg)
Image interpolation
Recall how a digital image is formed
• It is a discrete point-sampling of a continuous function• If we could somehow reconstruct the original function, any new
image could be generated, at any resolution and scale
1 2 3 4 5
Adapted from: S. Seitz
d = 1 in this example
![Page 8: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/8.jpg)
Image interpolation
1 2 3 4 5
d = 1 in this example
Recall how a digital image is formed
• It is a discrete point-sampling of a continuous function• If we could somehow reconstruct the original function, any new
image could be generated, at any resolution and scale
Adapted from: S. Seitz
![Page 9: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/9.jpg)
Image interpolation
1 2 3 4 52.5
1
• Convert to a continuous function:
• Reconstruct by convolution with a reconstruction filter, h
• What if we don’t know ?• Guess an approximation:• Can be done in a principled way: filtering
d = 1 in this example
Adapted from: S. Seitz
![Page 10: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/10.jpg)
Image interpolation
Nearest-neighbor interpolation
Linear interpolation
Source: B. Curless
![Page 11: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/11.jpg)
Reconstruction filters• What does the 2D version of this hat function look like?
Better filters give better resampled images• Bicubic is common choice
performs linear interpolation
(tent function) performs bilinear interpolation
Cubic reconstruction filter
![Page 12: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/12.jpg)
Image interpolation
“Ideal” reconstruction
Nearest-neighbor interpolation
Linear interpolation
Gaussian reconstruction
Source: B. Curless
![Page 13: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/13.jpg)
Image interpolation
Nearest-neighbor interpolation Bilinear interpolation Bicubic interpolation
Original image: x 10
![Page 14: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/14.jpg)
Image interpolation
Also used for resampling
![Page 15: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/15.jpg)
Feature detection and matching
![Page 16: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/16.jpg)
Reading
• Szeliski: 4.1
![Page 17: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/17.jpg)
Motivation: Automatic panoramas
Credit: Matt Brown
![Page 18: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/18.jpg)
http://research.microsoft.com/en-us/um/redmond/groups/ivm/HDView/HDGigapixel.htmHD View
Also see GigaPan: http://gigapan.org/
Motivation: Automatic panoramas
![Page 19: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/19.jpg)
Why extract features?
• Motivation: panorama stitching– We have two images – how do we combine them?
![Page 20: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/20.jpg)
Why extract features?
• Motivation: panorama stitching– We have two images – how do we combine them?
Step 1: extract featuresStep 2: match features
![Page 21: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/21.jpg)
Why extract features?
• Motivation: panorama stitching– We have two images – how do we combine them?
Step 1: extract featuresStep 2: match features
Step 3: align images
![Page 22: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/22.jpg)
Example: estimating “fundamental matrix” that corresponds two views
Slide from Silvio Savarese
![Page 23: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/23.jpg)
Example: structure from motion
![Page 24: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/24.jpg)
Applications • Feature points are used for:
– Image alignment – 3D reconstruction– Motion tracking– Robot navigation– Indexing and database retrieval– Object recognition
![Page 25: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/25.jpg)
Matching can be challenging
Fei-Fei Li
![Page 26: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/26.jpg)
Image matching
by Diva Sian
by swashford
![Page 27: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/27.jpg)
Harder case
by Diva Sian by scgbt
![Page 28: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/28.jpg)
Harder still?
![Page 29: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/29.jpg)
NASA Mars Rover imageswith SIFT feature matches
Answer below (look for tiny colored squares…)
![Page 30: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/30.jpg)
Approach
Feature detection: find it
Feature descriptor: represent it
Feature matching: match it
Feature tracking: track it, when motion
![Page 31: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/31.jpg)
Features• Global vs. local representations• Local: describe and match local regions• Robust to
– Occlusions– Articulation– Intra-category variation
Fei-Fei Li
![Page 32: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/32.jpg)
Advantages of local features
Locality – features are local, so robust to occlusion and clutter
Quantity– hundreds or thousands in a single image
Distinctiveness: – can differentiate a large database of objects
Efficiency– real-time performance achievable
![Page 33: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/33.jpg)
Invariant local featuresFind features that are invariant to transformations
– geometric invariance: translation, rotation, scale– photometric invariance: brightness, exposure, …
Feature Descriptors
![Page 34: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/34.jpg)
Overview
K. Grauman, B. Leibe
Af Bf
B1
B2
B3A1
A2 A3
Tffd BA ),(
1. Find a set of distinctive features
3. Extract and normalize the region content
2. Define a region around each feature
4. Compute a local descriptor from the normalized region
5. Match local descriptors
![Page 35: Lecture 5: Image Interpolation and Features CS4670: Computer Vision Kavita Bala](https://reader035.vdocuments.site/reader035/viewer/2022062500/5697c02d1a28abf838cd9ae8/html5/thumbnails/35.jpg)
Goals for Features
Detect points that are repeatable and distinctive