Download - More on Features
![Page 1: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/1.jpg)
More on Features
Digital Visual Effects, Spring 2008Yung-Yu Chuang2008/3/25
with slides by Trevor Darrell Cordelia Schmid, David Lowe, Darya Frolova, Denis Simakov, Robert Collins, Brad Osgood, W W L Chen, and Jiwon Kim
![Page 2: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/2.jpg)
Announcements
• Project #1 is due. Voting will start from next Tuesday.
• Project #2 handout is out. Due four weeks later. A checkpoint at 4/11.
![Page 3: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/3.jpg)
Multi-Scale Oriented Patches
![Page 4: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/4.jpg)
Multi-Scale Oriented Patches• Simpler than SIFT. Designed specifically for im
age matching. [Brown, Szeliski and Winder, CVPR’2005]
• Feature detector– Multi-scale Harris corners– Orientation from blurred gradient– Geometrically invariant to rotation
• Feature descriptor– Bias/gain normalized sampling of local patch (8x8)– Photometrically invariant to affine changes in inten
sity
![Page 5: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/5.jpg)
Multi-Scale Harris corner detector
• Image stitching is mostly concerned with matching images that have the same scale, so sub-octave pyramid might not be necessary.
2s
![Page 6: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/6.jpg)
Multi-Scale Harris corner detector
smoother version of gradients
Corner detection function:
Pick local maxima of 3x3 and larger than 10
![Page 7: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/7.jpg)
Keypoint detection function
Experiments show roughly the same performance.
![Page 8: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/8.jpg)
Non-maximal suppression
• Restrict the maximal number of interest points, but also want them spatially well distributed
• Only retain maximums in a neighborhood of radius r.
• Sort them by strength, decreasing r from infinity until the number of keypoints (500) is satisfied.
![Page 9: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/9.jpg)
Non-maximal suppression
![Page 10: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/10.jpg)
Sub-pixel refinement
![Page 11: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/11.jpg)
Orientation assignment
• Orientation = blurred gradient
![Page 12: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/12.jpg)
Descriptor Vector• Rotation Invariant Frame
– Scale-space position (x, y, s) + orientation ()
![Page 13: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/13.jpg)
MOPS descriptor vector• 8x8 oriented patch sampled at 5 x scale.
See TR for details. • Sampled from with
spacing=5
8 pixels40 pixels
![Page 14: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/14.jpg)
MOPS descriptor vector• 8x8 oriented patch sampled at 5 x scale. See T
R for details. • Bias/gain normalisation: I’ = (I – )/• Wavelet transform (distance=coeff. distance)
8 pixels40 pixels
![Page 15: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/15.jpg)
Detections at multiple scales
![Page 16: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/16.jpg)
Summary
• Multi-scale Harris corner detector• Sub-pixel refinement• Orientation assignment by gradients• Blurred intensity patch as descriptor
![Page 17: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/17.jpg)
Feature matching• Exhaustive search
– for each feature in one image, look at all the other features in the other image(s)
• Hashing– compute a short descriptor from each feature vect
or, or hash longer descriptors (randomly)• Nearest neighbor techniques
– k-trees and their variants (Best Bin First)
![Page 18: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/18.jpg)
Wavelet-based hashing• Compute a short (3-vector) descriptor from an
8x8 patch using a Haar “wavelet”
• Quantize each value into 10 (overlapping) bins (103 total entries)
• [Brown, Szeliski, Winder, CVPR’2005]
![Page 19: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/19.jpg)
Nearest neighbor techniques• k-D tree
and
• Best BinFirst(BBF)
Indexing Without Invariants in 3D Object Recognition, Beis and Lowe, PAMI’99
![Page 20: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/20.jpg)
Applications of features
![Page 21: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/21.jpg)
Recognition
SIFT Features
![Page 22: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/22.jpg)
3D object recognition
![Page 23: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/23.jpg)
3D object recognition
![Page 24: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/24.jpg)
Office of the past
Video of desk Images from PDF
Track & recognize
T T+1
Internal representation
Scene Graph
Desk Desk
![Page 25: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/25.jpg)
…> 5000images
change in viewing angle
Image retrieval
![Page 26: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/26.jpg)
22 correct matches
Image retrieval
![Page 27: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/27.jpg)
…> 5000images
change in viewing angle
+ scale change
Image retrieval
![Page 28: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/28.jpg)
Robot location
![Page 29: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/29.jpg)
Robotics: Sony AiboSIFT is used for Recognizing
charging station
Communicating with visual cards
Teaching object recognition
soccer
![Page 30: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/30.jpg)
Structure from Motion
• The SFM Problem– Reconstruct scene geometry and camera
motion from two or more images
Track2D Features Estimate
3D Optimize(Bundle Adjust)
Fit Surfaces
SFM Pipeline
![Page 31: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/31.jpg)
Structure from Motion
Poor mesh Good mesh
![Page 32: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/32.jpg)
Augmented reality
![Page 33: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/33.jpg)
Automatic image stitching
![Page 34: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/34.jpg)
Automatic image stitching
![Page 35: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/35.jpg)
Automatic image stitching
![Page 36: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/36.jpg)
Automatic image stitching
![Page 37: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/37.jpg)
Automatic image stitching
![Page 38: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/38.jpg)
Project #2 Image stitching
• Assigned: 3/21• Checkpoint: 11:59pm 4/11• Due: 11:59pm 4/21• Work in pairs
![Page 39: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/39.jpg)
Reference software
• Autostitchhttp://www.cs.ubc.ca/~mbrown/autostitch/autostitch.html
• Many others are available online.
![Page 40: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/40.jpg)
Tips for taking pictures
• Common focal point• Rotate your camera to increase vertical
FOV• Tripod• Fixed exposure?
![Page 41: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/41.jpg)
Bells & whistles
• Recognizing panorama• Bundle adjustment• Handle dynamic objects• Better blending techniques
![Page 42: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/42.jpg)
Artifacts
• Take your own pictures and generate a stitched image, be creative.
• http://www.cs.washington.edu/education/courses/cse590ss/01wi/projects/project1/students/allen/index.html
![Page 43: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/43.jpg)
Submission
• You have to turn in your complete source, the executable, a html report and an artifact.
• Report page contains: description of the project, what do you learn,
algorithm, implementation details, results, bells and whistles…
• Artifacts must be made using your own program.
![Page 44: More on Features](https://reader035.vdocuments.site/reader035/viewer/2022070418/568157c2550346895dc5470c/html5/thumbnails/44.jpg)
Reference• Chris Harris, Mike Stephens,
A Combined Corner and Edge Detector, 4th Alvey Vision Conference, 1988, pp147-151.
• David G. Lowe, Distinctive Image Features from Scale-Invariant Keypoints, International Journal of Computer Vision, 60(2), 2004, pp91-110.
• Yan Ke, Rahul Sukthankar, PCA-SIFT: A More Distinctive Representation for Local Image Descriptors, CVPR 2004.
• Krystian Mikolajczyk, Cordelia Schmid, A performance evaluation of local descriptors, Submitted to PAMI, 2004.
• SIFT Keypoint Detector, David Lowe.• Matlab SIFT Tutorial, University of Toronto.