recognition of 3d objects or, 3d recognition of objects alec rivers
Post on 23-Dec-2015
222 Views
Preview:
TRANSCRIPT
Overview
• 3D object recognition was dead, now it’s coming back– These papers are within the last 2 years
• Doesn’t really work yet, but it’s just a beginning
Papers• The Layout Consistent Random Field
for Recognizing and Segmenting Partially Occluded Objects– CVPR 2006
• 3D LayoutCRF for Multi-View Object Class Recognition and Segmentation– CVPR 2007
• 3D Generic Object Categorization, Localization and Pose Estimation– ICCV 2007
The Layout Consistent Random Field forRecognizing and Segmenting Partially Occluded
Objects
John WinnMicrosoft Research
Cambridge
Jamie ShottonUniversity ofCambridge
Introduction
• Needed to understand next paper– It’s 2D
• What does it try to solve?– Recognize one class of object at one pose and one
scale, but with occlusions
• Does it work?– Yes, really well, especially given occlusions
Introduction
• What is interesting about it?– Segments objects– Interesting methods
• No sliding windows
– Multiple instances for free
Overview
• Instead of sparse parts at features, use a densely covering part grid
[Fischler & Elschlager 73]
[Winn & Shotton 06]
Layout Consistency
• Defined pairwise between two pixels:PI, PJ => Bool
• Means pixels I, J could be part of one instance• Toy example:
Object: 1,2,3,4,5Image:
2,3,4,5,0,0,1,2,3,4,5,2,3,4,5,0,0
Layout Consistency
• Defined pairwise between two pixels:PI, PJ => Bool
• Means pixels I, J could be part of one instance• Toy example:
Object: 1,2,3,4,5Image:
2,3,4,5,0,0,1,2,3,4,5,2,3,4,5,0,0
occlusion
instance 2 instance 3instance 1
Layout Consistency
• In 2D, consistent IFF their relative assignments could exist in a deformed regular grid
• Formally:
Layout Consistency
3. Find consistent regions; create instances
Possible due to layout inconsistency at occluding borders
Implementation Details
• Trained on manually segmented data• Crux of algorithm is conditional distribution
– Like a probability for each possibility, or a score
• Algorithm is just finding maximum
Part Appearance
• Each pixel prefers parts that match surrounding image data
• Randomized decision trees– Multiple trees, each trained on a subset of the
data– Node is maximal-information-gain binary test on
two nearby pixels’ intensities– Leaf of node is histogram of part possibilities– Actual preference is average over all trees
Deformed Training Part Labelings
• Fits parts tighter1. Label by grid2. Learn from data3. Apply to data4. Set guesses as
truth5. Relearn
Part Layout• Preference for layout consistency plus additional
pairwise costs:
• Helps remove noise• Align edges along image edges
Part Layout
• Return to toy exampleJust appearance:
1,2,0,4,5,0,0,1,2,3,3,4,0,0,1,0With layout costs:
1,2,3,4,5,0,0,1,2,3,3,4,0,0,0,0instance 2instance 1
Instance Layout
• Apply weak force trying to keep parts at sane positions relative to instance data (centroid, L/R flip)
• Toy example: 0,1,1,1,1,1,2,3,4,5 is bad!
Implementation
• Theoretically, finding global maximum of
• This is “MAP” estimation– MAP = Maximum A Posteriori
• In reality, using tricks to find a local maximum– α-expansion, annealed expansion move
Approximating MAP Estimation
• Global maximum is intractable• α-expansion
– Start with given configuration– For a given new label, ask each pixel: do you want to
switch?– Can be solved efficiently with graph cuts
• Repeat over all part labels• Annealed expansion move
– Relabel grid, but offset to avoid local maxima
Thoughts
• Bottom-up system is great– No sliding windows– Multiple instances for free
• Information about segment boundaries: occlusion vs. completion– Reason about complete segment boundaries?
John Winn
3D LayoutCRF for Multi-View Object Class Recognition and Segmentation
Derek HoiemCarnegie Mellon
University
Carsten Rother Microsoft Research
Cambridge
Introduction
• What does it try to solve?– Extend LayoutCRF to be pose and scale invariant
• Does it work?– Improvements to LayoutCRF work;
3D information does little
• What is interesting about it?– One method for combining 2D methods with a 3D
framework– The improvements to 2D are good
Instance Cost
• Eliminates false positives– LayoutCRF: object-background cost
• Explain multiple groups with one instance
Instance appearance
• Learn color distribution for each instance• Separate groups of pixels: definitely object,
definitely background• Use these to learn colors• Apply cost to non-standard-color pixels
This would fail…
Implementation Details
• Parts are learned separately for each 45o viewing range, and for different scales
• Instance layout is also discretized by viewpoint
Results – Comparison to LCRF
• A little better(+ 8% recall)
• BUT they actually turn off 3D information for this comparison
• Better segmentation
Results – PASCAL 2006
• 61% precision-recall– Previous best: 45%– But, reduced test set
• Without 3D: -5%• Without color: -5%
Thoughts
• Color, instance costs very nice• Shoehorns LCRF into 3D without much success• LCRF is already somewhat viewpoint-
invariant: segments can stretch
3D Generic Object Categorization, Localization and Pose Estimation
Silvio SavareseUniversity of Illinois at
Urbana-Champaign
Fei-Fei LiPrinceton University
Introduction
• What does it try to solve?– Multiclass pose-invariant, scale-invariant object
recognition
• Does it work?– Not well. But it may be due to implementation
• Why is it interesting?– Attempt learn actual 3D structure of an object– Interesting data structure for 3D info
Overview – Data Structure• Decompose object into large parts; find “canonical view”• Relate parts by mutual appearance
Related Work – Aspect Graphs
• Represent stable views rather than parts
Image [Khoh & Kovesi, 99]
Aspect graph of a cube:
Implementation – Links
• Link from canonical PI to PJ consists of
• Matrix defines transformation to observe PJ when PI is viewed canonically
• AIJ is skew, tIJ is translation
Overview
• Learn data structure from images (unsupervised)
• Apply to new image by recognizing parts and selecting model that best accounts for their appearances
Implementation – Learning Parts
• Tricky implementation!• Part = collection of SIFT featuresFor each pair of images of the same
instance:1. Find set M of shared SIFT features2. RANSAC M to find a group of pairs
that transform together3. Group close-together parts of M
into candidate parts
Background: What is RANSAC?
• Finds subset of data that is accounted for by some model; ignores outliers
1. Guess points2. Fit model3. Select matching points4. Calculate errorRepeat!
RANSAC
• In our case: find points for which a homographic transformation of the points in image I yield the points in image J
Implementation – Canonical Views• Goal: front-facing view of part• Construct directed graph
– Direction means “more front-facing”• Traverse to find canonical view
• How to go from pairwise-defined to graph?
Recognizing a New Image
1. Extract SIFT features2. Use scanning windows to get 5 best canonical
part matches3. For every pair of found parts, for each model,
score how well the model accounts for their relative appearances
4. Select the model with the best score
Thoughts
• Low performance may make it useless as a system, but the data structure is very nice
• Implementation has a lot of tricky parts– Doesn’t seem to select great canonical parts– I wonder if there’s a simpler way– Are SIFT features the right choice?
Extremely Confusing Figure
• “Each dashed box indicates a particular view. A subset of the canonical parts is presented for each view. Part relationships are denoted by arrows.”
top related