structure of vision problems - alex.smola.org · machine learning & bayes machine learning...
TRANSCRIPT
![Page 1: Structure of Vision Problems - alex.smola.org · Machine Learning & Bayes Machine Learning seems to concentrate on discrimination problems. A whole range of other vision problems](https://reader034.vdocuments.site/reader034/viewer/2022052615/607f730da61ebf182a489864/html5/thumbnails/1.jpg)
Structure of Vision Problems
Alan Yuille (UCLA).
![Page 2: Structure of Vision Problems - alex.smola.org · Machine Learning & Bayes Machine Learning seems to concentrate on discrimination problems. A whole range of other vision problems](https://reader034.vdocuments.site/reader034/viewer/2022052615/607f730da61ebf182a489864/html5/thumbnails/2.jpg)
Machine Learning
Theory of Machine Learning is beautifuland deep.
But, how useful is it for vision?Vision rarely has an obvious vector
space structure.
![Page 3: Structure of Vision Problems - alex.smola.org · Machine Learning & Bayes Machine Learning seems to concentrate on discrimination problems. A whole range of other vision problems](https://reader034.vdocuments.site/reader034/viewer/2022052615/607f730da61ebf182a489864/html5/thumbnails/3.jpg)
Image Formation
Images formation is complicated.E.g. the image of a face depends on
viewpoint, lighting, facial expression.
![Page 4: Structure of Vision Problems - alex.smola.org · Machine Learning & Bayes Machine Learning seems to concentrate on discrimination problems. A whole range of other vision problems](https://reader034.vdocuments.site/reader034/viewer/2022052615/607f730da61ebf182a489864/html5/thumbnails/4.jpg)
Image Formation.
Parable of the Theatre, the Carpenter, the Painter, and the Lightman. (Adelson and Pentland).
How many ways can you construct ascene so that the image looks the samewhen seen from the Royal Box?
![Page 5: Structure of Vision Problems - alex.smola.org · Machine Learning & Bayes Machine Learning seems to concentrate on discrimination problems. A whole range of other vision problems](https://reader034.vdocuments.site/reader034/viewer/2022052615/607f730da61ebf182a489864/html5/thumbnails/5.jpg)
Nonlinear Transformations
Mumford suggested that images involvebasic nonlinear transformations.
(I) Image warping: x W(x) (e.g. change of viewpoint, expression, etc.).(II) Occlusion: foreground objects
occlude background objects.(III) Shadows, Multi-Reflectance.
→
![Page 6: Structure of Vision Problems - alex.smola.org · Machine Learning & Bayes Machine Learning seems to concentrate on discrimination problems. A whole range of other vision problems](https://reader034.vdocuments.site/reader034/viewer/2022052615/607f730da61ebf182a489864/html5/thumbnails/6.jpg)
Complexity of Images
Easy, Medium, and Hard Images.
![Page 7: Structure of Vision Problems - alex.smola.org · Machine Learning & Bayes Machine Learning seems to concentrate on discrimination problems. A whole range of other vision problems](https://reader034.vdocuments.site/reader034/viewer/2022052615/607f730da61ebf182a489864/html5/thumbnails/7.jpg)
Discrimination or Probabilities
Statistical Edge Detection (Konishi,Yuille, Coughlan, Zhu).Use segmented image database to
learn probability distributions ofP(f|on) and P(f|off), where “f” is filterresponse.
![Page 8: Structure of Vision Problems - alex.smola.org · Machine Learning & Bayes Machine Learning seems to concentrate on discrimination problems. A whole range of other vision problems](https://reader034.vdocuments.site/reader034/viewer/2022052615/607f730da61ebf182a489864/html5/thumbnails/8.jpg)
P-on and P-offLet f(I(x)) = |grad I(x)|Calculate empirical
histograms P(f=y|ON)and P(f=y|OFF).
P(f=y|ON)/P(f=yOFF)is monotonic in y.
So loglikelihood test is threshold on |grad (I(x)|.
![Page 9: Structure of Vision Problems - alex.smola.org · Machine Learning & Bayes Machine Learning seems to concentrate on discrimination problems. A whole range of other vision problems](https://reader034.vdocuments.site/reader034/viewer/2022052615/607f730da61ebf182a489864/html5/thumbnails/9.jpg)
P-on and P-off
P-on and P-off become more powerfulwhen combining multiple edge cues (byjoint distributions).
Results as good, or better than, standard edge detectors when evaluated on images with groundtruth.
![Page 10: Structure of Vision Problems - alex.smola.org · Machine Learning & Bayes Machine Learning seems to concentrate on discrimination problems. A whole range of other vision problems](https://reader034.vdocuments.site/reader034/viewer/2022052615/607f730da61ebf182a489864/html5/thumbnails/10.jpg)
P-on and P-offWhy not do discrimination and avoid
learning the distributions? (Malik et al).Learning the distributions and using log-
likelihood is optimal provided there issufficient data.
But “Don’t solve a harder problem than youhave to”.
![Page 11: Structure of Vision Problems - alex.smola.org · Machine Learning & Bayes Machine Learning seems to concentrate on discrimination problems. A whole range of other vision problems](https://reader034.vdocuments.site/reader034/viewer/2022052615/607f730da61ebf182a489864/html5/thumbnails/11.jpg)
Probabilities or Discrimination
Two Reasons for Probabilities:(I) They can be used for other problems
such as detecting contours by combining local edge cues.
(II) They can be used to synthesize edges as a “reality check”.
![Page 12: Structure of Vision Problems - alex.smola.org · Machine Learning & Bayes Machine Learning seems to concentrate on discrimination problems. A whole range of other vision problems](https://reader034.vdocuments.site/reader034/viewer/2022052615/607f730da61ebf182a489864/html5/thumbnails/12.jpg)
Combining Local Edge Cues
Detect contours by edge cues with shape priors P_g (Geman & Jedynak).
.ondistributiuniformis(.)
,)()(
log1
)()(log1}){},({
1
1
UtUtP
N
yPyP
Nytr
N
ii
ig
N
iioff
ionii
∑
∑
=
=
+
=
![Page 13: Structure of Vision Problems - alex.smola.org · Machine Learning & Bayes Machine Learning seems to concentrate on discrimination problems. A whole range of other vision problems](https://reader034.vdocuments.site/reader034/viewer/2022052615/607f730da61ebf182a489864/html5/thumbnails/13.jpg)
Manhattan World
Coughlan and Yuille use P-on, P-off toestimate scene orientation wrt viewer.
![Page 14: Structure of Vision Problems - alex.smola.org · Machine Learning & Bayes Machine Learning seems to concentrate on discrimination problems. A whole range of other vision problems](https://reader034.vdocuments.site/reader034/viewer/2022052615/607f730da61ebf182a489864/html5/thumbnails/14.jpg)
Synthesis as Reality Check
Synthesis of Images using P-on, P-offdistributions (Coughlan & Yuille).
![Page 15: Structure of Vision Problems - alex.smola.org · Machine Learning & Bayes Machine Learning seems to concentrate on discrimination problems. A whole range of other vision problems](https://reader034.vdocuments.site/reader034/viewer/2022052615/607f730da61ebf182a489864/html5/thumbnails/15.jpg)
Machine Learning Success
Fixed geometry, lighting, viewpoint.AdaBoost Learning: Viola and Jones.
![Page 16: Structure of Vision Problems - alex.smola.org · Machine Learning & Bayes Machine Learning seems to concentrate on discrimination problems. A whole range of other vision problems](https://reader034.vdocuments.site/reader034/viewer/2022052615/607f730da61ebf182a489864/html5/thumbnails/16.jpg)
Machine Vision Success
Other examples:Classification (Le Cun et al, Scholkopf
et al, Caputo et al).Demonstrate the power of statistics –
rather than the power of machine learning?
![Page 17: Structure of Vision Problems - alex.smola.org · Machine Learning & Bayes Machine Learning seems to concentrate on discrimination problems. A whole range of other vision problems](https://reader034.vdocuments.site/reader034/viewer/2022052615/607f730da61ebf182a489864/html5/thumbnails/17.jpg)
Bayesian Pattern Theory.
This approach seeks to model thedifferent types of image patterns.
Vision as statistical inference – inversecomputer graphics.
Analysis by Synthesis (Bayes).Computationally expensive?
![Page 18: Structure of Vision Problems - alex.smola.org · Machine Learning & Bayes Machine Learning seems to concentrate on discrimination problems. A whole range of other vision problems](https://reader034.vdocuments.site/reader034/viewer/2022052615/607f730da61ebf182a489864/html5/thumbnails/18.jpg)
Example: Image Segmentation
Standard computer vision task.Pattern Theory formulation (Zhu,Tu):
Decompose images into their underlyingpatterns.
Requires a set of probability modelswhich can describe image patterns.Learnt from data.
![Page 19: Structure of Vision Problems - alex.smola.org · Machine Learning & Bayes Machine Learning seems to concentrate on discrimination problems. A whole range of other vision problems](https://reader034.vdocuments.site/reader034/viewer/2022052615/607f730da61ebf182a489864/html5/thumbnails/19.jpg)
Image Pattern Models
Images (top) and Synthesized (bottom).
![Page 20: Structure of Vision Problems - alex.smola.org · Machine Learning & Bayes Machine Learning seems to concentrate on discrimination problems. A whole range of other vision problems](https://reader034.vdocuments.site/reader034/viewer/2022052615/607f730da61ebf182a489864/html5/thumbnails/20.jpg)
Image Parsing: Zhu & Tu
![Page 21: Structure of Vision Problems - alex.smola.org · Machine Learning & Bayes Machine Learning seems to concentrate on discrimination problems. A whole range of other vision problems](https://reader034.vdocuments.site/reader034/viewer/2022052615/607f730da61ebf182a489864/html5/thumbnails/21.jpg)
Image Parsing: Zhu & Tu.
Bayesian Formulation: model image asbeing composed of multiple regions.
Boundaries of regions obey (probabilistic) constraints (e.g. smoothness)
Intensity properties within regions aredescribed by a set of models with unknownparameters (to be estimated).
![Page 22: Structure of Vision Problems - alex.smola.org · Machine Learning & Bayes Machine Learning seems to concentrate on discrimination problems. A whole range of other vision problems](https://reader034.vdocuments.site/reader034/viewer/2022052615/607f730da61ebf182a489864/html5/thumbnails/22.jpg)
Image Parsing Results:
Input, Segmentation, and Synthesis.
![Page 23: Structure of Vision Problems - alex.smola.org · Machine Learning & Bayes Machine Learning seems to concentrate on discrimination problems. A whole range of other vision problems](https://reader034.vdocuments.site/reader034/viewer/2022052615/607f730da61ebf182a489864/html5/thumbnails/23.jpg)
Regions, Curves, Occlusions.
![Page 24: Structure of Vision Problems - alex.smola.org · Machine Learning & Bayes Machine Learning seems to concentrate on discrimination problems. A whole range of other vision problems](https://reader034.vdocuments.site/reader034/viewer/2022052615/607f730da61ebf182a489864/html5/thumbnails/24.jpg)
Removing Foreground.
“Denoising” images by removing foreground clutter.
![Page 25: Structure of Vision Problems - alex.smola.org · Machine Learning & Bayes Machine Learning seems to concentrate on discrimination problems. A whole range of other vision problems](https://reader034.vdocuments.site/reader034/viewer/2022052615/607f730da61ebf182a489864/html5/thumbnails/25.jpg)
Image Parsing Solution Space
No. regions, Types of regions, Properties of regions.
![Page 26: Structure of Vision Problems - alex.smola.org · Machine Learning & Bayes Machine Learning seems to concentrate on discrimination problems. A whole range of other vision problems](https://reader034.vdocuments.site/reader034/viewer/2022052615/607f730da61ebf182a489864/html5/thumbnails/26.jpg)
Machine Learning & Bayes.
Zhu-Tu’s algorithm is called DDMCMCData-Driven Markov Chain Monte Carlo.
Discrimination methods (e.g. AdaBoost)can be used as proposal probabilities, which can be verified by Bayesian pattern models.
![Page 27: Structure of Vision Problems - alex.smola.org · Machine Learning & Bayes Machine Learning seems to concentrate on discrimination problems. A whole range of other vision problems](https://reader034.vdocuments.site/reader034/viewer/2022052615/607f730da61ebf182a489864/html5/thumbnails/27.jpg)
Machine Learning & BayesMachine Learning seems to concentrate
on discrimination problems.A whole range of other vision problems –
image segmentation, image matching, viewpoint estimation, etc.
Probability models for image patterns arelearnable. These models give reality checksby synthesis.
![Page 28: Structure of Vision Problems - alex.smola.org · Machine Learning & Bayes Machine Learning seems to concentrate on discrimination problems. A whole range of other vision problems](https://reader034.vdocuments.site/reader034/viewer/2022052615/607f730da61ebf182a489864/html5/thumbnails/28.jpg)
Machine Learning & Bayes
Machine Learning’s big advantage overBayes is speed (when applicable).
AdaBoost may be particularly usefulfor combining local cues.
Machine Learning for computationalsearch to enable Bayesian estimation?