edges/curves /blobs
DESCRIPTION
Edges/curves /blobs. Image Parsing. Regions. curve groups. Sketches coding. Rectangles recognition. Objects recognition. segmentation. segmentation. Image Parsing: Regions, curves, curve groups, curve trees, edges, texture. 2008/11/5 Hueihan Jhuang. Zhuowen Tu and Song-Chun Zhu , - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/1.jpg)
Edges/curves/blobs
![Page 2: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/2.jpg)
Image Parsing
Rectanglesrecognition
Regions curve groups Objectsrecognition
Sketchescodingsegmentationsegmentation
![Page 3: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/3.jpg)
Image Parsing: Regions, curves, curve groups, curve trees, edges,
texture2008/11/5 Hueihan Jhuang
![Page 4: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/4.jpg)
Zhuowen Tu and Song-Chun Zhu ,
Image segmentation by Data-Driven Markov Chain Monte Carlo , PAMI,2002
![Page 5: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/5.jpg)
Objective
Rectanglesrecognition
Regions curve groups Objectsrecognition
Sketchescodingsegmentationsegmentation
* Boundary is not necessary rectangle
![Page 6: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/6.jpg)
Segmentation
• Non-probabilistic: k-means(Bishop, 1995), mean-shift(Comaniciu and Meer, 2002), agglomerative(Duda et al, 2000)
• Probabilistic: GMM-EM(Moon, 1996), DDMCMC (Tu and Zhu, 2002)
• Spectral factorization (Kannan et al, 2004; Meila and Shi, 2000; Ng et al, 2001; Perona and Freeman , 1998; Shi and Malik, 2000; Zass and Shashua 2005)
![Page 7: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/7.jpg)
Bayesian Formulation -notation
• I: Image• K: # regions
• Ri:the i-th region i = 1..K
• li: the type of model underlying Ri
• Θi: the parameters of the model of Ri
• W: region representation
Ex: K = 11,l1=U(a,b)l2=N(u,σ)
![Page 8: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/8.jpg)
Bayesian Formulation -prior
Number of regions Model type
Model parametersregion
θ Θ α β
Ex:
Region size
Region boundary
![Page 9: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/9.jpg)
Bayesian Formulation -likelihood
p( I | W) = N(I1;0,1) x N(I2;1,2) x …
Ex: K = 11,l1=U(a,b)l2=N(u,σ)
I1~N(0,1)
I2~N(1,2)
I3~U(0,1)
![Page 10: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/10.jpg)
Proposed Image models
Gaussian Intensity Histogram
Gabor ResponseHistogram (Zhu, 1998)
B-Spline
Uniform Clutter Texture Shading
Examples:
![Page 11: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/11.jpg)
Objective in Bayesian Formulation –Maximize posterior
W* = arg max p( W | I) α arg max p( I | W) p( W )
Given a image, find the most likely partition
![Page 12: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/12.jpg)
Solution spaceA possible solution
clutter
texture
uniform
clutter
K regions x ways to partition the image x 4 possible models for each of the k regions
Enumerate all possible solutions Monte Carlo Markov Chain
![Page 13: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/13.jpg)
Monte Carlo Method
• A class of algorithms that relies on repeat sampling to get the results
• Ex: • f(x)g(x)h(x) (known but complicated probability density function)
• Expectation? Integration ? maximum ? • sol: Draw a lot of samples->mean/sum/max
![Page 14: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/14.jpg)
Markov Chain Monte Carlo
• MCMC is a technique for generating fair samples from a probability in high-dimensional space.
Zhu 2005 ICCV tutoiral
![Page 15: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/15.jpg)
Design transitions (probability) to explore the solution space
The papers designs all the transition probabilities
Boundary diffusion
model adaptation
merge
splitswitching model
![Page 16: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/16.jpg)
Results
Region segmentation
Proposed segmentation
Assume images consist of 2D compact regions, results become cluttered in the presence of 1D curve objects
image
Zhu 2005 ICCV tutoiral
![Page 17: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/17.jpg)
images
Segmentation using MCMC
Curve groups
regions
Results
![Page 18: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/18.jpg)
Zhuowen Tu and Song-Chun Zhu,
Parsing Images into Regions, Curves, and Curve Groups,
IJCV, 2005.
![Page 19: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/19.jpg)
Tu and Zhu, 2005
Define new components-curve family
![Page 20: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/20.jpg)
Bayesian formulation
C: curvePg: parallel curvesT: curve trees
p(I | Wr) -> as (Tu and Zhu, 2002), Gaussian/ historam/B-sline modelsp(I | Wc) -> B spline model
* Minor difference from previous setting
![Page 21: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/21.jpg)
Prior of curve / curve group/curve trees
# pixels in curve
Length of curve
Width of the curve
exp{ }
Attributes for each curve:Center, orientation, Length, width
p(pg)~ exp{ -difference of attributes }
Compatibility of curves:Width, orientation continuity,end point gap
p(t)~ exp{ -incompatibility }
Ex
![Page 22: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/22.jpg)
Solution space is largerregion
Free curves
Curve groups
Curve trees
MCMC again
![Page 23: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/23.jpg)
Less cluttered, but still doesn’t provide a good start for object recognition
Tu et al, 2005
Results
![Page 24: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/24.jpg)
Results
(Tu et al, ICCV, 2003)
![Page 25: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/25.jpg)
Objective
Rectanglesrecognition
Regions curve groups Objectsrecognition
Sketchescodingsegmentationsegmentation
![Page 26: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/26.jpg)
Image may be divided into two components
structure (Texton, token, edges)
texture (Symbolic)
(Julesz, 1962,1981; Marr, 1982)
![Page 27: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/27.jpg)
Cheng-en Guo, Song-Chun Zhu, and Ying Nian WuPrimal sketch: Integrating Texture and Structure ,PAMI, 2005
![Page 28: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/28.jpg)
Example
= +
+
+ +
+Likelihood? Gaussian? Histogram?
Likelihood? Histogram as (Tu and Zhu, 2005)
Edges/ blobs
![Page 29: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/29.jpg)
Proposed Image primitives
![Page 30: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/30.jpg)
Bayesian Formulation
priors
Image primitives
Likelihood of non-sketchableLikelihood of sketchable
K is the number of primitives, not # regions : this is not segmentation
![Page 31: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/31.jpg)
Sketch pursuit algorithm
Blob/edge detectorLOG, DG, D2G
Sequentially adding edges that have gain more than some threshold What threshold?
![Page 32: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/32.jpg)
Adding one stroke into sketchable group
gain
![Page 33: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/33.jpg)
Sketch pursuit algorithmFor each local region, check if using other image primitives increaseThe posterior probability
![Page 34: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/34.jpg)
Results
![Page 35: Edges/curves /blobs](https://reader031.vdocuments.site/reader031/viewer/2022020208/568137df550346895d9f827a/html5/thumbnails/35.jpg)
More Results