![Page 1: 1 How do ideas from perceptual organization relate to natural scenes?](https://reader031.vdocuments.site/reader031/viewer/2022032522/56649d6a5503460f94a48384/html5/thumbnails/1.jpg)
1
How do ideas from perceptual
organization relate to natural
scenes?
![Page 2: 1 How do ideas from perceptual organization relate to natural scenes?](https://reader031.vdocuments.site/reader031/viewer/2022032522/56649d6a5503460f94a48384/html5/thumbnails/2.jpg)
2
Brunswik & Kamiya 1953
• Thesis: Gestalt rules reflect the structure of the natural world
• Attempted to validate the grouping rule of proximity of similars
• Brunswik was ahead of his time… we now have the tools. Egon Brunswik (1903-1955)
![Page 3: 1 How do ideas from perceptual organization relate to natural scenes?](https://reader031.vdocuments.site/reader031/viewer/2022032522/56649d6a5503460f94a48384/html5/thumbnails/3.jpg)
3
• Can we define these cues for real images?• Are these cues “ecologically valid”?• How informative are different cues?
Grouping Figure/Ground
Ecological Statistics of Perceptual Organization
![Page 4: 1 How do ideas from perceptual organization relate to natural scenes?](https://reader031.vdocuments.site/reader031/viewer/2022032522/56649d6a5503460f94a48384/html5/thumbnails/4.jpg)
4
• Task: detect generic pattern or group
• Signal: class of patterns, known null hypothesis
• Cues: optimal test is usually obvious
• Result: mathematically precise characterization of when detection is possible
• Task: capture “useful” information about the scene
• Signal: natural image statistics, clutter
• Cues: something computable from real pixels
• Result: empirical statistics about relative power of different cues
![Page 5: 1 How do ideas from perceptual organization relate to natural scenes?](https://reader031.vdocuments.site/reader031/viewer/2022032522/56649d6a5503460f94a48384/html5/thumbnails/5.jpg)
5
Berkeley Segmentation DataSet [BSDS]
![Page 6: 1 How do ideas from perceptual organization relate to natural scenes?](https://reader031.vdocuments.site/reader031/viewer/2022032522/56649d6a5503460f94a48384/html5/thumbnails/6.jpg)
6
Cues:a) distance [proximity]b) region cues [similarity]c) boundary cues [connectedness, closure, convexity]
What image measurements allow us to gauge the probability that pixels i and j belong to the same group?
![Page 7: 1 How do ideas from perceptual organization relate to natural scenes?](https://reader031.vdocuments.site/reader031/viewer/2022032522/56649d6a5503460f94a48384/html5/thumbnails/7.jpg)
7
Learning Pairwise AffinitiesSij – indicator variable as to whether pixels i and j were marked as belonging to the same group by human
subjects.
Wij – our estimate of the likelihood that pixel i and j belong to the same group conditioned on the image measurements.
• Use the ground truth given by human segmentations to calibrate cues.• Learn “statistically optimal” cue combination in a supervised learning framework• Ecological Statistics: Measure the relative power of different cues for natural scenes
![Page 8: 1 How do ideas from perceptual organization relate to natural scenes?](https://reader031.vdocuments.site/reader031/viewer/2022032522/56649d6a5503460f94a48384/html5/thumbnails/8.jpg)
8
Color
a*b*
Brightness
L*
TextureOriginal Image
Wij
Distance
ED
2
Boundary Processing
Textons
A
B
C
A
B
C
2
Region Processing
![Page 9: 1 How do ideas from perceptual organization relate to natural scenes?](https://reader031.vdocuments.site/reader031/viewer/2022032522/56649d6a5503460f94a48384/html5/thumbnails/9.jpg)
9
Evaluation Measures
1. Precision-Recall of same-segment pairs– Precision is P(Sij=1 | Wij > t)
– Recall is P(Wij > t | Sij = 1)
2. Mutual Information between W and S
Groundtruth SijEstimate Wij
∫ p(s,w) log [p(s)p(w) / p(s,w)]
![Page 10: 1 How do ideas from perceptual organization relate to natural scenes?](https://reader031.vdocuments.site/reader031/viewer/2022032522/56649d6a5503460f94a48384/html5/thumbnails/10.jpg)
10
Individual Features
Patches Gradients
![Page 11: 1 How do ideas from perceptual organization relate to natural scenes?](https://reader031.vdocuments.site/reader031/viewer/2022032522/56649d6a5503460f94a48384/html5/thumbnails/11.jpg)
11
Affinity Model vs. Human Segmentation
![Page 12: 1 How do ideas from perceptual organization relate to natural scenes?](https://reader031.vdocuments.site/reader031/viewer/2022032522/56649d6a5503460f94a48384/html5/thumbnails/12.jpg)
12
Findings
• Both Edges and Patches provide useful “independent” information.
• Texture gradients can be quite powerful
• Color patches better than gradients
• Brightness gradients better than patches.
• Proximity is a result, not a cause of grouping
![Page 13: 1 How do ideas from perceptual organization relate to natural scenes?](https://reader031.vdocuments.site/reader031/viewer/2022032522/56649d6a5503460f94a48384/html5/thumbnails/13.jpg)
13
Figure-Ground Labeling
- start with 200 segmented images of natural scenes- boundaries labeled by at least 2 different human subjects- subjects agree on 88% of contours labeled
![Page 14: 1 How do ideas from perceptual organization relate to natural scenes?](https://reader031.vdocuments.site/reader031/viewer/2022032522/56649d6a5503460f94a48384/html5/thumbnails/14.jpg)
14
Local Cues for Figure/Ground
• Assume we have a perfect segmentation
• Can we predict which region a contour belongs to based on it’s local shape?– Size/Surroundedness– Convexity– Lower Region
![Page 15: 1 How do ideas from perceptual organization relate to natural scenes?](https://reader031.vdocuments.site/reader031/viewer/2022032522/56649d6a5503460f94a48384/html5/thumbnails/15.jpg)
15Size(p) = log(AreaF / AreaG)
Size and Surroundedness [Rubin 1921]
GFp
![Page 16: 1 How do ideas from perceptual organization relate to natural scenes?](https://reader031.vdocuments.site/reader031/viewer/2022032522/56649d6a5503460f94a48384/html5/thumbnails/16.jpg)
16
Convexity(p) = log(ConvF / ConvG)
ConvG = percentage of straight lines that lie completely within region G
pG F
Convexity [Metzger 1953, Kanizsa and Gerbino 1976]
![Page 17: 1 How do ideas from perceptual organization relate to natural scenes?](https://reader031.vdocuments.site/reader031/viewer/2022032522/56649d6a5503460f94a48384/html5/thumbnails/17.jpg)
17
LowerRegion(p) = θG
Lower Region[Vecera, Vogel & Woodman 2002]
θ
p
center of mass
![Page 18: 1 How do ideas from perceptual organization relate to natural scenes?](https://reader031.vdocuments.site/reader031/viewer/2022032522/56649d6a5503460f94a48384/html5/thumbnails/18.jpg)
18
Figural regions tend to be convex
![Page 19: 1 How do ideas from perceptual organization relate to natural scenes?](https://reader031.vdocuments.site/reader031/viewer/2022032522/56649d6a5503460f94a48384/html5/thumbnails/19.jpg)
19
Figural regions tend to lie below ground regions
![Page 20: 1 How do ideas from perceptual organization relate to natural scenes?](https://reader031.vdocuments.site/reader031/viewer/2022032522/56649d6a5503460f94a48384/html5/thumbnails/20.jpg)
20
Size
LowerRegion
Convexity
![Page 21: 1 How do ideas from perceptual organization relate to natural scenes?](https://reader031.vdocuments.site/reader031/viewer/2022032522/56649d6a5503460f94a48384/html5/thumbnails/21.jpg)
21
Power of cue depends on support of the analysis window.
![Page 22: 1 How do ideas from perceptual organization relate to natural scenes?](https://reader031.vdocuments.site/reader031/viewer/2022032522/56649d6a5503460f94a48384/html5/thumbnails/22.jpg)
22
Power of cue depends on support of the analysis window.
![Page 23: 1 How do ideas from perceptual organization relate to natural scenes?](https://reader031.vdocuments.site/reader031/viewer/2022032522/56649d6a5503460f94a48384/html5/thumbnails/23.jpg)
23
“Upper Bounding” Local Performance
• Present human subjects with local shapes, seen through an aperture.
![Page 24: 1 How do ideas from perceptual organization relate to natural scenes?](https://reader031.vdocuments.site/reader031/viewer/2022032522/56649d6a5503460f94a48384/html5/thumbnails/24.jpg)
24
Human Performance on Local Figure-Ground
![Page 25: 1 How do ideas from perceptual organization relate to natural scenes?](https://reader031.vdocuments.site/reader031/viewer/2022032522/56649d6a5503460f94a48384/html5/thumbnails/25.jpg)
25
Extension to Real Images
• Build up library of prototypical contour configurations by clustering local shape descriptors– Geometric Blur [Berg & Malik 01]
• Train a classifier which uses similarities to these prototype shapes to predict figure/ground label
![Page 26: 1 How do ideas from perceptual organization relate to natural scenes?](https://reader031.vdocuments.site/reader031/viewer/2022032522/56649d6a5503460f94a48384/html5/thumbnails/26.jpg)
26
Shapemes
Classifier using 64 shapeme features: 61%
![Page 27: 1 How do ideas from perceptual organization relate to natural scenes?](https://reader031.vdocuments.site/reader031/viewer/2022032522/56649d6a5503460f94a48384/html5/thumbnails/27.jpg)
27
Globalization of Figure/Ground Measurements
• Averaging local shapeme cue over human-marked boundaries: 71%
• Prior over junction types and label continuity: 79%