Globally Optimal Globally Optimal Grouping for Symmetric Grouping for Symmetric Closed Boundaries with Closed Boundaries with
Stahl/ Wang MethodStahl/ Wang Method
Vida MovahediVida MovahediSeptember 2007September 2007
2/31
IntroductionIntroduction• Grouping seeks to identify some
perceptually salient structure in noisy images
• Minimize a pre-defined grouping cost (function) that negatively measures the perceptual saliency of the resulting structure based on some psychological vision rules, such as the Gestalt laws
3/31
Combine Boundary & Region Combine Boundary & Region infoinfo
• Two boundary properties: – proximity and closure
• One region property: – enclosed region area
• Grouping cost: – Ratio between the total gap length along the
boundary and the area enclosed by the boundary
4/31
Problem FormulationProblem Formulation• Step 1: edge detection, construct a set of line
segments• Step 2: gap-filling (line) segments• Step 3: Find a boundary that minimizes the
grouping cost
5/31
Grouping CostGrouping Cost• Solid and dashed edges • Graph G can be called a solid-dashed (SD) graph• Alternate cycle (alternating between solid and
dashed)• Construct a pair of edges e+ and e- for each line
segment (e+ left to right direction)
6/31
Grouping Cost (Cont.)Grouping Cost (Cont.)• Define two edge-weight functions
• w1(e) = 0 if solid edge
= length of the line segment if dashed
• w2(e) = signed area associated to the corresponding line segment
7/31
Grouping Cost (Cont.)Grouping Cost (Cont.)
• Cost function= W1/W2
• Looking for a C that minimizes above cost• Corresponds to a boundary B, that minimizes
)(
Β
Barea
8/31
Symmetry as a cueSymmetry as a cue• Many structures show (bilateral) symmetry
• Grouping for symmetric boundaries is a challenging problem:
a) Boundary symmetry is not a simple local measureb) Need a unified grouping cost to flexibly integrate
different grouping cues (proximity, etc)c) The cost should avoid undesirable explicit or
implicit biases, such as bias toward shorter boundaries
9/31
Changing the grouping Changing the grouping tokenstokens
• Previously the grouping tokens were line segments, detected (solid) segments vs. gap-filling (dashed) segments
• n detected segments, 2n endpoints, ideally n(2n-2) gap filling segments
• Encoding symmetry into an individual line segment?!
• Symmetry can be encoded to a pair of segments Symmetric Trapezoids
10/31
Symmetric TrapezoidsSymmetric Trapezoids1) Find the angle-bisector line l2) Find the projections of segments to l called the
axis segment3) Map this axis segment back to segments,
resulting in a (symmetric) trapezoid• If no overlaps, no symmetric trapezoid constructed• pair every two detected segments, or pair a gap-filling
segment with a detected segment
11/31
Gap-filling quadrilateralsGap-filling quadrilaterals
• constructed by connecting a parallel side of one trapezoid and a parallel side of another trapezoid
• May not be symmetric, its axis segment constructed simply by connecting the endpoints of the axis segments of two neighboring symmetric trapezoid
• Two endpoints for a trapezoid four different gap-filling quadrilaterals
12/31
Grouping Cost FunctionGrouping Cost Function
: The total gap along the boundary, reflecting the preference of a boundary with good proximity
: A measure related to the collinearity of the boundary’s axis, reflecting the preference of a boundary with good symmetry
: The region area enclosed by the boundary, sets a preference to produce larger rounder structures
13/31
Graph ModelingGraph Modeling• Model the trapezoids and quadrilaterals using an
undirected graph
• A pair of solid edges eT+ and eT
- for each trapezoid T
• A pair of dashed edges eG+ and eG
- for each quadrilateral G
• mirror edges: an abstraction of the axis segment
14/31
Graph Modeling (Cont.)Graph Modeling (Cont.)
• Consider only the quadrilaterals that lead to a non-intersected boundary (not P2P6P3P7)
• Special case where the constructed gap-filling quadrilateral contains a self intersection (P2P3 and P6P7 not intersecting)
• e+ for counterclockwise, e- for clockwise
15/31
Graph Modeling (Cont.)Graph Modeling (Cont.)
16/31
Edge Weight FunctionsEdge Weight Functions
• For each e, two weight functions
• w1(e): measuring gap and symmetry– w1(e)= 0 if corresponding trapezoid constructed
from detected segments
– w1(e)= length of gap-filling segment if constructed from detected and gap-filling
– If e is dashed, w1(e)=
total gap length + collinearity of the axis
17/31
Auxiliary EdgesAuxiliary Edges
Four auxiliary edges
W1(e)
= |P1P12|D + |P6P7|D
18/31
Optimal BoundaryOptimal Boundary• w2(e)= signed area of the T or G corresponding to
e , w2(e)=0 for auxiliary edges
• Note w1(e+)= w1(e-) and w2(e+)= -w2(e-)>0
• Mirror cycles:
•
• Cycle ratio:
19/31
Optimal Boundary (Cont.)Optimal Boundary (Cont.)• Proven that C does not contain more than one
auxiliary edge
• Use an available graph algorithm to find an alternate cycle C with minimum cycle ratio
• e.g. the minimum-ratio-alternate-cycle algorithm introduced in a previous paper finds the optimal cycle in polynomial time
20/31
• Consider quadrilateral P2P3P6P7
• Contribution to |B|D =|P2P3|D+|P6P7|D≠|P2P3|+|P6P7|
• (a) • (b)
Gap-Length MeasureGap-Length Measure
21/31
ImplementationImplementation• n detected segments• For every endpoint, consider K shortest
gap-filling segments (K=5) O(n) gap-filling segments
• Constructing trapezoids(a) Pair every two detected segments (b) Pair every detected with every gap filling O(n2) trapezoids
• Constructing quadrilateralsPair every two trapezoids O(n4) ?!
22/31
StrategiesStrategies• 3 strategies to reduce this number (reduce the
number of dashed edges in the constructed graph):(1) Consider the quadrilateral that has the shortest axis
segment out of four choices, do not consider if the total gap length introduced is >D1 (D1=30 pixels)
(2) Avoid constructing a quadrilateral to connect two trapezoids that share a same portion of a detected segment
(3) Avoid constructing quadrilaterals that lead to an axis with low collinearity or |sin (angles)| > D2 (D2=0.5)
• To reduce the number of auxiliary edges:– Only consider the axis-segment end-points around which
the gap length is less than a given threshold <D3 (D3=20)
23/31
Experiments- Synthetic DataExperiments- Synthetic Data
• A pair of synthetic boundaries (one desired symmetric, another non-symmetric)
• Introducing gaps along boundary and adding noise segments in image
• Comparing with RC (authors’ method without symmetry) and EZ (Elder & Zucker method ‘96)
24/31
Performance evaluationPerformance evaluation• Performance using a region coincidence
measure with ground truth– R: region enclosed by the desired ground-truth
boundary– R’: region enclosed by the detected boundary– Region coincidence measure:
• How good is this measure?
25/31
Synthetic Data- ResultsSynthetic Data- Results
26/31
Experiments- Real ImagesExperiments- Real ImagesSRC RC EZ
27/31
Other experimentsOther experiments• Detecting multiple boundaries
– Remove all the trapezoids along the detected boundary and then repeat
– Problem: detecting the same boundary again especially when boundary has multiple symmetry axes
• Effects of changing – Related to image size
• Effects of changing other thresholds– No effect on results, just on running time
• Special cases:– Two disjoint closed boundaries that are symmetric– Two disjoint closed boundaries that form a ring
28/31
ExtensionsExtensions• Use boundary continuity, or smoothness
– Not so good when the actual boundary is NOT smooth
• Use region’s intensity homogeneity
•
29/31
Running TimeRunning Time• Worst-case time complexity:
= O(|V|3/4.|E|)= O(n5.5) if n detected segments
• Actual running times, not so bad, because of introduced strategies
30/31
SummarySummary• A new grouping algorithm for detecting
closed boundaries that show good bilateral symmetry
• Combining boundary and region information
• Combining local and global information
31/31
ReferencesReferences
• J.S. Stahl and S. Wang, “Edge Grouping Combining Boundary and Region Information”, accepted for publication in IEEE Trans. on Image Processing, 2007.
• J.S. Stahl and S. Wang, “Globally Optimal Grouping for Symmetric Closed Boundaries by Combining Boundary and Region Information”, accepted for publication in IEEE Trans. on Pattern Analysis and Machine Intelligence, 2007.
