Globally Optimal Globally Optimal Grouping for Symmetric Grouping for Symmetric Closed Boundaries with Closed Boundaries with

Stahl/ Wang MethodStahl/ Wang Method

Vida MovahediVida MovahediSeptember 2007September 2007

2/31

IntroductionIntroduction• Grouping seeks to identify some

perceptually salient structure in noisy images

• Minimize a pre-defined grouping cost (function) that negatively measures the perceptual saliency of the resulting structure based on some psychological vision rules, such as the Gestalt laws

3/31

Combine Boundary & Region Combine Boundary & Region infoinfo

• Two boundary properties: – proximity and closure

• One region property: – enclosed region area

• Grouping cost: – Ratio between the total gap length along the

boundary and the area enclosed by the boundary

4/31

Problem FormulationProblem Formulation• Step 1: edge detection, construct a set of line

segments• Step 2: gap-filling (line) segments• Step 3: Find a boundary that minimizes the

grouping cost

5/31

Grouping CostGrouping Cost• Solid and dashed edges • Graph G can be called a solid-dashed (SD) graph• Alternate cycle (alternating between solid and

dashed)• Construct a pair of edges e+ and e- for each line

segment (e+ left to right direction)

6/31

Grouping Cost (Cont.)Grouping Cost (Cont.)• Define two edge-weight functions

• w1(e) = 0 if solid edge

= length of the line segment if dashed

• w2(e) = signed area associated to the corresponding line segment

7/31

Grouping Cost (Cont.)Grouping Cost (Cont.)

• Cost function= W1/W2

• Looking for a C that minimizes above cost• Corresponds to a boundary B, that minimizes

)(

Β

Barea

8/31

Symmetry as a cueSymmetry as a cue• Many structures show (bilateral) symmetry

• Grouping for symmetric boundaries is a challenging problem:

a) Boundary symmetry is not a simple local measureb) Need a unified grouping cost to flexibly integrate

different grouping cues (proximity, etc)c) The cost should avoid undesirable explicit or

implicit biases, such as bias toward shorter boundaries

9/31

Changing the grouping Changing the grouping tokenstokens

• Previously the grouping tokens were line segments, detected (solid) segments vs. gap-filling (dashed) segments

• n detected segments, 2n endpoints, ideally n(2n-2) gap filling segments

• Encoding symmetry into an individual line segment?!

• Symmetry can be encoded to a pair of segments Symmetric Trapezoids

10/31

Symmetric TrapezoidsSymmetric Trapezoids1) Find the angle-bisector line l2) Find the projections of segments to l called the

axis segment3) Map this axis segment back to segments,

resulting in a (symmetric) trapezoid• If no overlaps, no symmetric trapezoid constructed• pair every two detected segments, or pair a gap-filling

segment with a detected segment

11/31

Gap-filling quadrilateralsGap-filling quadrilaterals

• constructed by connecting a parallel side of one trapezoid and a parallel side of another trapezoid

• May not be symmetric, its axis segment constructed simply by connecting the endpoints of the axis segments of two neighboring symmetric trapezoid

• Two endpoints for a trapezoid four different gap-filling quadrilaterals

12/31

Grouping Cost FunctionGrouping Cost Function

: The total gap along the boundary, reflecting the preference of a boundary with good proximity

: A measure related to the collinearity of the boundary’s axis, reflecting the preference of a boundary with good symmetry

: The region area enclosed by the boundary, sets a preference to produce larger rounder structures

13/31

Graph ModelingGraph Modeling• Model the trapezoids and quadrilaterals using an

undirected graph

• A pair of solid edges eT+ and eT

- for each trapezoid T

• A pair of dashed edges eG+ and eG

- for each quadrilateral G

• mirror edges: an abstraction of the axis segment

14/31

Graph Modeling (Cont.)Graph Modeling (Cont.)

• Consider only the quadrilaterals that lead to a non-intersected boundary (not P2P6P3P7)

• Special case where the constructed gap-filling quadrilateral contains a self intersection (P2P3 and P6P7 not intersecting)

• e+ for counterclockwise, e- for clockwise

15/31

Graph Modeling (Cont.)Graph Modeling (Cont.)

16/31

Edge Weight FunctionsEdge Weight Functions

• For each e, two weight functions

• w1(e): measuring gap and symmetry– w1(e)= 0 if corresponding trapezoid constructed

from detected segments

– w1(e)= length of gap-filling segment if constructed from detected and gap-filling

– If e is dashed, w1(e)=

total gap length + collinearity of the axis

17/31

Auxiliary EdgesAuxiliary Edges

Four auxiliary edges

W1(e)

= |P1P12|D + |P6P7|D

18/31

Optimal BoundaryOptimal Boundary• w2(e)= signed area of the T or G corresponding to

e , w2(e)=0 for auxiliary edges

• Note w1(e+)= w1(e-) and w2(e+)= -w2(e-)>0

• Mirror cycles:

•

• Cycle ratio:

19/31

Optimal Boundary (Cont.)Optimal Boundary (Cont.)• Proven that C does not contain more than one

auxiliary edge

• Use an available graph algorithm to find an alternate cycle C with minimum cycle ratio

• e.g. the minimum-ratio-alternate-cycle algorithm introduced in a previous paper finds the optimal cycle in polynomial time

20/31

• Consider quadrilateral P2P3P6P7

• Contribution to |B|D =|P2P3|D+|P6P7|D≠|P2P3|+|P6P7|

• (a) • (b)

Gap-Length MeasureGap-Length Measure

21/31

ImplementationImplementation• n detected segments• For every endpoint, consider K shortest

gap-filling segments (K=5) O(n) gap-filling segments

• Constructing trapezoids(a) Pair every two detected segments (b) Pair every detected with every gap filling O(n2) trapezoids

• Constructing quadrilateralsPair every two trapezoids O(n4) ?!

22/31

StrategiesStrategies• 3 strategies to reduce this number (reduce the

number of dashed edges in the constructed graph):(1) Consider the quadrilateral that has the shortest axis

segment out of four choices, do not consider if the total gap length introduced is >D1 (D1=30 pixels)

(2) Avoid constructing a quadrilateral to connect two trapezoids that share a same portion of a detected segment

(3) Avoid constructing quadrilaterals that lead to an axis with low collinearity or |sin (angles)| > D2 (D2=0.5)

• To reduce the number of auxiliary edges:– Only consider the axis-segment end-points around which

the gap length is less than a given threshold <D3 (D3=20)

23/31

Experiments- Synthetic DataExperiments- Synthetic Data

• A pair of synthetic boundaries (one desired symmetric, another non-symmetric)

• Introducing gaps along boundary and adding noise segments in image

• Comparing with RC (authors’ method without symmetry) and EZ (Elder & Zucker method ‘96)

24/31

Performance evaluationPerformance evaluation• Performance using a region coincidence

measure with ground truth– R: region enclosed by the desired ground-truth

boundary– R’: region enclosed by the detected boundary– Region coincidence measure:

• How good is this measure?

25/31

Synthetic Data- ResultsSynthetic Data- Results

26/31

Experiments- Real ImagesExperiments- Real ImagesSRC RC EZ

27/31

Other experimentsOther experiments• Detecting multiple boundaries

– Remove all the trapezoids along the detected boundary and then repeat

– Problem: detecting the same boundary again especially when boundary has multiple symmetry axes

• Effects of changing – Related to image size

• Effects of changing other thresholds– No effect on results, just on running time

• Special cases:– Two disjoint closed boundaries that are symmetric– Two disjoint closed boundaries that form a ring

28/31

ExtensionsExtensions• Use boundary continuity, or smoothness

– Not so good when the actual boundary is NOT smooth

• Use region’s intensity homogeneity

•

29/31

Running TimeRunning Time• Worst-case time complexity:

= O(|V|3/4.|E|)= O(n5.5) if n detected segments

• Actual running times, not so bad, because of introduced strategies

30/31

SummarySummary• A new grouping algorithm for detecting

closed boundaries that show good bilateral symmetry

• Combining boundary and region information

• Combining local and global information

31/31

ReferencesReferences

• J.S. Stahl and S. Wang, “Edge Grouping Combining Boundary and Region Information”, accepted for publication in IEEE Trans. on Image Processing, 2007.

• J.S. Stahl and S. Wang, “Globally Optimal Grouping for Symmetric Closed Boundaries by Combining Boundary and Region Information”, accepted for publication in IEEE Trans. on Pattern Analysis and Machine Intelligence, 2007.