mser taormina sep03

8/3/2019 MSER Taormina Sep03

http://slidepdf.com/reader/full/mser-taormina-sep03 1/156

Taormina September 10, 2003

Maximally Stable Extremal RegionsJ. Matas

Centre for Machine Perception,Czech Technical University,

Prague, Czech Republic.

15 COIL

19 Traff

ZuBu 22

Maximally Stable Extremal Regions (MSER) Example

15 COIL

19 Traff

ZuBu 22

Maximally Stable Extremal Region (MSER) - Definition

I is a mapping I : D ⊂ Z2 → P. The set of extremal regions Es is defined by:

1. a mapping s : P → S, where S is some totally ordered set (i.e. reflexive, antisymmetric andtransitive binary relation ≤ exists). Examples: (inverted) intensity, saturation, gradient

magnitude, . . .

2. An adjacency (neighbourhood) relation A ⊂ D × D is defined. In this paper 4-neighbourhoods

are used, i.e. p, q ∈ D are adjacent ( pAq) iff n

i=1 | pi − qi| ≤ 1.

Region

Q is a contiguous subset of D, i.e. for each p, q ∈ Q there is a sequence p, a1, a2, . . . , an, qand pAa1, aiAai+1, anAq.

(Outer) Region Boundary

∂ Q = {q ∈ D \ Q : ∃ p ∈ Q : qAp}, i.e. the boundary ∂ Q of region Q is the set of pixels

being adjacent to at least one pixel of Q but not belonging to Q.

Extremal RegionQ ⊂ D is a region such that for all p ∈ Q, q ∈ ∂ Q : I ( p) > I (q) (maximum intensity region)

or I ( p) < I (q) (minimum intensity region).

Maximally Stable Extremal Region (MSER)

Let Q1, . . . , Qi−1, Qi, . . . be a sequence of nested extremal regions, i.e. Qi ⊂ Qi+1. Extremal

region Qi∗ is maximally stable iff q(i) = |Qi−∆ \ Qi+∆|/|Qi| has a local minimum at i∗ (|.|

denotes cardinality). ∆ is a parameter of the method.

15 COIL

19 Traff

ZuBu 22

Maximally Stable Extremal Region (MSER) - Motivation

There are only n extremal regions (w.r.t. a given total ordering) in an image withn pixels.

Invariance to monotonic transformation. M :S → S

of image intensities.The set of extremal regions is unchanged after transformation M ,I ( p) < I (q) → M (I ( p)) = I ( p) < I (q) = M (I (q)) since M does not affectadjacency (and thus contiguity) and intensity ordering is preserved.

Invariance to adjacency preserving (continuous) transformation T : D → D

on the image domain.

Stability, since only extremal regions whose support is virtually unchanged overa range of thresholds is selected.

Multi-scale detection. Since no smoothing is involved, both very fine and verylarge structure is detected.

The set of all extremal regions can be enumerated in O(n log log n), i.e. almost

in linear time for 8 bit images. MSERs are of data-dependent shape. Often (except for elliptical regions)

multiple points can be reliably detected on their boundary. This in turn allowsconstruction of Local Affine Frames (coordinate system).

Local Affine Frame (LAF) correspondeces provide strong matching constraints

(e.g. Epipolar Geometry can be estimated from 3 correspondence by a modifiedversion of RANSAC).

15 COIL

19 Traff

ZuBu 22

Maximally Stable Extremal Region (MSER) - Algorithm

Algorithm 1: Enumeration of Extremal Regions. (outline)

Input: Image I Output: list of nested extremal regions

1. For all pixels sorted by intensity

2. Place pixel in the image.

3. Update the connected component structure.

4. Update the area for the effected connected component.

5. For all connected components

6. Local minima of the rate of change of its area define stable thresholds.

Computational complexity is dominated by step 2 (for 8 bit images). The list of connected

components and their areas is maintained using the efficient union-find algorithm with

O(n log log n) complexity.

Relation to watershed . Computationally essentially identical to efficient watershed algorithms.

The structure of output of the two algorithms is different. The watershed is a partitioning.

Relation to thresholding . Every extremal region is a connected component of a thresholded image.

However, all thresholds are tested and the stability of the connected components evaluated.

Output of MSER detection is not a binarized image. For some parts of the image, multiple stablethresholds exist and a system of nested subsets is output in this case.

15 COIL

19 Traff

ZuBu 22

Correspondence via Local Affine Frame Matching

1. Detect DistinguishedRegions (DRs). MSERs

[2] used here, but any

process producing imageregions stable under affine

transformations can be

exploited.

2. Build Local Coordinate

Systems (LAFs), applying

various affine covariantconstructions.

3. Define a Measurement

Region (MR) in terms

of the local coordinate

systems. A square

−1, 2 × −1, 2 is

[1] J. Matas, S. Obdrzalek, O. Chum. Local Affine Frames for Wide-Baseline Stereo. In ICPR 2002, pages 363–366, Quebec City, Canada.

[2] J. Matas, O. Chum, M. Urban, T. Pajdla. Robust Wide baseline Stereo from Maximally Stable Extremal Regions. In BMVC’02, pages384–393, Cardiff, UK.

15 COIL

19 Traff

ZuBu 22

The structure of the LAF matching algotithm (continued)

4. GeometricNormalisation: Trans-

form MRs of individual

LAFs into a canonical

5. PhotometricallyNormalise RGB values in

6. Local Correspondences

are established by

correlation of normalised

15 COIL

19 Traff

ZuBu 22

The structure of the LAF matching algotithm (continued)

Localisation: A robust voting scheme is applied

Every local correspondence provides a single estimate of the localisation

An example of local correspondences: Corresponding regions from two images.

15 COIL

19 Traff

ZuBu 22

Local Affine Frame Constructions

CG+DE Geometrical Center and Distance Extrema

1. Compute geometrical center of the region

2. Compute covariance matrix of region’s characteristic function

3. Compute distances of the contour points with respect to the covariance matrix

4. Select stable distance extrema

5. Construct local affine frame for every selected extrema

2TP+CG: Bi-tangent Points and the Geometrical Center

1. Compute bi-tangent points of a region’s contour concavity

2. Compute geometrical center of the region

3. Construct a local affine frame from three points

2TP+DM Bi-tangent Points and the Distance Maxima

2. Select the point on the region’s contour, that is farthest from the bitangent line

2TP+DP Bi-tangent Points and the Deepest Point of the concavity

2. Select the point on the concavity, that is farthest from the bitangent line

Note.The last two constructions allow for partially occluded regions, since neither thegeometrical center nor the covariance matrix is used.

15 COIL

19 Traff

ZuBu 22

Local Affine Frames - CG+DE frames

0 10 20 30 40 50 60 70 80 90 Contour vertex

Normalized distance

Top left:region with a fittedellipse.

Top right:

the regiontransformed to anormalized frame

Bottom:Distance betweencenter of gravity and

contour vertices, inthe normalized frame

15 COIL

19 Traff

ZuBu 22

Local Affine Frames CG+DE example

Two CG+DE frames for a single region. From left to right: original image; regioncontour, the ellipse given by the covariance matrix, and two detected frames (centerof gravity, distance extrema, ellipse); patch normalized frame according to the twoframes.

15 COIL

19 Traff

ZuBu 22

Local Affine Frames 2TP+CG example

2TP+CG frame. From left to right: original image; region contour, the ellipse givenby the covariance matrix, and the frame (two tangent points, region’s center of gravity); normalised image patch.

15 COIL

19 Traff

ZuBu 22

Local Affine Frames 2TP+DP example

2TP+DP frame. From left to right: original image; region contour, the ellipse givenby the covariance matrix, and the detected frame (two tangent points, the deepestpoint of the concavity); normalised image patch.

15 COIL

19 Traff

ZuBu 22

L l Affi 2 P DM l

Local Affine Frames 2TP+DM example

2TP+DM frame. From left to right: original image; region contour, the ellipse givenby the covariance matrix, and the detected frame (two tangent points, the contourpoint farthest to the bitangent); normalised image patch.

15 COIL

19 Traff

ZuBu 22

L l Affi F E l

Local Affine Frames Example

15 COIL

19 Traff

ZuBu 22

COIL 100 d t b

COIL 100 database

• The Columbia Object Library Database, 2 views (0◦ and 90◦) out of 72 for eachobject.

MPEG files showing all views.

15 COIL

19 Traff

ZuBu 22

COIL 100 res lts comparison to other methods

COIL 100 results - comparison to other methods

# of training views per objectMethod 18 8 4 2 1

5400 tests 6400 tests 6800 tests 7000 tests 7100 tests

LAFs 99.9% 99.4% 94.7% 87.8% 77.1%SNoW / edges [1] 94.1% 89.2% 88.3% - -SNoW / intensity [1] 92.3% 85.1% 81.5% - -Linear SVM [1] 91.3% 84.8% 78.5% - -Spin-Glass MRF [2] 96.8% 88.2% 69.4% 57.6% 49.9%Nearest Neighbor [1] 87.5% 79.5% 74.6% - -

[1] M. H. Yang, D. Roth and N. Ahuja. Learning to recognize 3D objects withSNoW. In ECCV-2000, pages 439-454

[2] B. Caputo, J. Hornegger, D. Paulus and H. Niemann. A Spin-Glass MarkovRandom Field for 3-D Object Recognition. Technical report LME-TR-2002-01,Lehrstuhl fur Mustererkennung, Institut fur Informatik, Universitat

Erlangen-Nurnberg.

15 COIL

19 Traff

ZuBu 22

COIL 100: The Occlusion Experiment 1

COIL-100: The Occlusion Experiment 1

The COIL-100 occlusion experiment, left: training views (full images), right: testviews (occluded images).

15 COIL

19 Traff

ZuBu 22

COIL 100: The Occlusion Experiment 3 The Results

COIL-100: The Occlusion Experiment 3 - The Results

Recognition rate for occluded test objects compared to the recognition rate withoutocclusion.

training views per object 18 8 4 2 1recognition rate/full views 99.9% 99.4% 94.7% 87.8% 76.0%recognition rate/occluded 92.6% 89.1% 82.6% 69.9% 63.3%

Recognition rates of published methods (full view):

# of training views per objectMethod 18 8 4 2 1

5400 tests 6400 tests 6800 tests 7000 tests 7100 tests

LAFs 99.9% 99.4% 94.7% 87.8% 77.1%SNoW / edges [1] 94.1% 89.2% 88.3% - -SNoW / intensity [1] 92.3% 85.1% 81.5% - -Linear SVM [1] 91.3% 84.8% 78.5% - -Spin-Glass MRF [2] 96.8% 88.2% 69.4% 57.6% 49.9%Nearest Neighbor [1] 87.5% 79.5% 74.6% - -

15 COIL

19 Traff

ZuBu 22

Experiments: Recognition of Traffic Signs

Recognition of traffic signs in real-world images

Total Correctly Failures Recognition

Images Recognised Rate54 51 3 94%

Known signs (models) The only failures, signs not detected

The database of scenes

15 COIL

19 Traff

ZuBu 22

Experiments: Recognition of Buildings ZuBuD dataset [1]

Experiments: Recognition of Buildings, ZuBuD dataset [1]

Recognition of buildings in real-world images

Application: Eg. navigation in urban scenes

ZuBuD dataset [1]: Database of 201 known buildings, 5 views each building,Separate set of 115 queries

Different viewpoints, different illumiantion, background clutter, occlusion bytraffic, trees, etc.

Method Average recall rR

r1 r2 r3 r4

LAFs 100.0% 99.1% 97.4% 92.8%HPAT [2] 86.1%

Corresponding query and database images

15 COIL

19 Traff

ZuBu 22

Examples of ZuBuD database images (known objects)

[1] H. Shao, T. Svoboda, L. Gool. ZuBuD — Zurich Buildings Database for Image Based Recognition. Technical report, Computer VisionLaboratory, Swiss Federal Institute of Technology.

[2] H. Shao, T. Svoboda, T. Tuytelaars, L. Gool. HPAT indexing for fast object/scene recognition based on local appearance. In CIVR03, toappear.

15 COIL

19 Traff

ZuBu 22

0 10 20 30 40 50 60 70 80 90 Contour vertex

Normalized distance

mser taormina sep03

Documents

taormina powerpoint

region detectors mser

what is ddos and the mitigation strategies sep03 public …

mser version pml update may 2013

hotel imperiale - taormina

linear time mser from books google

carta di taormina

mser-fpd 1

taormina - arrivalguides

declaracion derechos sexuales sep03 2014

taormina 2

+ taormina sicilia

seminario taormina romano

taormina city streets

recurrent convolutional neural network mser-based approach

taormina, sicilia

mser (maximally stable extremal regions) - unifi (maximally...

taormina, sicilia italy

tours & sightseeing taormina, guided tours · taormina &...

taormina matt matt rectified v2 10 faces · taormina matt...