algebraic relational approach to conflating images central washington university ellensburg, wa...
TRANSCRIPT
Algebraic relational approach to conflating images
Central Washington UniversityEllensburg, WA 98926, USA
Boris Kovalerchuk Dept of Computer
Science [email protected]
William SumnerDept of Geological
Supported by NIMA University Research Initiative (NURI)
grants (2002-2005). Collaboration: PNNL, NGC, LSU
Agenda
Introduction Example Algebraic summary Similarity measures Conflation/registration process Example Conclusions Questions
Example of matching problem:Corona and Landsat satellite images
Assume that there is no meta data about scales, rotations, types and names of features. It is hard to extract tie points, but easier to extract polylines..
Corona and Landsat satellite images
Feature overlap
It is hard to extract tie points, but relatively easy to extract a lot of polylines (continuous or with gaps) for features like shorelines, ridges, streams
Match of features provides more points than the tie point method. Key idea #1: Matching features
Algebraic invariants
Algebraic invariants is a new methodology that automates the matching of raster and vector images from many sources with various resolutions and reliability, giving them common scales and coordinates.
By using the term matching we mean fusion, correlation, registration and conflation of images and geospatial databases.
Terminology
Registration – georeferencing Co-registration – matching images one to another. If
one image registered then other one get registration via co-registration.
Conflation – matching features of two images and joining attributes of matched features from both sources.
Flexibility
In contrast to many traditional approaches, algebraic invariants:
(1) do not rely on the identification of control points,
(2) do not require common scales.
(3) do not require that the orientations of individual images be known and
(4) do not require that the types or names of linear features be known.
Key idea #1: Attempt to match features Key idea #2: Define features algebraically Key idea #3: Match features structurally Key idea #4: Permit gaps in features High speeds and automation are possible in
principal since registration is done using only a tiny fraction of the total image data.
Algebraic Invariants: Approach tomatching features
Comparison of Math alternatives
GeometryDistances, angles
Too variant
Algebra, relations between angles and distances, >,
Practically invariant
TopologyTopol. invariants Not robust (clouds)
Too invariant
Match changed Match robust Match refusal
Comparison of Math alternatives
“Topological invariants such as Betti numbers are insensitive to scale, and do not distinguish between tiny holes and large ones. Moreover, features such as pockets, valleys, and ridges---which are sometimes crucial in applications--- are not usually treated as topological features at all” [Bern et al., 1999, NSF workshop].
The number of connected components (in the same drainage system) can vary significantly between different maps, aerial photographs, and sensor data for the same area. It depends on characteristics such as human error, map resolution, sensor capabilities and parameters, obstacles (e.g. clouds), data processing methods. Thus, in general, topological invariants are not invariants for the conflation problem.
Support for Key idea #2: Define features algebraically.
Algebraic Invariants
Provide a mathematical model where:– There is a well established math language from abstract algebra– Polylines that represent geospatial features are introduced as
formal algebraic systems– Theorems on computational efficiency are proven
Provide conflating algorithms – That reduce computational complexity – That are robust (practically invariant)
Definitions
Definition. A pair a = A, is called an algebraic system if A is a set of elements and is a set of predicates {P} and operators {F} on A and on its Cartesian products, where P: AA...A [0,1] and F: AA...A A.
Further definitions complete the specification of linear features (polylines) and the model.
We omit the detailed mathematics, but will show an example to explain the concepts.
Note, this abstract algebraic system is not a traditional system of linear algebraic equations
Abstract Polylines
a3a1
a1 = [v0,v1], a2 = [v1,v2], ... , an = [vn-1,vn]
. . .
a2 a4
L1 =100o L2=200o L1 < L2
L1`=110o L2`=190o L1`< L2`
v1
v2
v0a5
a
b L1`
L2'
Invariant relation between angles,despite variation of angles
Possible gap
Structure of feature a as a matrix of relations between angles
Angle L1 L2 L3 L4 L5 L6
L1 1 0 0 1 0 1
L2 1 0 1 0 0
L3 1 0 1 0
L4 1 1 0
L5 1 1
L6 1
Li<Lj P(Li,Lj)=0
A similar matrix can be constructed for relations between lengths of line segments or other task relevant characteristics of the feature. Key idea #5: Algebraic structure is expandable
Conflating algorithms
Based on this algebraic model several algorithms may be designed to conflate images.
One is to build matrices of relations that describe structures of two features a and b which are then searched for their largest common part. This is done by “sliding down” the diagonals of the matrices in the following way.
Illustrative matrix feature b and match with a (in grayed box)
Angle L1 L2 L3 L4 L5 L6 L7 L8 L9 L10
L1 1 0 0 1 0 1 1 1 0 1
L2 1 0 1 0 0 0 0 0 1
L3 1 0 1 0 1 0 0 1
L4 1 1 0 1 1 1 1
L5 1 1 0 0 0 0
L6 1 1 1 0 0
L7 1 0 0 0
L8 1 1 1
L9 1 1
L10 1
Example
07/17/2002 Central Washington Universtiy
An Example
CoronaLandsat
After rotation, scaling, and translation
Theorems are proven, such as
Theorem. If the number of elements in linear features a and b equals n, then their maximum co-reference subsystem e can be found in O(n3) matrix comparisons for the worst-case scenario that is equivalent to O(n5) binary comparisons.
Useful, but . . .
Comparison techniques are needed for polylines that have different sampling densities.
Similarity measures have been developed to do this.
Similarity Measures
To define measures we consider a polyline a with two end points p1=(x1,y1,z1) and p2=(x2,y2,z2).
The recurrent function G(n) produces a set of simpler polylines that interpolate polyline a. Arguments of the function G are n = 2k where k = 1,2,…, that is 2, 4, 8, 16, 32…
Structural interpolations of a polyline
We introduced measures of structural similarities of features based on comparison of structural interpolations and generalized this to measuring structural similarities of entire images.
Conflation/registration process
Process illustration
Two vector image data sets are considered.
Image 1 consists of 1497 line segments.
Image 2 consists of 407 line segments.
Thanks to S. Sento and P. Brennan (Northrop Grumman) for providing data.
Input vector images
“Are these two images of the same scene?”
The two images conflated and combined
Features in common
Features not in common
Red-Image 1, blue –Image 2
Observations
The number of polylines in common supports the conclusion that these are two images of the same scene.
The number of polylines not in common illustrates the need for additional information to know whether these are the result of incomplete or faulty feature information, or changes in the scene between the times of acquisition.
Key idea #1: Attempt to match features Key idea #2: Define features algebraically Key idea #3: Match features structurally Key idea #4: Permit gaps in features High speeds and automation are possible in
principal since registration is done using only a tiny fraction of the total image data.
Summary: Key ideas
Summary: conclusion
This algebraic relational approach is a promising way of conflating both raster and vector images
– with no control points– with unknown scales– with unknown orientations
Can be used to automatically search for specific objects of interest by predefining their abstracted linear shapes
Can be used to automate the identification of locations that change in time
More work ahead, e.g., to incorporate prior imagery knowledge. Test data welcome
Super features
The following slides illustrate super features
8
538
1 897 4 284 -1 47.7897 30.7046 7.51
1 897 -1 284 2 47.7897649413 30.7049247065 0
1 897 5 -1 -1 47.7898 30.7051 7.51
1 897 6 -1 -1 47.7899 30.7053 7.51
1 897 7 -1 -1 47.79 30.7055 7.51
1 897 8 -1 -1 47.7902 30.7059 7.26
1 897 9 -1 -1 47.7905 30.7062 7.51
1 897 10 -1 -1 47.791 30.7065 7.5
1 897 11 -1 -1 47.7911 30.7066 7.38
1 897 12 -1 -1 47.7913 30.7067 7.25
1 897 13 -1 -1 47.7918 30.7069 7.25
1 897 -1 -1 -1 47.7919613336 30.7069968001 0
1 897 14 -1 -1 47.7923 30.7072 7.62
1 897 15 -1 -1 47.7928 30.7075 7.62
1 897 16 -1 -1 47.7932 30.7079 7.13
1 897 17 -1 -1 47.7937 30.7083 7.13
1 897 -1 -1 -1 47.793769313 30.7084617304 0
1 897 18 -1 -1 47.794 30.709 6.89
1 897 19 -1 -1 47.7942 30.7096 6.89