detecting & modeling change in time-varying...

Detecting & Modeling Change in

Time-Varying Imagery

Peggy Agouris

Dept. of Spatial Information Engineering University of Maine

Overview

Problem(s)

Change Detection in Time-Varying Aerial Imagery

Tracking Positional Change and Modeling Spatiotemporal Behavior in Motion Imagery (incl. Video Sequences)

Examples

Problem

Change detection: one component of successful conflation of geospatial information

Integrated Environment

Translation onto new image

Object shape information (from GIS)

Shape accuracy estimates (from GIS)

Differential Snakes

Change detection

Versioning

GIS Updating

Traditional Snakes

Semi-automatic tool for object extraction

Based on the optimization of a model of

curve contrast and smoothness using

content-derived forces and an elastodynamic

model

edgecurvcontsnake EEEE ⋅+⋅+⋅= γβα

)( iedge vIE −∇=

211 2 +− +−= iiicurv vvvE

1−−−= iicont vvdE

Traditional Snakes Model

Total energy:

Continuity term:

Curvature term:

Edge term:

Optimization

Greedy algorithm : current point location is optimized, while previous and next points are fixed Stop criteria : number of points moved, change of total energy

vi

vi+1vi-1

Quality Evaluation of Extracted Road Network

Need a posteriori evaluation of object extractionResults are useful input for spatiotemporal change detectionAssumption: known energy function valuesFor sample points along an extracted object, values of uncertainty are generated using fuzzy rules

Quality Evaluation Rules

High Medium Low

Low High Low Low

High High Medium High

DEE

Points of interest are determined based on statistical properties of energy

Fuzzy rules of the form :

• If Et is LOW and DEt is LOW then U is LOW

d

Fin

Fout

vi

v0i

uncedgecurvcontsnake EEEEE ⋅+⋅+⋅+⋅= δγβα

Additional energy term (uncertainty)

Action is similar to an elastic spring force

Differential Snake Model

Uncertainty Energy

dvUncD

Eii

unc ⋅⋅

=)0(

1

dDi

1/Unc(v0i)

0

Eunc

Change Detection vs. Versioning

Change is detected if a road segment has moved beyond the stochastic range of older information

Versioning identifies road segments that can be delineated in the new image with better accuracy than their current database record

Example

Prior (red) and current (blue) road shape information

Change Detection & Versioning Experiments

Buffer zones of influence of prior

information

Result of change detection (blue line)

Result of versioning (purple line)

Change Detection & Versioning Experiments (cont.)

Prior and current road shape information

Buffer zones of influence of prior

information

After change detection (blue)

and versioning (purple)

1

2

3

4

5

6

1

2

3

4

5

6

Differential Snakes GUI (Change Detection & Versioning)

QuickTime™ and aMicrosoft Video 1 decompressorare needed to see this picture.

Performance and Accuracy Issues

Typical Performance Metrics

Average values for a road segment spanning

a 512x512 image window:

• Change Detection: 2.095 sec

• Versioning: 0.561 sec

Change Detection for

Closed Contour Objects

Important tool for dynamic scene analysis

Applications: surveillance, environmental, transportation, biomedical, etc.

Quick and efficient, requires proper initialization, assumes frequent monitoring (small movement of object between frames)

Differential Snakes for Tracking Object Contours

Extracted Object Contour from

Previous Frame

New Frame Information

Differential Snakes

New Object Contour

Estimation of Translation and Rotation

Translation: difference of positions of two geometric centersRotation: difference of direction of principal axes

Δϕ

Estimation of Uniform Expansion

Ratio of areas = (ratio of perimeters)2

Estimation of Radial Deformation

Use of polygon clipping techniques: - calculate the intersections between two input polygons- label edges as inside, outside, or shared- find the minimal polygons which are created by intersection- classify all minimal polygons into the output sets A∩B, A/B, and A\B

Experiments with Moving Objects

Track changes in the shape of an object

Example: a liquid that deforms non-uniformly

We show five distinct frames and the detected

change between them (frames n, n+1)

Area threshold to ignore small polygon changes

Integration of spatiotemporal tracking process in

a GUI (in Matlab)

Experiment with a Moving Object

100 150 200 250 300 350 400 450 500

100

150

200

250

300

350

400100 150 200 250 300 350 400 450 500

100

150

200

250

300

350

400100 150 200 250 300 350 400 450 500

100

150

200

250

300

350

400100 150 200 250 300 350 400 450 500

100

150

200

250

300

350

400

GUI for Spatiotemporal Change Detection


Remarks

Integration of object extraction and change detection

Introduction of uncertainty as external energy in a deformable modelChange detection using the uncertainty of the extraction

Framework for spatiotemporal tracking of object deformations

Estimate translation, rotation, radial deformations using geometric properties

Positional Change Detection & Modeling

in Motion Imagery

Problem

Detecting change in position and shape/extent of objects or events across time and space Modeling their spatiotemporal behavior

Trends in imagery collection: from static to motion and from single to multiple sensors.

Tremendous amounts of data.

Bottleneck in the analyst workforce.

Rationale

Automated motion imagery analysis solutions

Automation at various levels of the analysisprocess:

automated identification of trajectories in single video feeds (i.e. tracking positional change over time)automated content analysis to identify interesting spatiotemporal activities and support queries

Needs

Motion Trajectory Identification

Nodal Representation of Trajectories

Spatiotemporal Helix Modeling

Essential Issues

QuickTime™ and aMPEG-4 Video decompressor

are needed to see this picture.

0

200

400

801001201401601802002202400

10

20

30

40

50

60

70

80

90

x (column pixels)

1

3

2

55

1

3

2

5

2

3

1

5

2

5

3

5

1

3

7

2

5

1

7

3

5

2

6

7

1

3

4

2

6

1

7

3

4

2

6

8

3

4

1

2

6

8

3

4

6

2

1

8

3

4

6

2

1

Moving Object Trajectories

8

3

4

6

2

1

4

2

y (row pixels)

6

1

66

6

fram

e (t

ime)

Modeling Spatiotemporal Change

Over time objects/events may change their:

location (movement)

outline (deformation)

Need:

an integrated representation of movement

and deformation

Trajectories of moving objects:

3-d collections of points evolving through S-T space

•Generalization

•Summarization

•Behavior Analysis

QuickTime™ and a decompressor



are needed to see this picture.QuickTime™ and a

decompressorare needed to see this picture.



Moving Objects in the SpatioTemporal Domain

YUV420 codec decompressorare needed to see this picture.

YUV420 codec decompressorare needed to see this picture.

x

y

t

x

y

t

Summarization

The SpatioTemporal HelixAn integrated representation of movement and

deformation, and

A signature of an object’s spatiotemporal behavior.

Comprises a spine and prongsSpine models trajectory• Nodes: acceleration, deceleration, rotation

Prongs express deformation• Changes of a predefined magnitude• Recorded as time, percent change, azimuth

Helix RepresentationSpine:Spine: expresses spatioexpresses spatio--

temporal 3temporal 3--D movement of D movement of

the center of mass.the center of mass.

Prongs:Prongs: express expansion or express expansion or

collapse of the objectcollapse of the object’’s outline s outline

The Helix as a Spatiotemporal Database Index

Helixobjidt1,t2={node1,…noden; prong1,..prongm}

•• Node:Node: nnii(x,y,t,q(x,y,t,q))

•• Prong: Prong: ppjj(t,r,a(t,r,a11,a,a22))

Collecting Spine & Prong Information

Two novel image analysis techniques:

SOM with geometric analysis (g-SOM)•Describes ST trajectory of center of mass

Differential snakes•Allows calculation of percent change in

outline

Generalization

Hurricane Frances

Hurricane Charley

Hurricane Helixes

Hurricane Helixes: Charlie

QuickTime™ and aCinepak decompressor




Hurricane Helixes: Frances





Concluding Remarks

Incorporating uncertainty in change detection improves conflation and eliminates false positivesDetection of positional and shape change in motion imagery contributes to a better understanding of behavior of evolving events

For more information:

Peggy AgourisAnthony Stefanidis

{peggy, tony}@spatial.maine.edu

http://dipa.spatial.maine.edu

detecting & modeling change in time-varying...

Documents