Locally adaptive template sizes for matching repeat images of Earth surface mass movements

Misganu Debella-Giloa and Andreas Kääbb

a, bInstitute of Geosciences, University of Oslo, P. O. Box 1047, Oslo, Norway

a m.d.gilo@geo.uio.no

ContentsIntroduction

Methods

Results

Conclusion

Introduction Repeat optical images are often used to monitor the displacement of Earth

surface masses (e.g. glacier flow, permafrost creep, rock slide, etc)

Independently ortho-rectified and co-registered images are matched using

certain similarity (dissimilarity) measure

Most commonly the normalized cross-correlation (NCC) is used

Image subset (template) from reference image (usually older) is taken and

its conjugate is searched in the target image (usually newer)

The distance between the central pixels of the reference and the search

templates is comupted as the horizontal displacement

Compute cross-correlation coefficientLocate the peak

Compute displacement

The issue …what template size is appropriate? Small templates lack adequate signal variance for the matching …. Ambiguity (noise) problem

Large templates may contain within template displacement gradient… geometric distortion

There is a need to compromise between ambiguity and geometric distortion

Noise and distortion levels may vary spatially

Need for spatially adaptive template sizes

Ideal template:

Contains optimum SNR,

Contains no geometric distortion, and

Is able to identify distinct feature when encountered

Its true match exists with optimized correlation coefficient

The study presents an algorithm which tries to meet these characteristics

A B C

Template size vs. SNR SNR=σ2s/ σ2n

Noise variance is computed using Immerkær’s (1996) method where the difference

between two Laplacian masks is used to convolve the image

To explore the relationship between SNR and template size:

Syntethic image containing distinct feature of 61 by 61 pixels which repeat itself

was created

The SNR was then computed for varying template sizes starting from 3 by 3 pixels

up to a maximum size set depending on the spatial resolution of the image and mass

movement type

The computation was then conducted on real images

Image sections of good contrast and poor contrast are included

SNR

Template (window) size

For small templates the noise variance is

very high while the signal variance is low

therefore the SNR is very low….. Poor

information content

As the size increases, the SNR increases.

If edge is encountered the SNR rises

sharply, and then decreases after crossing

the edge

The first encountered peak shows some

kind of feature boundary

It can also be saturation of signal

61

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

20000

0 50 100 150

Inte

nsity

var

ianc

e

Window size (pixels by pixels)

A

B

C

0

500

1000

1500

2000

2500

3000

3500

0 20 40 60 80 100 120 140

Noi

se v

aria

nce

Window size (pixels by pixels)

A

B

C

59

6137

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

0 20 40 60 80 100 120 140

Sign

al-to

-noi

se ra

tio

Window size (pixels by pixels)

A

B

C

A

B

C

0

500

1000

1500

2000

2500

3000

3500

0 50 100 150

Inte

nsity

var

ianc

e

Window size (pixels by pixels)

Aerial (Rockglacier)

Shadow

A

0

100

200

300

400

500

600

700

0 50 100 150

Noi

se v

aria

nce

Window size (pixels by pixels)

Aerial (rockglacier)

Shadow

B

39

0

200

400

600

800

1000

1200

1400

0 50 100 150

Sign

al to

Noi

se ra

tio

Window size (pixels by pixels)

Aerial (rockglacier)

Shadow

C

Table 1

0

10

20

30

40

50

60

70

0 50 100 150

Inte

nsity

var

ianc

e

Window size (pixels by pixels)

Snow (landsat)

Smooth rockglacier (Aerial)

A

0

100

200

300

400

500

600

700

0 50 100 150

Noi

se v

aria

nce

Window size (pixels by pixels)

Snow (Landsat)

Smooth Rockglacier (Aerial)

B

29

0

5

10

15

20

25

0 50 100 150

Sign

al-t

o-no

ise ra

tio

Window size (pixels by pixels)

Snow (Landsat)

Smooth rockglacier (aerial)

C

•Thus SNR peak can be

used to identify matchable

templates

•But the matable template

may be occluded or changed

unrecognizably

•But the NCC needs to be

computed to know if the size

is optimum and if the

template has a match

Template size and NCC To know the effect of template size on NCC peak, NCC peak is computed for varying template

sizes

For small templates, the maximum NCC is very high but ambiguous due to inadequate signal variance

B1

B2

B3

B40.820.840.860.880.900.920.940.960.981.00

0 20 40 60 80 100

Max

. Cor

rela

tion

Coeffi

cien

t

Template size (pixels by pixels)B

C1

C2

C3

0

20

40

60

80

100

0 20 40 60 80 100

Horiz

onta

l disp

lace

men

t (p

ixel

)

Template size (pixels by pixels)C

0.00

0.20

0.40

0.60

0.80

1.00

0 20 40 60 80 100 120 140

Max

imum

cor

rela

tion

coeffi

cien

t

Template size(pixels by pixels)

020406080

100120140160180

0 50 100 150

Horiz

onta

l disp

lace

men

t (p

ixel

s)

Template size(pixels by pixels)

A

C B A

The presence or absence of the peak of the SNR can help in selecting

matchable templates and excluding unmatchable ones

The presence or absence of the peak of the NCC maxima can help in

determining optimum template sizes (which have true matches with optimized

correlation-coefficient) and in excluding occluded templates

The algorithm tries to satisfy these conditions at each location

Method• Procedures:

• Take central pixels at intervals

• Compute SNR until it attains the first peak

• If it attains a peak within the set maximum value, take the template and its size (Tw),

• If no peak is attained, reject that central pixel

• For the central pixels that passed the SNR test, compute the NCC using the template sizes ranging beteen half Tw and twice Tw

• If the NCC attained a peak and the matching position is fixed over 3 consecutive template sizes, take that size as optimum and record the matching positions and compute the displacements

• The algorithm is applied and evaluated on:

• artificially deformed image: a glacier image subset was deformed artificially with full pixel displacement and Gaussian noise was added to model bi-temporal images

• Landsat PAN image pair of Baltoro glacier (Himalaya) with one year apart,

• Radarsat2 intensity image pair of Cronebreen glacier (Svalbard) with 25 days apart

Evaluation Visual: ….looking at the pattern of the computed displacement vectors

Mean Absolute Difference (MAD) between the computed and actual displacement for the artificially deformed images

SNR gain of reconstructing the reference image from the deformed image

Compute the global correlation coefficient (ρg) between the reference and the reconstructed

SNR =(ρg)/(1- ρg)

SNRgain=SNR (reconstructed)-SNR(origional)

Both the MAD and SNR gain are compared to that of different globally (image-wide) fixed template sizes

ResultsArtificially deformed images

D

B

C

A

Displacement vectors

computed using globally

fixed template sizes of 11

pixels (A), 61 pixels (B)

and the locally adaptive

algorithm (C) together

with the histogram of the

template sizes of the

locally adaptive algorithm

(D) for the noisy test

image.

Mean absolute error of displacement (MAEd) of the globally fixed template sizes (dotted line) and the locally adaptive algorithm (horizontal line) for the noise-free (left) and noisy (right) test images

Global correlation coefficients between the original reference image and the search image before (dashed horizontal lines), after reconstructing using the globally fixed (dotted lines) and the locally adapted (smooth horizontal lines) template sizes for the noise-free (left) and the noisy (right) test images

Statistics

Statistic Actual displacement data

61 pixels 11 pixel

Locally adapted

Minimum 0 0 1 0 Maximum 37 35.90 96.88 36.67 Mean 16.04 15.17 (1.76) a 27.62 (7.49) a 16.05 (0.65) a Standard deviation 9.18 8.84 22.90 9.16

aThe numbers in the brackets are the corresponding MAEd. Notice that by using the locally adaptive algorithm, the MAEd of the large template size is reduced by about 63% while that of the small template size is reduced by about 91%.

Table 1. Displacement statistics for the small, large and locally adapted template sizes for the central pixel of their respective templates of the noisy test image.

Real glacier images: Baltoro

Real glacier images: Cronebreen

Conclusions A new algorithm for locally adaptive template sizes in Normalized Cross-

Correlation (NCC)-based image matching for displacement measurement of Earth surface mass movements is tested and evaluated.

The algorithm performs better than globally fixed template sizes in its accuracy of matching and displacement estimation

It removes the mismatches due to ambiguity (noise) in small template sizes and reduces the errors of misrepresentation due to displacement gradient in large template sizes

Errors due to geometric distortion remain only where high noise level or lack of good signal variance necessitate the use of large template sizes.

The algorithm discards most of the templates which lack sufficient SNR and occluded templates (i.e. templates whose matches do not exist).

Pushes one step towards automation The computational efficiency of the algorithm is low and needs to be

improved.

Thank you!

•This study was supported by The Research Council of Norway (NFR) through the CORRIA project (no. 185906/V30),

Acknowledgements