divide-and-conquer strategies for hyperspectral image ... · divide-and-conquer strategies for...

TU Graz - Advanced Signal Processing 1

Divide-and-Conquer Strategies for HyperspectralImage Processing [1]

Bernd Bachofner, Gernot Riegler

Advanced Signal Processing 1

14.01.2013

Bernd Bachofner, Gernot Riegler 14.01.2013 page 1/59


OutlineIntroduction in Hyperspectral Image Processing

Image Acquisition

Decorrelation Methods

Devide and Conquer Methods

Wavelet TransformSTFTContinuous Wavelet Transform

Recursive KLTRecursive Strategy Example: AVIRIS Hyperspectral ImageCoding

POT

Comparison of Strategies

Another Example: Anomaly Detection




Image Acquisition





POT





Overview

I Human eye sees visible light in three bands (red, green andblue)

I Spectral imaging divides the spectrum into many more bands

I Can be extended beyond the visible

I Sensors look at objects using a vast portion of theelectromagnetic spectrum

I Certain objects leave unique fingerprints across theelectromagnetic spectrum

I Enables identification of materials (spectral signature of oil)



General

I Sensors collect information as a set of images

I Each image represents a range of the electromagneticspectrum (spectral band)

I Images are combined to form a 3-D data cube for processing

I Generated from airborne sensors or satellites



Automatic Target Detection

I A hyperspectral remot sensig system has four partsI Radiation (or illuminating) sourceI Atmospheric pathI The imaged surfaceI The sensor



Quality of the image

I Spectral resolution(precision of the sensor) is equivalent to thewidth of each band of the spectrum

I Dependent from the spatial resolutionI Identify objects even if they are only captured with a few pixels

(spatial resolution ok)I If pixels are too large, objects cannot easily be identified

(spatial resolution to low)I If pixels are too small, the energy captured by the sensor is low

(spatial resolution to high)I Decrease of the signal to noise ratio



Consequence

I We need ’good’ data for further processing

I Airborne sensors or satellites have limited resources(calculation power, memory, transmission band width)

I We need efficient algorithms to reduce redundancy in data

I Divide and Conquer methods of the KLT(Karhunen-Loevetransform) are reasonable to use




Image Acquisition





POT





General

I Decorrelation is used to reduce autocorrelation within a signalI Autocorrelation is the cross-correlation of a signal with itself

I It is a tool for finding repeating patterns

I Cross-correlation is a measure of similarity of two waveforms



Karhunen Loeve Transform (KLT)

I Is a powerful decorrelation transform

I Minimizes the total mean square error ⇒ optimally compactsthe energy

I Maximizes the variance and minimizes the reconstruction error

I No correlation remains among its outputs

I Very high computational cost

I Very high memory requirements

I Lack of component scalability (e.g. different results would beobtained if one uses Fahrenheit rather than Celsius)



Principal component analysis (PCA)I The discrete version (coefficients computed from samples) of

the KLTI Orthogonal transformation to convert a set of possibly

correlated variables into a set of values of linearly uncorrelatedvariables ⇒ principal components

I Number of principal components is ≤ the number of originalvariables

I This transformation is defined such that the first-pc has thelargest possible variance

I PCs are independent only if the date set is jointly normallydistributed

I PCA is sensitive to the relative scaling of the original variablesI Can be done by eigenvalue decomposition of the covariance

matrix or by Singular Value Decompostion (SVD) of the datamatrix



PCA continued

I Can be used for dimensional reduction of high dimensionaldata

I DetailsI XT ... data matrix with zero meanI The SVD of X is X = WΣV T

I W is the matrix of eigenvectors of the covariance matrix XXT

I Σ matrix is a rectangular diagonal matrix with non-negativereal numbers on the diagonal

I V is the matrix of eigenvectors of XTX

I PCA transform is then Y T = V ΣT

I Y T is a linear transformation of the corresponding row of XT

I First column of Y T contains the first principal components

I The second column of Y T contains the second principalcomponents and so on




Image Acquisition





POT





General

I Allows to approximate the KLT at a fraction of thecomputational cost

I Reduces the memory requirements

I Provides us with some component of scalabilityI The KLT is a transform that adapts to the statistics of its

inputI yi = KLTΣx = QT (xi − x)I QT matrix of the eigenvalue-decomposition of the covariance

matrix Σx of the input dataI X = {xi} ∀iI The term x is the input vector average, used to guarantee

zero-mean data

I The computational cost of the KLT is dominated by thequadratic cost of the matrix/vector multiplication, and by thecovariance matrix calculation



General continued II Divide and conquer methods overcome this problem by

dividing the KLT into a collection of smaller transforms with alesser overall cost

I Smaller transforms have to be arranged in a way that they areapplied only where they are more effective

I It is worth noting that transforms provide little overall benefitsin portions of data with low amounts of information

I Example: A first level of KLT transforms is applied to providelocal decorrelation, with the most significant half of theoutputs of each transform forwarded to a next level



General continued III In the above example one large transformation is replaced by

seven smaller transforms each of1

4of the original size

I The transform cost is mainly quadratic, each smaller

transform has1

16of the original cost

I Yielding a cost for the whole approach of7

16≈ 45% of the

original costI The approach improves component scalability, which allows

random access to specific components in a compressedcodestream

I Reduces cost for inverse transform operations (In the aboveexample only 8 outputs required to perform the operation)

I Allows decoding of portions of a compressed image withouthaving to process or download the full compressed data



Problems with Divide and Conquer methods

I With D&C methods we have the problem of combinatorialexplosion in the number of possible D&C schemes

I With no constraints we have 8.77 ∗ 1026 possible D&Cschemes for a 16-input KLT

I Not all of the D&C schemes have equal decorrelationperformance

I Data do not always follow the Gaussian model on which thetheory is based



Divide and Conquer Strategies

I Can be classified in four families:I Recursive,I Single-Level,I Two-Level andI Multilevel strategies



RecursiveI Based on successive subdivision of a KLT into three half-sized

KLTsI Two half-sized KLTs provide a first level of local decorrelation,

while the third one provides partial global decorrelation fromthe outputs of the other two

I The use of recursion proves a computational complexity belowthat of the KLT

I Performance very close to the KLT



Single LevelI Based on a single level of small transforms that provide only

local decorrelationI Decorrelation properties are limited, since it produces low

amounts of side informationI May work well in situations where the size of side information

is a significant portion of the bit rateI Example: Very low bit rates



Static Two LevelI Works without any recursionI Achieve decorrelation locally on a first level and globally on a

second levelI Segments the first level of decorrelation in a large number of

small KLTsI In the second level the important outputs of the first level

KLT are decorrelated together with the equivalent output ofthe other first-level KLTs



Dynamic Two LevelI Works without any recursionI Achieve decorrelation locally on a first level and globally on a

second levelI Segments the first level of decorrelation in a large number of

small KLTsI In the second level the important outputs of the first level

KLT are decorrelated together with the equivalent output ofthe other first-level KLTs

I Pruning is performed after the transform is trained to removeless contributing inputs of second-level KLTs



Multilevel Strategies

I At each level, components are sliced into clusters of KLTs,and for each cluster some of the outputs are forwarded to anext level

I Until one last level decorrelates together all the remainingcomponents

I No permutation of components between each level, howeverthey provide good performance



Regular MultilevelI The most simplest and naive member of the family of

multilevel strategiesI Includes strong regularity constraints to avoid

explosion on multilevel structures



Static MultilevelI Uses eigenthresholding methods as constraintsI Eigenthresholding are analytically methods used to quantify

the relevant outputs of each KLTI On the static variant, the possible structures are reduced from

millions to a few hundredI Clusters are all of the same size, and the same number of

components are forwarded to the next levelI Best structures are empirically selected for and from a training

data set



Dynamic MultilevelI Produces one structure of equal cluster size in all levelsI Different number of important outputs for each small KLT

may be selected as the transform is applied



Pairwise Orthogonal Transform (POT)I Characterized by its minimal structure of two component

KLTsI Provides the possibility of operation under strong memory

constraintsI Eliminates the numerically cumbersome eigendecomposition

procedure which is required in other structures




Image Acquisition





POT





Motivation

I Provide moderate spectral decorrelation [2]

I Low computational cost

I Presence in the hyperspectral image coding literature [3]



Short Term Fourier Transform

STFT

X(f, τ) =

∫x(t)w(t− τ)e−j2πftdt

I Windowed FT

I Gabor-Transformation: w(t) = e− t2

(∆t)2

I narrow window: good time resolution, bad frequencyresolution

I broad window: bad time resolution, good frequency resolution

I STFT: local analysis with fixed frequency resolution

I Goal: local analysis with variable frequency resolution



CWT

I Idea: Window function contains frequency information

I Adjust window function during analysis



Wavelets

I Basic operationsI TranslationI Dilation

Example: Haar-Wavelet

Ψ(t) =

1 0 ≤ t < 0.5

−1 0.5 ≤ t < 1

0 else

Ψab =1√a

Ψ(t− ba

)



Wavelet

I Ψ(t) is an oscillating function

I Ψ(t) is localized in a finite interval

I If Ψ(t) fulfills: CΨ = 2π∫∞−∞

| ˆΨ(ω)|2|ω| dω <∞

I Then Ψab(t) = 1√aΨ( t−ba ) is a wavelet-basis



CWT

I WΨ(a, b) = 1√a

∫∞∞ x(t)Ψ( t−ba )dt

I x(t) is projected onto Ψab(t)

I The coefficient WΨ(a, b) measures how well x(t) fits Ψab(t)

I Small a⇒ small frequencies and vice versa



DWT

I Introduced by Mallat [4]

I Successive lowpass and highpass filtering




Image Acquisition





POT





Recursive KLT

I Origninally not proposed for hyperspectral image coding [5],[6]

I Successive subdivision of a KLT

I Local decorrelation

I Global decorrelation



Recursive KLT

I The N -dimensional signal is equally divided into two groups

I Each group is decorrelated by the KLT

I Outputs are sorted according to the variance

I Outputs with high variance are transformed again



Airborne Visible Infrared Imaging Spectrometer

I AVIRIS is an airborne hyperspectral sensor from the NASA

I 224 contiguous spectral bands each .01µm wide

I Ranging from 0.4µm to 2.5µm



Recursive Strategy

I 614× 2206× 224 pixel Image

I ⇒ 614× 2206 input vectors ∈ R224

I 81 small KLTs, each of 14 inputs

I For non-zero mean inputs the mean is subtractedI And the Q matrix is computed

I QR algorithm on covariance matrix



Structure of the recursive divide-and-conquer strategyapplied to 224 components, recursion depth of four



Comparison

I Recursive KLT and JPEG2000 yields a SNR of 54.12 dB

I 1.2 min on a 1 Gigaflop/s CPU

I KLT and JPEG2000 yields a SNR of 54.13 dB

I 4 min on a 1 Gigaflop/s CPU

I DWT CDF 9/7 and JPEG2000 yields a SNR 50.67 dB

I 9 s on a 1 Gigaflop/s CPU




Image Acquisition





POT





Pairwise Orthogonal Transform - Overview

I The POT [7] is a special case of a multilevel strategy

I Minimal structure of two-component KLTs

I Low memory consumption

I Numerically stable



Transform

I In the first level: Two-component KLT for every pairI The first component will be further decorrelated in the next

levelI The most energy is in the first component

I The same approach is repeated in the next levelI Direct forwarding in an odd case



Transform (Math)

I Recall KLT: Y = QTX X ∈ RM×N

I The ED procedure in 2D is simple

2D Eigenvalue Decomposition

Σ = QΛQ−1

Σ =

[a bb d

]Q =

[p qt u

]Λ =

[λ1 00 λ2

]



Transform (Math)

2D Eigenvalue Decomposition

t = −q =b

|b|

√1

2−a− d

2s

p = u =

√1

2+a− d

2s=

√1− t2

λ1 =a+ d+ s

2λ2 =

a+ d− s2

s =√

(a− d)2 + 4b2

I Assuming b|b| ∈ {−1, 1} is a sufficient solution

I s ' 0⇒ Inputs are similar and share almost no energyI Division by zeroI Remedy: Assume identity matrix as Q



Side Information

I The KLT needs for the inverse transform side information

I For the KLT the full transformation matrix (and mean vector)

I The POT is more efficient

I For every KLT the parameter t is sufficient



Performance




Image Acquisition





POT





Definitions

Coding performance

Tradeoff between quality (SNR) and bit rate (bpppb).

Computational cost

Number of operations for a given decorrelation transform.

Component scalability

The ability to retrieve a single component (e.g. for false colorvisualization).

Memory requirements

The peak of computer memory capacity needed to apply thetransformation.



Transform




Image Acquisition





POT





Airborne Detection of Land Mines

I Demonstration of an RX anomaly detector based ondivide-and-conquer strategies

I State of the art methods: support vector methods [8], KernelRX [9]

I An RX anomaly detector [10] is based on the distance of apixel r to the overall background

I Using Mahalanobis distance: RX(r) = (r − µ)Σ−1(r − µ)



Airborne Detection of Land Mines

I Substitution Σ−1 = QΛ−1QT

I Yields RX(r) = (QT (r − µ))TΛ−1(QT (r − µ))

I QT (r − µ) is the KLT of (r − µ)

I Can be approximated by divide-and-conquer strategies



Visual Results



I. Blanes, J. Serra-sagrista, M. W. Marcellin, and J. Bartrina-rapesta,“Divide-andConquer Strategies for Hyperspectral Image Processing,” IEEESignal Processing Magazine, no. MAY, pp. 71–81, 2012.

P. Craigmile and D. Percival, “Asymptotic decorrelation of between-scale waveletcoefficients,” Information Theory, IEEE, 2005. [Online]. Available:http://ieeexplore.ieee.org/xpls/abs all.jsp?arnumber=1397939

J. E. Fowler and J. T. Rucker, “3-D wavelet-based compression of hyperspectralimages,” in Hyperspectral Data Exploitation: Theory and Applications, 2007, pp.379–407.

S. Mallat, “A theory for multiresolution signal decomposition: the waveletrepresentation,” Pattern Analysis and Machine Intelligence, vol. II, no. 7, 1989.[Online]. Available: http://ieeexplore.ieee.org/xpls/abs all.jsp?arnumber=192463

W. Yodchanan, “Lossless compression for 3-D MRI data using reversible KLT,”International Conference on Audio, Language and Image Processing, pp.1560–1564, Jul. 2008. [Online]. Available:http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4590113

Y. Wongsawat, S. Oraintara, and K. Rao, “Integer sub-optimal Karhunen-Loevetransform for multichannel lossless EEG compression,” Proc. European Signal,no. Figure 6, pp. 4–8, 2006. [Online]. Available: http://www.eurasip.org/Proceedings/Eusipco/Eusipco2006/papers/1568981760.pdf


http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1397939


http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4590113

http://www.eurasip.org/Proceedings/Eusipco/Eusipco2006/papers/1568981760.pdf

http://www.eurasip.org/Proceedings/Eusipco/Eusipco2006/papers/1568981760.pdf


I. Blanes and J. Serra-Sagrista, “Pairwise orthogonal transform for spectralimage coding,” Geoscience and Remote Sensing, vol. 49, no. 3, pp. 961–972,2011. [Online]. Available:http://ieeexplore.ieee.org/xpls/abs all.jsp?arnumber=5599290

A. Banerjee, P. Burlina, and R. Meth, “Fast hyperspectral anomaly detection viaSVDD,” in Image Processing, 2007. ICIP 2007. IEEE International Conferenceon, vol. 4. IEEE, 2007, pp. IV—-101.

H. Kwon and N. M. Nasrabadi, “Kernel RX-algorithm: a nonlinear anomalydetector for hyperspectral imagery,” Geoscience and Remote Sensing, IEEETransactions on, vol. 43, no. 2, pp. 388–397, 2005.

I. S. Reed and X. Yu, “Adaptive multiple-band CFAR detection of an opticalpattern with unknown spectral distribution,” Acoustics, Speech and SignalProcessing, IEEE Transactions on, vol. 38, no. 10, pp. 1760–1770, 1990.



divide-and-conquer strategies for hyperspectral image ... · divide-and-conquer strategies for...

Documents