sparse representation-based analysis of hyperspectral...

1

Sparse Representation-based Analysis of Hyperspectral Remote Sensing Data

Ribana Roscher

Institute of Geodesy and Geoinformation

Remote Sensing Group, University of Bonn

2

Remote Sensing Image Data

Infrared-Red-Green

Remote Sensing: Observing something from a distance without physical contact

3

Remote Sensing Image Data

altigator.com, modified

4

Hyperspectral Image Data

wavelength [μm]

reflecta

nce

5

Remote Sensing Tasks

Unmixing

Anomaly detection

Classification

Sparse representation-based analysis

6

Sparse Representation

7

Unmixing

8

Unmixing TaskProcessed satellite image

Pixel with class information (labeled)

Pixel without class information (unlabeled)

Endmember extraction

Manually or

Automatically

Reconstruction by sparse representation

Evaluation

Sub-pixel quantification

?

9

Unmixing Task

1. task: Find suitable endmembers

manually derived spectral library

archetypal dictionary

2. task: Estimate fractions (activations)

Sparse representation

10

Archetypal Analysis: SiVM

Archetypal analysis finds the extreme points (archetypes) in feature space

Efficient determination by Simplex Volume Maximization (SiVM)

Assumption: Convex hull consists of points, which maximize the volume

Christian Thurau, Kristian Kersting, Christian Bauckhage (2010): Yes We Can – Simplex Volume Maximization for Descriptive Web-Scale Matrix Factorization

11

Archetypal Analysis: SiVM

1. Randomly choose (virtual) starting point

2. Choose sample which is farthest away

3. Set this sample as first archetype

4. Choose next sample which is farthest away from all previous archetypes

Christian Thurau, Kristian Kersting, Christian Bauckhage (2010): Yes We Can – Simplex Volume Maximization for Descriptive Web-Scale Matrix Factorization

12

Data

Berlin-Urban-Gradient dataset 2009

http://dataservices.gfz-potsdam.de/enmap/showshort.php?id=escidoc:1823890&show_gcmdcitation=false

13

Data

http://dataservices.gfz-potsdam.de/enmap/showshort.php?id=escidoc:1823890&show_gcmdcitation=false

Study site: Southwest of Berlin

Hyperspectral image

Manually derived spectral library

Reference land cover information

Simulated EnMAP scene

14

Hyperspectral Data

Airborne Sensor: HyMap

111 spectral bands

Observed wavelength 450nm – 2500nm

Spatial resolution of 3.6m

Visualized as RGB-image with the wavelengths R=640nm, G=540nm and B=450nm

15

Reference Information

Reference information was manually obtained

digital orthophotos

cadastral data

4 land cover classes

Impervious surface

Vegetation

Soil & Sand

Water

16

Simulated EnMAP Data

Task: Reconstruction of fractions of simulated EnMAP data

Simulated EnMAP scene of the same area

Spatial resolution of 30m

1495 EnMAP pixels were obtained from the simulation tool, containing the fractions of the land cover classes ranging from 0 to 100%

17

Archetypal dictionaries were interpreted using reference data

High total amount of spectra in the manually derived spectral library

Archetypal Dictionary vs. Manually Derived Spectral Library

Archetypaldictionary

Manually derived library

Imp. Surface 25 39

Vegetation 12 31

Soil 2 4

Water 1 1

∑ 40 75

18

Evaluation

High number of elementary spectra in library results in a small reconstruction error

All dictionaries achieve similar and satisfactory solutions

Archetypaldictionary

Manually derived library

1.1 0.0

MA

E [

%]

Imp. Surface 12.2 16.0

Vegetation 11.0 9.2

Soil 2.5 2.1

Water 1.9 12.2

Ø 6.8 9.9

Drees, L. and Roscher, R. (2017). Archetypal analysis for sparse representation-based hyperspectral sub-pixel quantification, ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., IV-1/W1, 133-139

19

Anomaly Detection

20

Anomaly Detection Task

Dataset

• Hyperspectral images (healthy and infected plants)

• 3D pointclouds

• No label information

Dictionary elements extraction

Sparse representation

• Standard dictionary

• Mixed topographic dictionary

Evaluation

Detected symptoms

21

Biological Material

Sugar beet plants (cultivar Pauletta; KWS GmbH, Einbeck, Germany) partially infected by the plant pathogen Cercospora beticola

22

Hyperspectral 3D Data

Hyperspectral pushbroom sensor with 1600 pixel observing a spectral signature from 400nm to 1000nm

Perceptron laser triangulation scannerHyperspectral image

Inclination map

Depth map

23

Hyperspectral 3D Data

Characteristic of hyperspectral data obtained from a healthy plant

24

Hyperspectral 3D Data

• Arrow direction: Difference of two wavelengths to reference signature (black dot)

• Arrow length: Value of difference

• Color (HSV): Angle and value of difference

25

Mixed Topographic Dictionaries

Integration of prior knowledge into dictionary by means of inclination and depth groups

Learned from healthy plant pixels

Optimization with Group Orthogonal Matching Pursuit

26

Results

Blue: specular reflections

Green: leaf veins

Orange: disease symptoms

27

Results

Standard dictionary

Mixed topographic dictionary

Specular reflections Leaf veins Disease symptomsRoscher, R., Behmann, J., Mahlein, A.-K., & Plümer, L. (2016). On the Benefit of Topographic Dictionaries for Detecting Disease Symptoms on Hyperspectral 3D Plant Models, Workshop on Hyperspectral Image and Signal Processing.

28

Results

Original image Reconstruction error index with standard dictionary

Recontruction error index with mixed topographic dictionary

Roscher, R., Behmann, J., Mahlein, A.-K., & Plümer, L. (2016). On the Benefit of Topographic Dictionaries for Detecting Disease Symptoms on Hyperspectral 3D Plant Models, Workshop on Hyperspectral Image and Signal Processing.

29

Classification with Self-taught Learning

30

Classification TaskProcessed satellite images

Pixel with class information (labeled)

Pixel without class information (unlabeled)

Feature learning

Classification

Learning step

Testing step

Evaluation + Post-processing

Land use and land cover map

Land use and land cover map Posterior

probabilities

31

Self-taught Learning

Dictionary contains unlabeled data

Assumption: samples of the same class are reconstructed with a similar set of dictionary elements and similar weights

Goal: new representation is highly discriminative

training

32

Feature/Representation Learning

Learning a new data representation which is more suitable for a given task than the original data representation

Powerful feature representation Discriminative Robust Lower complexity Easier to interpret

33

What is a Good Representation?

infrared

greenclass 1

class 2

feature 2

feature 1 class 1

class 2

Original representation Discriminative representation

34

Self-taught Learning

Dictionary is fixed

Classifier is trained and tested with new representation

testing

35

Self-taught Learning for Land Cover Classification

Landsat 5 TM image near Novo Progresso (Brazil)

Ca. 8000x8000 pixel

30x30m spatial resolution

Area characterized by fire clearing

Reference information: Forest, deforestation (fire clearing) and arable land

Subarea: ~900km2

36

Dictionary Choice

~1 Mio. image patches

Archetypal analysis to decide on most important ones

37

Self-taught Learning for Land Cover Classification

Roscher, R., Römer, C., Waske, B., Plümer, L. (2015). Landcover Classification with Self-taught Learning on Archetypal Dictionaries, IGARSS, Symposium Paper Prize Award

Satellite image Land cover Maximum posterior probability (certainty)

Posterior probability: arable

38

Summary

Sparse representation is a versatile tool

Exploitation of unlabeled samples for learning

Self-taught learning

Unsupervised learning (such as anomaly detection)

More and more research goes into the direction of unsupervised pre-training in combination with supervised learning