classification of hyperspectral data using spectral...

106
Classification of Hyperspectral Data Using Spectral-Spatial Approaches Yuliya Tarabalka GIPSA-Lab, Grenoble Institute of Technology, France Department of Electrical and Computer Engineering, University of Iceland, Iceland Thesis committee Melba CRAWFORD President Jean-Yves TOURNERET Opponent Xiuping JIA Opponent Antonio PLAZA Opponent Paolo GAMBA Opponent Johannes R. SVEINSSON Examinator Jón Atli BENEDIKTSSON Thesis advisor Jocelyn CHANUSSOT Thesis advisor June 14, 2010

Upload: others

Post on 08-Aug-2020

5 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

Classification of Hyperspectral Data UsingSpectral-Spatial Approaches

Yuliya Tarabalka

GIPSA-Lab, Grenoble Institute of Technology, FranceDepartment of Electrical and Computer Engineering, University of Iceland, Iceland

Thesis committee

Melba CRAWFORD PresidentJean-Yves TOURNERET OpponentXiuping JIA OpponentAntonio PLAZA OpponentPaolo GAMBA OpponentJohannes R. SVEINSSON ExaminatorJón Atli BENEDIKTSSON Thesis advisorJocelyn CHANUSSOT Thesis advisor

June 14, 2010

Page 2: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Hyperspectral image

Every pixel contains a detailed spectrum(>100 spectral bands)

+ More information per pixel

− Dimensionality increases

⇓Advanced algorithms are required!

λ pixel vector xi

xi xix1 x2 x3x2i

xn

lines

samples

Tens or hundreds of bands wavelength λ

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 2

Page 3: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Supervised classification problem

ROSIS imageSpatial resolution: 1.3m/pixSpectral resolution: 103 bands

Ground-truth data Task

Nine classes: alphalt, meadows, gravel, trees, metal sheets, bare soil,bitumen, bricks, shadows

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 3

Page 4: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Supervised classification problem

AVIRIS imageSpatial resolution: 20m/pix

Spectral resolution: 200 bands

Ground-truthdata

Task

16 classes: corn-no till, corn-min till, corn, soybeans-no till, soybeans-mintill, soybeans-clean till, alfalfa, grass/pasture, grass/trees,grass/pasture-mowed, hay-windrowed, oats, wheat, woods,

bldg-grass-tree-drives, stone-steel towers

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 4

Page 5: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Classification approaches

Only spectral information

Pixelwise approach

Spectrum of each pixel is analyzedVariety of methods

Support Vector Machines (SVM) → goodclassification results [Camps-Valls05]

alphaltmeadowsgraveltrees

metal sheetsbare soilbitumenbricks

shadows

Overall accuracy (OA) = 81.01%

Average accuracy (AA) = 88.25%

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 5

Page 6: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Classification approaches

Only spectral information

Pixelwise approach

Spectrum of each pixel is analyzedVariety of methods

Support Vector Machines (SVM) → goodclassification results [Camps-Valls05]

Spectral + spatial information

Info about spatial structures is includedSince neighboring pixels are related

How to define spatial structures?

How to combine spectral and spatialinformation?

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 6

Page 7: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

State of the art: Approaches for extracting spatial info

1 Closest fixed neighborhoodsMarkov Random Field [Pony00, Jackson02,Farag05]Contextual features [Camps-Valls06]+ Simplicity− Imprecision at the border of regions

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 7

Page 8: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

State of the art: Approaches for extracting spatial info

1 Closest fixed neighborhoodsMarkov Random Field [Pony00, Jackson02,Farag05]Contextual features [Camps-Valls06]+ Simplicity− Imprecision at the border of regions

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 7

Page 9: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

State of the art: Approaches for extracting spatial info

1 Closest fixed neighborhoodsMarkov Random Field [Pony00, Jackson02,Farag05]Contextual features [Camps-Valls06]+ Simplicity− Imprecision at the border of regions

2 Morphological and area filteringMorphological profiles [Pesaresi01, Dell’Acqua04, Benediktsson05]Self-complementary area filtering [Fauvel07]+ Neighborhoods are adapted to the structures− Neighborhoods are scale dependent ⇒ imprecision in the spatial info

Closing − Original − Opening

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 7

Page 10: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

State of the art: Approaches for extracting spatial info

1 Closest fixed neighborhoodsMarkov Random Field [Pony00, Jackson02,Farag05]Contextual features [Camps-Valls06]+ Simplicity− Imprecision at the border of regions

2 Morphological and area filteringMorphological profiles [Pesaresi01, Dell’Acqua04, Benediktsson05]Self-complementary area filtering [Fauvel07]+ Neighborhoods are adapted to the structures− Neighborhoods are scale dependent ⇒ imprecision in the spatial info

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 7

Page 11: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

State of the art: Approaches for extracting spatial info

3 Segmentation map = exhaustive partitioning of the image intohomogeneous regions

Extraction and Classification of Homogeneous Objects [Kettig76]+ Has become a standard spectral-spatial classification technique− Statistical approach ⇒ not well adapted for hyperspectral data

Recent works:

Multiscale segmentation + SVM classification [Linden07, Huang09]− Computationally demanding

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 8

Page 12: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

State of the art: Approaches for extracting spatial info

3 Segmentation map = exhaustive partitioning of the image intohomogeneous regions

Extraction and Classification of Homogeneous Objects [Kettig76]+ Has become a standard spectral-spatial classification technique− Statistical approach ⇒ not well adapted for hyperspectral data

Recent works:

Multiscale segmentation + SVM classification [Linden07, Huang09]− Computationally demanding

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 8

Page 13: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

State of the art: Approaches for extracting spatial info

3 Segmentation map = exhaustive partitioning of the image intohomogeneous regions

Extraction and Classification of Homogeneous Objects [Kettig76]+ Has become a standard spectral-spatial classification technique− Statistical approach ⇒ not well adapted for hyperspectral data

Recent works:

Multiscale segmentation + SVM classification [Linden07, Huang09]− Computationally demanding

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 8

Page 14: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Thesis objective: combine spectral and spatial info

Proposed spectral-spatial classification approaches

Using closest fixed neighborhoods

SVM and MRF-based classification

Using adaptive neighborhoods

Using spatial neighborhoods derived from unsupervised

segmentation

Segmentation Classification Using marker-based region growing segmentation

Marker selection ClassificationWatershed Partitional

clustering Hierarchical

segmentation Object-based

Composite kernels

Segmentation + pixelwise

classification

Using ML discriminant

values

Using probabilistic

SVM

Multiple classifier approach

Marker-controlled watershed

Construction of a Minimum

Spanning Forest

Strategy 1

Strategy 2

Strategy 3

High-performance computing

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 9

Page 15: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Thesis objective: combine spectral and spatial info

Proposed spectral-spatial classification approaches

Construction of a Minimum

Spanning Forest

Using closest fixed neighborhoods

SVM and MRF-based classification

Using adaptive neighborhoods

Using spatial neighborhoods derived from unsupervised

segmentation

Segmentation Classification Using marker-based region growing segmentation

Marker selection ClassificationWatershed Partitional

clustering Hierarchical

segmentation Object-based

Composite kernels

Segmentation + pixelwise

classification

Using ML discriminant

values

Using probabilistic

SVM

Multiple classifier approach

Strategy 1

Strategy 2

Marker-controlled watershed

Strategy 3

High-performance computing

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 9

Page 16: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Outline

1 Introduction

2 Classification using segmentation-derived adaptive neighborhoodsSegmentationSpectral-spatial classificationConcluding discussion

3 Segmentation and classification using automatically selected markersMarker selection

Using probabilistic SVMMultiple classifier approach

MSF-based marker-controlled region growingConcluding discussion

4 Conclusions and perspectives

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 10

Page 17: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

SegmentationSpectral-spatial classificationConcluding discussion

Objective

1 Automatically segment a hyperspectral image into homogeneousregions

2 Spectral info + segmentation map → classify image

+ → Classificationmap

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 11

Page 18: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

SegmentationSpectral-spatial classificationConcluding discussion

Outline

1 Introduction

2 Classification using segmentation-derived adaptive neighborhoodsSegmentationSpectral-spatial classificationConcluding discussion

3 Segmentation and classification using automatically selected markersMarker selection

Using probabilistic SVMMultiple classifier approach

MSF-based marker-controlled region growingConcluding discussion

4 Conclusions and perspectives

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 12

Page 19: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

SegmentationSpectral-spatial classificationConcluding discussion

1. Watershed segmentation

gradient⇒

Region growing method:

Minimum of a gradient = core of a homogeneous region

1 region = set of pixels connected to 1 local minimum of the gradient

Watershed lines = edges between adjacent regions

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 13

Page 20: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

SegmentationSpectral-spatial classificationConcluding discussion

1. Watershed segmentation

Originalimage

Y. Tarabalka, J. Chanussot, and J. A. Benediktsson, “Segmentation and classification of hyperspectralimages using watershed transformation,” Pattern Recognition, vol. 43, no. 7, pp. 2367-2379, July2010.

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 14

Page 21: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

SegmentationSpectral-spatial classificationConcluding discussion

1. Watershed segmentation

Originalimage

Robust ColorMorpho Gradient

Y. Tarabalka, J. Chanussot, and J. A. Benediktsson, “Segmentation and classification of hyperspectralimages using watershed transformation,” Pattern Recognition, vol. 43, no. 7, pp. 2367-2379, July2010.

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 14

Page 22: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

SegmentationSpectral-spatial classificationConcluding discussion

1. Watershed segmentation

Originalimage

Robust ColorMorpho Gradient

Watershed11802 regions

Y. Tarabalka, J. Chanussot, and J. A. Benediktsson, “Segmentation and classification of hyperspectralimages using watershed transformation,” Pattern Recognition, vol. 43, no. 7, pp. 2367-2379, July2010.

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 14

Page 23: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

SegmentationSpectral-spatial classificationConcluding discussion

1. Watershed segmentation

Originalimage

Robust ColorMorpho Gradient

Watershed11802 regions

Edges → toadjacent regions

Y. Tarabalka, J. Chanussot, and J. A. Benediktsson, “Segmentation and classification of hyperspectralimages using watershed transformation,” Pattern Recognition, vol. 43, no. 7, pp. 2367-2379, July2010.

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 14

Page 24: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

SegmentationSpectral-spatial classificationConcluding discussion

2. Partitional clustering (EM)

1 Clusteringpixels are grouped into C clustersin each cluster → pixels drawn froma Gaussian distributiondistribution parameters → EM algorithm

2 Labeling of connected components

10 clusters⇒

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 15

Page 25: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

SegmentationSpectral-spatial classificationConcluding discussion

2. Partitional clustering (EM)

1 Clusteringpixels are grouped into C clustersin each cluster → pixels drawn froma Gaussian distributiondistribution parameters → EM algorithm

2 Labeling of connected components

10 clusters⇒

21450regions

same cluster,but differentregions!

XXXXXXXz

HHHHHj

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 15

Page 26: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

SegmentationSpectral-spatial classificationConcluding discussion

3. Hierarchical SEGmentation (HSEG,[Tilton98])

Region growing + Spectral Clustering

Dissimilarity criterion (DC):Spectral Angle Mapper (SAM)between the region mean vectors ui and uj

SAM(ui , uj) = arccos(ui · uj

‖ui‖2‖uj‖2)

1 Each pixel - one region2 Find DCmin between adjacent regions3 Merge adjacent regions with DC = DCmin4 Merge non-adjacent regions withDC ≤ DCmin ·SpectralClusterWeight

5 If not converge, go to 2

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 16

Page 27: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

SegmentationSpectral-spatial classificationConcluding discussion

3. Hierarchical SEGmentation (HSEG,[Tilton98])

Region growing + Spectral Clustering

Dissimilarity criterion (DC):Spectral Angle Mapper (SAM)between the region mean vectors ui and uj

SAM(ui , uj) = arccos(ui · uj

‖ui‖2‖uj‖2)

1 Each pixel - one region2 Find DCmin between adjacent regions3 Merge adjacent regions with DC = DCmin4 Merge non-adjacent regions withDC ≤ DCmin ·SpectralClusterWeight

5 If not converge, go to 2

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 16

Page 28: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

SegmentationSpectral-spatial classificationConcluding discussion

3. Hierarchical SEGmentation (HSEG,[Tilton98])

Region growing + Spectral Clustering

Dissimilarity criterion (DC):Spectral Angle Mapper (SAM)between the region mean vectors ui and uj

SAM(ui , uj) = arccos(ui · uj

‖ui‖2‖uj‖2)

1 Each pixel - one region2 Find DCmin between adjacent regions3 Merge adjacent regions with DC = DCmin4 Merge non-adjacent regions withDC ≤ DCmin ·SpectralClusterWeight

5 If not converge, go to 2

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 16

Page 29: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

SegmentationSpectral-spatial classificationConcluding discussion

3. Hierarchical SEGmentation (HSEG,[Tilton98])

Region growing + Spectral Clustering

Dissimilarity criterion (DC):Spectral Angle Mapper (SAM)between the region mean vectors ui and uj

SAM(ui , uj) = arccos(ui · uj

‖ui‖2‖uj‖2)

1 Each pixel - one region2 Find DCmin between adjacent regions3 Merge adjacent regions with DC = DCmin4 Merge non-adjacent regions withDC ≤ DCmin ·SpectralClusterWeight

5 If not converge, go to 2

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 16

Page 30: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

SegmentationSpectral-spatial classificationConcluding discussion

3. Hierarchical SEGmentation (HSEG,[Tilton98])

Region growing + Spectral Clustering

Dissimilarity criterion (DC):Spectral Angle Mapper (SAM)between the region mean vectors ui and uj

SAM(ui , uj) = arccos(ui · uj

‖ui‖2‖uj‖2)

1 Each pixel - one region2 Find DCmin between adjacent regions3 Merge adjacent regions with DC = DCmin4 Merge non-adjacent regions withDC ≤ DCmin ·SpectralClusterWeight

5 If not converge, go to 2

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 16

Page 31: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

SegmentationSpectral-spatial classificationConcluding discussion

3. Hierarchical SEGmentation (HSEG,[Tilton98])

Region growing + Spectral Clustering

Dissimilarity criterion (DC):Spectral Angle Mapper (SAM)between the region mean vectors ui and uj

SAM(ui , uj) = arccos(ui · uj

‖ui‖2‖uj‖2)

1 Each pixel - one region2 Find DCmin between adjacent regions3 Merge adjacent regions with DC = DCmin4 Merge non-adjacent regions withDC ≤ DCmin ·SpectralClusterWeight

5 If not converge, go to 2

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 16

Page 32: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

SegmentationSpectral-spatial classificationConcluding discussion

3. Hierarchical SEGmentation (HSEG,[Tilton98])

Region growing + Spectral Clustering

Dissimilarity criterion (DC):Spectral Angle Mapper (SAM)between the region mean vectors ui and uj

SAM(ui , uj) = arccos(ui · uj

‖ui‖2‖uj‖2)

1 Each pixel - one region2 Find DCmin between adjacent regions3 Merge adjacent regions with DC = DCmin4 Merge non-adjacent regions withDC ≤ DCmin ·SpectralClusterWeight

5 If not converge, go to 2

7575 regions

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 16

Page 33: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

SegmentationSpectral-spatial classificationConcluding discussion

Outline

1 Introduction

2 Classification using segmentation-derived adaptive neighborhoodsSegmentationSpectral-spatial classificationConcluding discussion

3 Segmentation and classification using automatically selected markersMarker selection

Using probabilistic SVMMultiple classifier approach

MSF-based marker-controlled region growingConcluding discussion

4 Conclusions and perspectives

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 17

Page 34: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

SegmentationSpectral-spatial classificationConcluding discussion

Spectral-spatial classification: majority vote

Hyperspectral image (B bands)

Pixelwise classification Segmentation

Spectral-spatial classification

(majority voting)

Final classification map

Pixelwise classification map (dark blue, white

and light grey classes)

Segmentation map

(3 spatial regions)

Result of spectral-spatial classification (classification

map after majority vote)

Combination of unsupervised

segmentation and pixelwise

classification results (majority vote within

3 spatial regions)

1 1 1 1 1 1 1 1 1 2 2 2 1 1 2 2 2 2 1 1 2 2 2 2 1 1 3 3 3 2 1 3 3 3 3 3 3 3 3 3 3 3

Y. Tarabalka, J. A. Benediktsson, and J. Chanussot, “Spectral-spatial classification of hyperspectralimagery based on partitional clustering techniques,” IEEE Trans. on Geoscience and Remote Sensing,vol. 47, no. 8, pp. 2973-2987, Aug. 2009.

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 18

Page 35: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

SegmentationSpectral-spatial classificationConcluding discussion

Spectral-spatial classification

Originalimage

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 19

Page 36: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

SegmentationSpectral-spatial classificationConcluding discussion

Spectral-spatial classification

Originalimage

SVMclassification

OA = 81.01%AA = 88.25%

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 19

Page 37: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

SegmentationSpectral-spatial classificationConcluding discussion

Spectral-spatial classification

SVMclassification

OA = 81.01%AA = 88.25%

SVM +Watershed

OA = 85.42%AA = 91.31%

SVM +Partit.clustering

OA = 94.00%AA = 93.13%

SVM +HSEG

OA = 93.85%AA = 97.07%

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 19

Page 38: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

SegmentationSpectral-spatial classificationConcluding discussion

Classification accuracies (%)

SVM +Watersh. +Part.Cl. +HSEG EMP1 ECHO

Overall Acc. 81.01 85.42 94.00 93.85 85.22 87.58Average Acc. 88.25 91.31 93.13 97.07 90.76 92.16Kappa Coef.κ 75.86 81.30 91.93 91.89 80.86 83.90Asphalt 84.93 93.64 90.10 94.77 95.36 87.98Meadows 70.79 75.09 95.99 89.32 80.33 81.64Gravel 67.16 66.12 82.26 96.14 87.61 76.91Trees 97.77 98.56 85.54 98.08 98.37 99.31Metal sheets 99.46 99.91 100 99.82 99.48 99.91Bare soil 92.83 97.35 96.72 99.76 63.72 93.96Bitumen 90.42 96.23 91.85 100 98.87 92.97Bricks 92.78 97.92 98.34 99.29 95.41 97.35Shadows 98.11 96.98 97.36 96.48 97.68 99.37

1A. Plaza et al., "Recent advances in techniques for hyperspectral image processing," RemoteSensing of Environment, vol. 113, Suppl. 1, 2009.

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 20

Page 39: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

SegmentationSpectral-spatial classificationConcluding discussion

Outline

1 Introduction

2 Classification using segmentation-derived adaptive neighborhoodsSegmentationSpectral-spatial classificationConcluding discussion

3 Segmentation and classification using automatically selected markersMarker selection

Using probabilistic SVMMultiple classifier approach

MSF-based marker-controlled region growingConcluding discussion

4 Conclusions and perspectives

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 21

Page 40: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

SegmentationSpectral-spatial classificationConcluding discussion

1 Spectral-spatial classification improves accuracies when compared topixel-wise classification

2 Several segmentation techniques are investigated3 The HSEG segmentation map leads to the best classification4 Obtained classification accuracies > all previous results

However...

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 22

Page 41: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

SegmentationSpectral-spatial classificationConcluding discussion

Unsupervised segmentation

Unsupervised segmentation = exhaustive partitioning into homogeneousregionsHow to define a measure of homogeneity?

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 23

Page 42: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

SegmentationSpectral-spatial classificationConcluding discussion

Unsupervised segmentation

Unsupervised segmentation = exhaustive partitioning into homogeneousregionsHow to define a measure of homogeneity?

Hierarchical SEGmentation (HSEG, [Tilton98])

Image

2381reg. 1389reg. 280reg.

Oversegmentation! Undersegmentation?@@I ��� 6

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 23

Page 43: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

SegmentationSpectral-spatial classificationConcluding discussion

Unsupervised segmentation

Unsupervised segmentation = exhaustive partitioning into homogeneousregionsHow to define a measure of homogeneity?

Hierarchical SEGmentation (HSEG, [Tilton98])

Image

2381reg. 1389reg. 280reg.

Oversegmentation Undersegmentation?⇓

Preferred in previous works → not to lose objects

���

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 23

Page 44: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

SegmentationSpectral-spatial classificationConcluding discussion

Watershed segmentation

Originalimage

Robust ColorMorpho Gradient

Watershed1277 regions

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 24

Page 45: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

SegmentationSpectral-spatial classificationConcluding discussion

Watershed segmentation

Originalimage

Robust ColorMorpho Gradient

Watershed1277 regions

�������3

Severe oversegmentation!

Every local minimumof the gradient

↓one region

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 24

Page 46: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

SegmentationSpectral-spatial classificationConcluding discussion

Marker-controlled segmentation

Reduce oversegmentation ⇐ incorporate an additional knowledge intosegmentation

We propose to use markers

Determine markers foreach region of interest

Segmentan image

One regionin a segmentation map

⇑One marker

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 25

Page 47: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Objective

Determine markers automatically ← using probabilisticclassification results

Marker-controlled region growing→ segment and classify ahyperspectral image

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 26

Page 48: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Outline

1 Introduction

2 Classification using segmentation-derived adaptive neighborhoodsSegmentationSpectral-spatial classificationConcluding discussion

3 Segmentation and classification using automatically selected markersMarker selection

Using probabilistic SVMMultiple classifier approach

MSF-based marker-controlled region growingConcluding discussion

4 Conclusions and perspectives

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 27

Page 49: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Marker selection using probabilistic SVM

B-band hyperspectral imageX = {xj ∈ RB, j = 1, 2, ..., n}B ∼ 100

Hyperspectral image (B bands)

Probabilistic pixelwise

classification classification map

Selection of the most probability map

map of Marker-controlled

region growing markers reliable classified

pixels

Segmentation map + classification map

Y. Tarabalka, J. Chanussot, and J. A. Benediktsson, “Segmentation and classification of hyperspectralimages using Minimum Spanning Forest grown from automatically selected markers,” IEEE Trans. onSystems, Man, and Cybernetics, Part B: Cybernetics, 2010, DOI 10.1109/TSMCB.2009.2037132.

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 28

Page 50: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Marker selection using probabilistic SVM

SVM classifier* → well suited forhyperspectral images

Output:

classification map probability map

-

Hyperspectral image (B bands)

Probabilistic pixelwise

classification classification map

Selection of the most probability map

map of Marker-controlled

region growing markers reliable classified

pixels

Segmentation map + classification map

probability estimate for each pixel tobelong to the assigned class

*C. Chang and C. Lin, "LIBSVM: A library for Support Vector Machines," Software available at

http://www.csie.ntu.edu.tw/∼cjlin/libsvm, 2001.

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 29

Page 51: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Marker selection using probabilistic SVM

Analysis of classification and probability maps:classification map probability map

1 Perform connected components labeling ofthe classification map

2 Analyze each connected component:

If it is large (> 20 pixels) → use P%(5%) of its pixels with the highestprobabilities as a markerIf it is small → its pixels withprobabilities > T% (90%)are used as a marker

Hyperspectral image (B bands)

Probabilistic pixelwise

classification classification map

Selection of the most probability map

map of Marker-controlled

region growing markers reliable classified

pixels

Segmentation map + classification map

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 30

Page 52: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Marker selection using probabilistic SVM

Analysis of classification and probability maps:classification map probability map

1 Perform connected components labeling ofthe classification map

2 Analyze each connected component:

If it is large (> 20 pixels) → use P%(5%) of its pixels with the highestprobabilities as a markerIf it is small → its pixels withprobabilities > T% (90%)are used as a marker

Hyperspectral image (B bands)

Probabilistic pixelwise

classification classification map

Selection of the most probability map

map of Marker-controlled

region growing markers reliable classified

pixels

Segmentation map + classification map HH

HY

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 30

Page 53: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Marker selection using probabilistic SVM

Analysis of classification and probability maps:classification map probability map

1 Perform connected components labeling ofthe classification map

2 Analyze each connected component:

If it is large (> 20 pixels) → use P%(5%) of its pixels with the highestprobabilities as a markerIf it is small → its pixels withprobabilities > T% (90%)are used as a marker

Hyperspectral image (B bands)

Probabilistic pixelwise

classification classification map

Selection of the most probability map

map of Marker-controlled

region growing markers reliable classified

pixels

Segmentation map + classification map

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 30

Page 54: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Marker selection using probabilistic SVM

Analysis of classification and probability maps:classification map probability map

1 Perform connected components labeling ofthe classification map

2 Analyze each connected component:

If it is large (> 20 pixels) → use P%(5%) of its pixels with the highestprobabilities as a markerIf it is small → its pixels withprobabilities > T% (90%)are used as a marker

Hyperspectral image (B bands)

Probabilistic pixelwise

classification classification map

Selection of the most probability map

map of Marker-controlled

region growing markers reliable classified

pixels

Segmentation map + classification map

Must contain a marker!

HHHHH

HHHHHH

HHHHHj

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 30

Page 55: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Marker selection using probabilistic SVM

Analysis of classification and probability maps:classification map probability map

1 Perform connected components labeling ofthe classification map

2 Analyze each connected component:

If it is large (> 20 pixels) → use P%(5%) of its pixels with the highestprobabilities as a markerIf it is small → its pixels withprobabilities > T% (90%)are used as a marker

Hyperspectral image (B bands)

Probabilistic pixelwise

classification classification map

Selection of the most probability map

map of Marker-controlled

region growing markers reliable classified

pixels

Segmentation map + classification map

Must contain a marker!

HHHHH

HHHHHH

HHHHHj

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 30

Page 56: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Marker selection using probabilistic SVM

Analysis of classification and probability maps:classification map probability map

1 Perform connected components labeling ofthe classification map

2 Analyze each connected component:

If it is large (> 20 pixels) → use P%(5%) of its pixels with the highestprobabilities as a markerIf it is small → its pixels withprobabilities > T% (90%)are used as a marker

Hyperspectral image (B bands)

Probabilistic pixelwise

classification classification map

Selection of the most probability map

map of Marker-controlled

region growing markers reliable classified

pixels

Segmentation map + classification map

Has a marker only if it isvery reliable

HHHH

HHHHHH

HHHHHHj

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 30

Page 57: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Marker selection using probabilistic SVM

Analysis of classification and probability maps:classification map probability map

Each connected component → 1 or 0 marker(2250 regions → 107 markers)

Marker is not necessarily a connected set ofpixels

Each marker has a class label

Hyperspectral image (B bands)

Probabilistic pixelwise

classification classification map

Selection of the most probability map

map of Marker-controlled

region growing markers reliable classified

pixels

Segmentation map + classification map

map of 107 markers

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 30

Page 58: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Multiple classifier approach for marker selection

Previous method: strong dependence on the performances of theselected probabilistic classifier

Objective: mitigate this dependence→ using multiple classifiers

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 31

Page 59: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Multiple classifier approach for marker selection

Previous method: strong dependence on the performances of theselected probabilistic classifier

Objective: mitigate this dependence→ using multiple classifiers

Multiple classifier marker selection approach

1 Classify an image by severalindependent classifiers

2 Pixels assigned by all theclassifiers to the same class

⇓Map of markers

classifiers

Hyperspectral image

Map ofmarkers

Marker selection

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 31

Page 60: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Multiple spectral-spatial classifier marker selection

Take into account spatial context

Pixelwise classification

Hyperspectral image (B bands)

Majority voting within segmentation

regions

Watershed segmentation

Marker selection Map of markers Segmentation by EM for Gaussian mixture resolving

Hierarchical SEGmentation

Majority voting within segmentation

regions

Majority voting within segmentation

regions

classification map

classification map

classification map

Y. Tarabalka, J. A. Benediktsson, J. Chanussot, and J. C. Tilton, “Multiple spectral-spatialclassification approach for hyperspectral data,” submitted to IEEE Trans. on Geoscience andRemote Sensing (under review).

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 32

Page 61: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Multiple spectral-spatial classifier marker selection

Take into account spatial context

Pixelwise classification

Hyperspectral image (B bands)

Majority voting within segmentation

regions

Watershed segmentation

Marker selection Map of markers Segmentation by EM for Gaussian mixture resolving

Hierarchical SEGmentation

Majority voting within segmentation

regions

Majority voting within segmentation

regions

classification map

classification map

classification map

Y. Tarabalka, J. A. Benediktsson, J. Chanussot, and J. C. Tilton, “Multiple spectral-spatialclassification approach for hyperspectral data,” submitted to IEEE Trans. on Geoscience andRemote Sensing (under review).

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 32

Page 62: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Multiple spectral-spatial classifier marker selection

Take into account spatial context

Pixelwise classification

Hyperspectral image (B bands)

Majority voting within segmentation

regions

Watershed segmentation

Marker selection Map of markers Segmentation by EM for Gaussian mixture resolving

Hierarchical SEGmentation

Majority voting within segmentation

regions

Majority voting within segmentation

regions

classification map

classification map

classification map

Y. Tarabalka, J. A. Benediktsson, J. Chanussot, and J. C. Tilton, “Multiple spectral-spatialclassification approach for hyperspectral data,” submitted to IEEE Trans. on Geoscience andRemote Sensing (under review).

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 32

Page 63: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Multiple spectral-spatial classifier marker selection

Take into account spatial context

Pixelwise classification

Hyperspectral image (B bands)

Majority voting within segmentation

regions

Watershed segmentation

Marker selection Map of markers Segmentation by EM for Gaussian mixture resolving

Hierarchical SEGmentation

Majority voting within segmentation

regions

Majority voting within segmentation

regions

classification map

classification map

classification map

Y. Tarabalka, J. A. Benediktsson, J. Chanussot, and J. C. Tilton, “Multiple spectral-spatialclassification approach for hyperspectral data,” submitted to IEEE Trans. on Geoscience andRemote Sensing (under review).

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 32

Page 64: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Multiple spectral-spatial classifier marker selection

Take into account spatial context

Pixelwise classification

Hyperspectral image (B bands)

Majority voting within segmentation

regions

Watershed segmentation

Marker selection Map of markers Segmentation by EM for Gaussian mixture resolving

Hierarchical SEGmentation

Majority voting within segmentation

regions

Majority voting within segmentation

regions

classification map

classification map

classification map

Y. Tarabalka, J. A. Benediktsson, J. Chanussot, and J. C. Tilton, “Multiple spectral-spatialclassification approach for hyperspectral data,” submitted to IEEE Trans. on Geoscience andRemote Sensing (under review).

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 32

Page 65: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Outline

1 Introduction

2 Classification using segmentation-derived adaptive neighborhoodsSegmentationSpectral-spatial classificationConcluding discussion

3 Segmentation and classification using automatically selected markersMarker selection

Using probabilistic SVMMultiple classifier approach

MSF-based marker-controlled region growingConcluding discussion

4 Conclusions and perspectives

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 33

Page 66: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Construction of a Minimum Spanning Forest (MSF)

Hyperspectral image (B bands)

Classification

map of Construction of minimum spanning

forest

markers Marker selection

Majority voting within connected

components

Segmentation map + classification map

Y. Tarabalka, J. Chanussot, and J. A. Benediktsson, “Segmentation and classification of hyperspectralimages using Minimum Spanning Forest grown from automatically selected markers,” IEEE Trans. onSystems, Man, and Cybernetics, Part B: Cybernetics, 2010, DOI 10.1109/TSMCB.2009.2037132.

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 34

Page 67: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Construction of a Minimum Spanning Forest (MSF)

B bands x 9x 8x 7

x 6x 5x x1 x2 x3

4

map of markers

1 1 0 0 0 0 0 0 2

1) Map an image onto a graphWeight wi ,j indicates the degree of dissimilarity between pixels xi and xj .Spectral Angle Mapper (SAM) distance can be used:

wi ,j = SAM(xi , xj) = arccos

( ∑Bb=1 xibxjb

[∑B

b=1 x2ib]1/2[∑B

b=1 x2jb]1/2

)

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 35

Page 68: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Construction of a Minimum Spanning Forest (MSF)

B bands x 9x 8x 7

x 6x 5x x1 x2 x3

4

map of markers

1 1 0 0 0 0 0 0 2 image graph

1) Map an image onto a graphWeight wi ,j indicates the degree of dissimilarity between pixels xi and xj .Spectral Angle Mapper (SAM) distance can be used:

wi ,j = SAM(xi , xj) = arccos

( ∑Bb=1 xibxjb

[∑B

b=1 x2ib]1/2[∑B

b=1 x2jb]1/2

)

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 35

Page 69: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Construction of a Minimum Spanning Forest (MSF)

B bands x 9x 8x 7

x 6x 5x 4

x3 x2 x1

map of markers

1 1 0 0 0 0 0 0 2

w12

image graph

1) Map an image onto a graphWeight wi ,j indicates the degree of dissimilarity between pixels xi and xj .Spectral Angle Mapper (SAM) distance can be used:

wi ,j = SAM(xi , xj) = arccos

( ∑Bb=1 xibxjb

[∑B

b=1 x2ib]1/2[∑B

b=1 x2jb]1/2

)

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 35

Page 70: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Construction of a Minimum Spanning Forest (MSF)

B bands x 9x 8x 7

x 6x 5x 4

x3 x2 x1

map of markers

1 1 0 0 0 0 0 0 2 image graph

0 8 14 2

8 5 15

9 5164

1211 12 314 10

511

8

1) Map an image onto a graphWeight wi ,j indicates the degree of dissimilarity between pixels xi and xj .Spectral Angle Mapper (SAM) distance can be used:

wi ,j = SAM(xi , xj) = arccos

( ∑Bb=1 xibxjb

[∑B

b=1 x2ib]1/2[∑B

b=1 x2jb]1/2

)

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 35

Page 71: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Construction of a Minimum Spanning Forest (MSF)

1 1

2

B bands x 9x 8x 7

x 6x 5x 4

x3 x2 x1

map of markers

1 1 0 0 0 0 0 0 2

80 0 14 2

8 5 15

image graph

0 0 9 5

164 0 1211

12 314 10 5

11 80 0

1) Map an image onto a graphWeight wi ,j indicates the degree of dissimilarity between pixels xi and xj .Spectral Angle Mapper (SAM) distance can be used:

wi ,j = SAM(xi , xj) = arccos

( ∑Bb=1 xibxjb

[∑B

b=1 x2ib]1/2[∑B

b=1 x2jb]1/2

)

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 35

Page 72: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Construction of a Minimum Spanning Forest (MSF)

80

1 1

2

02 14

5 8 15 9

image graph

5164 0 0 0

11 12 14 12 3 10 11

8 5 0 0

Given a graph G, a MSF F ∗ rooted on vertices {r1, ..., rm} is:a non-connected graph without cycles

contains all the vertices of G

consists of connected subgraphs, each subgraph (tree) contains (isrooted on) one root risum of the edges weights of F ∗ is minimal

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 36

Page 73: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Construction of a Minimum Spanning Forest (MSF)

0

1 1

2

modified graph

0

0 0

0

0

0 0

3

8

164

8 5

5

14

5

2

9

14

10 1111 12

8

12

15

r2

r1 0

0

2) Add m extra vertices ri , i = 1, ..., m corresponding to m markers

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 37

Page 74: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Construction of a Minimum Spanning Forest (MSF)

0

1 1

2

0

0 0

0

0

0 0

3

8

164

8 5

5

14

5

2

9

14

10 1111 12

8

12

15

r2

r1 0

0

3) Construct a MSF F ∗ = (V ∗, E∗)

Initialization: V ∗ = {r1, r2, ..., rm} (roots are in the forest)1 Choose edge of the modified graph ei j with minimal weight such thati ∈ V ∗ and j /∈ V ∗

2 V ∗ = V ∗ ∪ {j}, E∗ = E∗ ∪ {ei ,j}3 If V ∗ 6= V , go to 1

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 37

Page 75: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Construction of a Minimum Spanning Forest (MSF)

0

1 1

2

0

0 0

0

0

0 0

3

8

164

8 5

5

14

5

2

9

14

10 1111 12

8

12

15

r2

r1 0

0

3) Construct a MSF F ∗ = (V ∗, E∗)

Initialization: V ∗ = {r1, r2, ..., rm} (roots are in the forest)1 Choose edge of the modified graph ei j with minimal weight such thati ∈ V ∗ and j /∈ V ∗

2 V ∗ = V ∗ ∪ {j}, E∗ = E∗ ∪ {ei ,j}3 If V ∗ 6= V , go to 1

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 37

Page 76: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Construction of a Minimum Spanning Forest (MSF)

0

1 1

2

0

0 0

0

0

0 0

3

8

164

8 5

5

14

5

2

9

14

10 1111 12

8

12

15

r2

r1 0

0

3) Construct a MSF F ∗ = (V ∗, E∗)

Initialization: V ∗ = {r1, r2, ..., rm} (roots are in the forest)1 Choose edge of the modified graph ei j with minimal weight such thati ∈ V ∗ and j /∈ V ∗

2 V ∗ = V ∗ ∪ {j}, E∗ = E∗ ∪ {ei ,j}3 If V ∗ 6= V , go to 1

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 37

Page 77: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Construction of a Minimum Spanning Forest (MSF)

0

1 1

2

0

0 0

0

0

0 0

3

8

164

8 5

5

14

5

2

9

14

10 1111 12

8

12

15

r2

r1 0

0

3) Construct a MSF F ∗ = (V ∗, E∗)

Initialization: V ∗ = {r1, r2, ..., rm} (roots are in the forest)1 Choose edge of the modified graph ei j with minimal weight such thati ∈ V ∗ and j /∈ V ∗

2 V ∗ = V ∗ ∪ {j}, E∗ = E∗ ∪ {ei ,j}3 If V ∗ 6= V , go to 1

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 37

Page 78: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Construction of a Minimum Spanning Forest (MSF)

0

1 1

2

0

0 0

0

0

0 0

3

8

164

8 5

5

14

5

2

9

14

10 1111 12

8

12

15

r2

r1 0

0

3) Construct a MSF F ∗ = (V ∗, E∗)

Initialization: V ∗ = {r1, r2, ..., rm} (roots are in the forest)1 Choose edge of the modified graph ei j with minimal weight such thati ∈ V ∗ and j /∈ V ∗

2 V ∗ = V ∗ ∪ {j}, E∗ = E∗ ∪ {ei ,j}3 If V ∗ 6= V , go to 1

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 37

Page 79: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Construction of a Minimum Spanning Forest (MSF)

0

1 1

2

0

0 1

0

0

0 0

3

8

164

8 5

5

14

5

2

9

14

10 1111 12

8

12

15

r2

r1 0

0

3) Construct a MSF F ∗ = (V ∗, E∗)

Initialization: V ∗ = {r1, r2, ..., rm} (roots are in the forest)1 Choose edge of the modified graph ei j with minimal weight such thati ∈ V ∗ and j /∈ V ∗

2 V ∗ = V ∗ ∪ {j}, E∗ = E∗ ∪ {ei ,j}3 If V ∗ 6= V , go to 1

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 37

Page 80: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Construction of a Minimum Spanning Forest (MSF)

0

1 1

2

0

0 1

0

2

0 0

3

8

164

8 5

5

14

5

2

9

14

10 1111 12

8

12

15

r2

r1 0

0

3) Construct a MSF F ∗ = (V ∗, E∗)

Initialization: V ∗ = {r1, r2, ..., rm} (roots are in the forest)1 Choose edge of the modified graph ei j with minimal weight such thati ∈ V ∗ and j /∈ V ∗

2 V ∗ = V ∗ ∪ {j}, E∗ = E∗ ∪ {ei ,j}3 If V ∗ 6= V , go to 1

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 37

Page 81: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Construction of a Minimum Spanning Forest (MSF)

0

1 1

2

0

1 1

1

2

2 2

3

8

164

8 5

5

14

5

2

9

14

10 1111 12

8

12

15

r2

r1 0

0

3) Construct a MSF F ∗ = (V ∗, E∗)

Initialization: V ∗ = {r1, r2, ..., rm} (roots are in the forest)1 Choose edge of the modified graph ei j with minimal weight such thati ∈ V ∗ and j /∈ V ∗

2 V ∗ = V ∗ ∪ {j}, E∗ = E∗ ∪ {ei ,j}3 If V ∗ 6= V , go to 1

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 37

Page 82: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Construction of a Minimum Spanning Forest (MSF)

0

1 1

2

0

1 1

1

2

2 2

3

8

164

8 5

5

14

5

2

9

14

10 1111 12

8

12

15

r2

r1 0

0

3) Construct a MSF F ∗ = (V ∗, E∗)

4) Class of each marker → class of the corresponding region(of all the pixels grown from this marker)

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 37

Page 83: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Construction of a Minimum Spanning Forest (MSF)

Pixelwiseclassification map

Map of107 markers

MSF-basedclassification map

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 38

Page 84: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Construction of a Minimum Spanning Forest (MSF)

Pixelwiseclassification map

Map of107 markers

MSF-basedclassification map

If a markeris classified

to the wrong class⇒

The whole region grownfrom this marker

risks to bewrongly classified!

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 38

Page 85: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Post-processing

Map of connected components

Pixelwise

classification map

Combination of segmentation and

spectral classification results

1 1 1 1 1 1 1 1 1 2 2 2 1 1 2 2 2 2 1 1 2 2 2 2 1 1 3 3 3 2 1 3 3 3 3 3 3 3 3 3 3 3

Final classification map

MSF-based classification map

Hyperspectral image (B bands)

Classification

map of Construction of minimum spanning

forest

markers Marker selection

Majority voting within connected

components

Segmentation map + classification map

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 39

Page 86: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Classification accuracies for the Indian Pines data (%)

SVM ECHOStrategy 2 Strategy 3: Marker-based classif.SVM+ SVM- SVM- MSSC-HSEG MSF MSF+MV MSF

Overall Accuracy 78.17 82.64 90.86 88.41 91.80 92.32Average Accuracy 85.97 83.75 93.96 91.57 94.28 94.22Kappa Coef. κ 75.33 80.38 89.56 86.71 90.64 91.19Corn-no till 78.18 83.45 90.46 90.97 93.21 89.74Corn-min till 69.64 75.13 83.04 69.52 96.56 86.99Corn 91.85 92.39 95.65 95.65 95.65 95.11Soybeans-no till 82.03 90.10 92.06 98.04 93.91 91.84Soybeans-min till 58.95 64.14 84.04 81.97 81.97 89.16Soybeans-clean till 87.94 89.89 95.39 85.99 97.16 97.34Alfalfa 74.36 48.72 92.31 94.87 94.87 94.87Grass/pasture 92.17 94.18 94.41 94.63 94.63 94.63Grass/trees 91.68 96.27 97.56 92.40 97.27 97.85Grass/pasture-mow 100 36.36 100 100 100 100Hay-windrowed 97.72 97.72 99.54 99.77 99.77 99.77Oats 100 100 100 100 100 100Wheat 98.77 98.15 98.15 99.38 99.38 99.38Woods 93.01 94.21 98.63 97.59 99.68 99.44Bldg-Grass-Tree-Dr 61.52 81.52 82.12 68.79 68.79 73.64Stone-steel towers 97.78 97.78 100 95.56 95.56 97.78

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 40

Page 87: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Classification of the Pavia image

SVMclassification

OA = 81.01%AA = 88.25%

Strategy 2:SVM + HSEG

OA = 93.85%AA = 97.07%

Strategy 3:SVM-MSF+MV

OA = 91.08%AA = 94.76%

Strategy 3:MSSC - MSF

OA = 97.90%AA = 98.59%

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 41

Page 88: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Classification accuracies for the Pavia image (%):

SVM ECHOStrategy 2 Strategy 3: Marker-based classif.

SVM+ SVM+ SVM- SVM- MSSC-Waters. HSEG MSF MSF+MV MSF

Overall Accuracy 81.01 87.58 85.42 93.85 84.14 91.08 97.90Average Accuracy 88.25 92.16 91.31 97.07 92.35 94.76 98.59Kappa Coef. κ 75.86 83.90 81.30 91.89 79.71 88.30 97.18Asphalt 84.93 87.98 93.64 94.77 93.05 93.16 98.00Meadows 70.79 81.64 75.09 89.32 72.30 85.65 96.67Gravel 67.16 76.91 66.12 96.14 89.15 89.15 97.80Trees 97.77 99.31 98.56 98.08 87.02 91.24 98.83Metal sheets 99.46 99.91 99.91 99.82 99.91 99.91 99.91Bare soil 92.83 93.96 97.35 99.76 97.11 99.91 100Bitumen 90.42 92.97 96.23 100 98.57 98.57 99.90Bricks 92.78 97.35 97.92 99.29 95.66 99.05 99.76Shadows 98.11 99.37 96.98 96.48 98.36 96.23 96.48

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 42

Page 89: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

Outline

1 Introduction

2 Classification using segmentation-derived adaptive neighborhoodsSegmentationSpectral-spatial classificationConcluding discussion

3 Segmentation and classification using automatically selected markersMarker selection

Using probabilistic SVMMultiple classifier approach

MSF-based marker-controlled region growingConcluding discussion

4 Conclusions and perspectives

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 43

Page 90: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Marker selectionMSF-based marker-controlled region growingConcluding discussion

1 Classification using automatically selected markers:significantly decreases oversegmentationimproves classification accuraciesprovides classification maps with homogeneous regions

2 Marker selection: it is advantageous to useSVM classifierspatial informationmultiple classifier approaches

3 Marker-controlled region growingMSF-based method has proven to be efficient and robust

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 44

Page 91: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Contributions

Three spectral-spatial classification strategies:1 Using SVM and MRF models

2 Using adaptive neighborhoods derived from segmentationSegmentation techniques working both in the spatial and spectral domain→ good performancesPixelwise classification + majority voting within regions→ simple and efficient technique

3 Using marker-based region growing segmentationAnalyzing probabilistic classification results for marker selectionInterest of using spatial information and multiple classifier approachesfor marker selectionMSF-based marker-controlled region growing→ efficient and robust

Possibilities of high-performance parallel computing using commodityprocessors (GPUs)

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 45

Page 92: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Perspectives

1 Spectral-spatial image analysisAutomatically select results in segmentation hierarchiesDevelop new similarity measuresPerform segmentation and classification concurrently

2 Further explore parallel strategies using commodity processors

3 Apply and adapt the proposed methods for other types ofdata/applications

medical imaging

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 46

Page 93: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

Classification of Hyperspectral Data UsingSpectral-Spatial Approaches

Yuliya Tarabalka

GIPSA-Lab, Grenoble Institute of Technology, FranceDepartment of Electrical and Computer Engineering, University of Iceland, Iceland

Thesis committee

Melba CRAWFORD PresidentJean-Yves TOURNERET OpponentXiuping JIA OpponentAntonio PLAZA OpponentPaolo GAMBA OpponentJohannes R. SVEINSSON ExaminatorJón Atli BENEDIKTSSON Thesis advisorJocelyn CHANUSSOT Thesis advisor

June 14, 2010

Page 94: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

1. Watershed segmentation for hyperspectral image

From B-band image →1-band segmentation map:

Feature extraction (PCA,ICA,...)?

Vector gradient?

Combine B gradients?

Combine B watershed regions?

Gradient (1 band – 1 band)

Watershed

Hyperspectral image (B bands)

Feature extraction (B bands – 1 band)

Combine gradients (B bands – 1 band)

(1 band – 1 band)

Segmentation map (1 band)

Combine regions (B bands – 1 band)

Y. Tarabalka, J. Chanussot, and J. A. Benediktsson, “Segmentation and classification of hyperspectralimages using watershed transformation,” Pattern Recognition, vol. 43, no. 7, pp. 2367-2379, July2010.

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 48

Page 95: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

1. Watershed segmentation for hyperspectral image

From B-band image →1-band segmentation map:

Feature extraction (PCA,ICA,...)?

Vector gradient?

Combine B gradients?

Combine B watershed regions?

Gradient (1 band – 1 band)

Watershed

Hyperspectral image (B bands)

Feature extraction (B bands – 1 band)

Combine gradients (B bands – 1 band)

(1 band – 1 band)

Segmentation map (1 band)

Combine regions (B bands – 1 band)

Y. Tarabalka, J. Chanussot, and J. A. Benediktsson, “Segmentation and classification of hyperspectralimages using watershed transformation,” Pattern Recognition, vol. 43, no. 7, pp. 2367-2379, July2010.

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 48

Page 96: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

1. Watershed segmentation for hyperspectral image

From B-band image →1-band segmentation map:

Feature extraction (PCA,ICA,...)?

Vector gradient?

Combine B gradients?

Combine B watershed regions?

Gradient (B bands – 1 band)

Watershed

Hyperspectral image (B bands)

Feature extraction (B bands – 1 band)

Combine gradients (B bands – 1 band)

(1 band – 1 band)

Segmentation map (1 band)

Combine regions (B bands – 1 band)

Y. Tarabalka, J. Chanussot, and J. A. Benediktsson, “Segmentation and classification of hyperspectralimages using watershed transformation,” Pattern Recognition, vol. 43, no. 7, pp. 2367-2379, July2010.

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 48

Page 97: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

1. Watershed segmentation for hyperspectral image

From B-band image →1-band segmentation map:

Feature extraction (PCA,ICA,...)?

Vector gradient?

Combine B gradients?

Combine B watershed regions?

Gradient (B bands – B bands)

Watershed

Hyperspectral image (B bands)

Feature extraction (B bands – 1 band)

Combine gradients (B bands – 1 band)

(1 band – 1 band)

Segmentation map (1 band)

Combine regions (B bands – 1 band)

Y. Tarabalka, J. Chanussot, and J. A. Benediktsson, “Segmentation and classification of hyperspectralimages using watershed transformation,” Pattern Recognition, vol. 43, no. 7, pp. 2367-2379, July2010.

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 48

Page 98: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

1. Watershed segmentation for hyperspectral image

From B-band image →1-band segmentation map:

Feature extraction (PCA,ICA,...)?

Vector gradient?

Combine B gradients?

Combine B watershed regions?

Gradient (B bands – B bands)

Watershed

Hyperspectral image (B bands)

Feature extraction (B bands – 1 band)

Combine gradients

(B bands – B bands)

Segmentation map (1 band)

Combine regions (B bands – 1 band)

Y. Tarabalka, J. Chanussot, and J. A. Benediktsson, “Segmentation and classification of hyperspectralimages using watershed transformation,” Pattern Recognition, vol. 43, no. 7, pp. 2367-2379, July2010.

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 48

Page 99: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Gradient

Robust Color Morphological Gradient (RCMG):

For each pixel xp, χ = [x1p, x2p, ..., xep] is aset of e vectors within E

Color Morphological Gradient (CMG):

CMGE(xp) = maxi ,j∈χ{‖xip − xjp‖2}

RCMG:xi_maxp , xj_maxp - pixels that define theCMG of xp

RCMGE(xp) = maxi ,j∈[χ−[xi_maxp ,xj_maxp ]]

{‖xip−xjp‖2}

Hyperspectral image X ( B bands)

Gradient

Feature extraction

Combine gradients

Watershed

Spectro-spatial classification

Spectral information

E

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 49

Page 100: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Gradient

Robust Color Morphological Gradient (RCMG):

For each pixel xp, χ = [x1p, x2p, ..., xep] is aset of e vectors within E

Color Morphological Gradient (CMG):

CMGE(xp) = maxi ,j∈χ{‖xip − xjp‖2}

RCMG:xi_maxp , xj_maxp - pixels that define theCMG of xp

RCMGE(xp) = maxi ,j∈[χ−[xi_maxp ,xj_maxp ]]

{‖xip−xjp‖2}

Hyperspectral image X ( B bands)

Gradient

Feature extraction

Combine gradients

Watershed

Spectro-spatial classification

Spectral information

E

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 49

Page 101: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Gradient

Robust Color Morphological Gradient (RCMG):

For each pixel xp, χ = [x1p, x2p, ..., xep] is aset of e vectors within E

Color Morphological Gradient (CMG):

CMGE(xp) = maxi ,j∈χ{‖xip − xjp‖2}

RCMG:xi_maxp , xj_maxp - pixels that define theCMG of xp

RCMGE(xp) = maxi ,j∈[χ−[xi_maxp ,xj_maxp ]]

{‖xip−xjp‖2}

Hyperspectral image X ( B bands)

Gradient

Feature extraction

Combine gradients

Watershed

Spectro-spatial classification

Spectral information

E

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 49

Page 102: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

MC-MSF classification

3-NN

ML

SVM

=

Map of MC markers

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 50

Page 103: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

MC-MSF classification

3-NN

ML

SVM

=

MC-MSF clas. map

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 50

Page 104: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

MSSC-MSF classification

WH+MV

EM+MV

RHSEG+MV

=

Map of markers

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 51

Page 105: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

MSSC-MSF classification

WH+MV

EM+MV

RHSEG+MV

=

MSSC-MSF clas. map

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 51

Page 106: Classification of Hyperspectral Data Using Spectral ...yuliya.tarabalka/files/Tarabalka_DefenseFinal.pdf · ClassificationofHyperspectralDataUsing Spectral-SpatialApproaches YuliyaTarabalka

IntroductionClassification using segmentation-derived adaptive neighborhoods

Segmentation and classification using automatically selected markersConclusions and perspectives

Segmentation and classification of the ROSIS image

SVM classif map13648 regions

RHSEG segm map7575 regions

RHSEG+MV clas map1863 regions

MSSC clas map802 regions

Yuliya Tarabalka ([email protected]) Spectral-Spatial Classification of Hyperspectral Data 52