The Bernard M. Gordon Center for Subsurface Sensing and Imaging Systems
Miguel Velez-Reyes,Thrust Leader
R2-C: Multi-spectral Discrimination
This work was supported by in part by the Gordon Center for Subsurface Sensing and Imaging Systems, under the Engineering Research Centers Program of the National Science Foundation (Award # EEC-9986821).
Multi-SpectralDiscrimination
(MSD)
Probe
Multi-BandDetectors
The MSD TEAMFaculty/Researchers
M. Velez-Reyes, UPRMS.D. Hunt, V. Manian, N. Santiago, UPRMJ. Goodman, U of Mia RSMAS & UPRMS. Rosario, UPRMB. Roysam, RPIM. Bystrom, BUM. Diem, NEU
MS (9)Andres Alarcon, UPRMCarolina Peña, UPRMLeidy P. Dorado, UPRMAndrea Santos, UPRMNicolas Rey, UPRMOsmarh Martinez, UPRMNestor Díaz, UPRMOrian Tzadik, UPRMKarin Griffis, BU
Ph.D (5)Miguel Goenaga, UPRMMaider Marín, UPRMMaria C. Torres, UPRMAmit Mukherjee, RPIEladio Rodriguez, BUTatiana Cherneko, NEU
UG (6)Suhaili Cardona, UPRMYahayra Gonzalez, UPRMJoralis Sanchez, UPRMChristine Cortes, UPRMYajaira Gonzalez, UPRMLuis Alvarado-Ortiz, UPRM
Spectral ImagingB. Saleh, Intro to SSI
Wavelengthsensitivedetection
Wavelengthsensitivedetection
object
MediumClutter
BroadbandProbe, P
BroadbandProbe, P
Imager-Spectrometer Configuration
BroadbandDetector
BroadbandDetector
object
MediumClutter
Probes at different wavelengths, Pi
Probes at different wavelengths, Piλ1 λ2 .. λn
Spectrometer-Imager Configuration
λ1 , λ
2 , …,λm
( ) ( )( )( ) ( )iiiii λ,wγ,S,λβα,Τλ,Y rrr +=
Sampling the Spectra
Spectral Physics-Based Signal Processing (R2C)
o Crop health o Chemical composition, pH, CO2o Metabolic information o Ion concentrationo Physiological changes (e.g., oxygenation)o Extrinsic markers (dyes, chemical tags)
Examples of β
Detect: presence of a target characterized by its spectral features α
or β
Classify: objects based on features exhibited in α
or β
Understand: object information, e.g., shape or other features based on α or β. Integrating spatial and spectral domains.
Or
Estimate: probed spectral signature {α (x,y,λ)}
physical parameter to be estimated {β(x,y,λ)}αβ
M
Challenge: Complexity of the media
High spectral resolution is needed to resolve the different components
From C.O.Davis, HSI of the Littoral Battle Space, NRL Code 7203
sc
Epidermis
Dermis
Re Rd(λ)
Venous Hb
Arterial Hb
R
100 μm
2 mm
Stratum Corneum
sc
Epidermis
Dermis
Re Rd(λ)
Venous Hb
Arterial Hb
R
100 μm
2 mm
Stratum Corneum
Benthic Habitat Monitoring Biomedical Imaging
Challenge: Small Signal from the Object of Interest
Reflected Bottom Radiance
Water Column Reflected Radiance
Reflected Bottom Radiance
Water Column Reflected Radiance
From NEMO OverviewNemo.nrl.navy.gov
What we Measure
What we want
• Optical Properties of the Water• Bottom Reflectance• Bathymetry
Challenge: Integration of Spectral and Spatial Information
Point-by-point spectroscopy still the major approach for HSI processing
Full HSI exploitation requires integration of spatial-spectral domain information
Pixel spatial coordinates randomly shuffled
Spectral processing Spectral processingSame final per- pixel analysis
results
Challenge: Data Overload Problem
1 Hyper-Spectral Image per sec
105 Gbytes per day
108 Books per day
Fast, Automated, (On-Board) Processing
Only ~2x107 Books inLibrary of Congress
From S. Adler, CenSSIS RICC 2004
MSD Research Across the Center
R2: MultispectralPhysics-Based Signal ProcessingFundamental
ScienceFundamentalScience
ValidatingTestBEDsValidatingTestBEDs
L1L1
L2L2
L3L3 S4
Bio -Med Enviro -Civil
R3: AlgorithmImplementation
Benthic HabitatMapping
R1: Multispectral Sensing
S1 Microscopy,Celular Imaging
R2C Research ProjectsComplexity of the Media
Hyperspectral Image UnmixingUnsupervised Methods using PMFSubsurface Unmixing for Benthic Habitat Mapping
Classification and DetectionSupport vector machinesCurve evolution methods
Small Signal from the targetSignal Enhancement
Denoising of Hyperspectral Imagery using Raman Spectroscopy
Subsurface Unmixing for Benthic Habitat Mapping
R2C Research ProjectsSpectral/Spatial Integration
Scale-Space Representation using Geometric PDEsDenoisingImage segmentation and registration
Vectorial TextureChange Detection
Data Management and ComputingToolboxes: HIAT, and HyCIATHyperspectral image processing using GPUs
Experimental WorkSeaBED – Testbed for coastal remote sensingVIS, MWIR, NIR cameras and spectrometer available on campusOther facilities available from other CenSSIS partners
Hyperspectral Image Registration (R2C.p3)
Image Registration
Reference Image
Sensed Image
Feature DetectionControl Points are
selected
Feature Matching Parameter ModelEstimation
Image Resampling
Transformed Image
SIFT Detector for Grayscale Images (Lowe 2002, 2004)
Threshold
-
-Generation of Scale Space
Gaussian SmoothingDifference-of-Gaussians
DoG
Local Maxima Pixel
Interest Points
Increasingof
Scales
rr
HDetHTr 22 )1(
)()( +
=
( )⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
∂∂
∂∂∂
∂∂∂
∂∂
=
2
22
2
2
2
yDoG
xyDoG
yxDoG
xDoG
yxDoGH ),(
Different scales
Increasingof
Scales
Statistical Decision
Original Image
Band by Band Approach (Mukherjee et.al. TGARS 2009)
Function for combining DoG responses along
spectral dimension
PCAprojection
Threshold
rr
HDetHTr 22 )1(
)()( +
=
Generation of Scale Spaceby Gaussian Smoothing
Local Maxima Pixel
Different scales
Interest Points
Comp. 1
Comp. 2
Comp. M
IncreasingScale
Comp. 1
Comp. M
Comp. 2
Difference-of-GaussiansDoG
( )⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
∂∂
∂∂∂
∂∂∂
∂∂
=
2
22
2
2
2
yDoG
xyDoG
yxDoG
xDoG
yxDoGH ),(
Increasing Scale
-
-
Comp. 1
Comp. 2
Comp. M
IncreasingScale
IncreasingScale
Original Image
New Approach: Multi-channel Approach
Scale Space Representation by Anisotropic Diffusion
Difference of adjacent scales
IncreasingScale
Local Maxima Pixel-vector Vector Ordering
Threshold
( )⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
∂∂
∂∂∂
∂∂∂
∂∂
=
∑∑
∑∑
==
==M
i
iM
i
i
M
i
iM
i
i
yD
xyD
yxD
xD
yxDII
12
2
1
21
2
12
2
),(
( )( ) r
rIIDet
IITr 22 )1( +=Interest Points
Scale t+1
Scale t
Scale t-1
Band Subset Selection
IncreasingScale
-Original Image
Experimental Results
4380 Interest PointsAnisotropic DiffusionVector Ordering andSecond Fundamental Form
2506 Interest PointsMukherjee’s Approach (Gaussian Smoothing and Non-linear function)
Benchmarking Unmixing using cPMF (R2C.p4)
Experimental comparison of cPMF with standard unmixing algorithms that retrieve endmembers from the image pixels using AISA hyperspectral images collected over Vieques Island in Puerto Rico.
Obtain endmembers by
cPMF SMACC Max D
ABUNDANCE ESTIMATION
EndmembersExtraction
EXPERIMENTS AND RESULTSEXPERIMENTS AND RESULTS
Red Mangrove
Bare soil/ dirt roads
High trees (Flamboyan tree)
Black mangrove and palm
Water
Shrubs (zarcillo)
EXPERIMENTS AND RESULTS: WATER ENDMEMBERS
Water
Water (Max D)
0
0.5
1
Water (SMACC)
00.20.40.60.8
Water (cPMF)
0
0.5
1
500 550 600 650 700 750 800 850 9000
100
200
300
400
500
600
700
800
900
1000Spectral Signatures Comparison of Water
Wavelength [Nanometers]
Am
plitu
de
Water (cPMF)Water (SMACC)Water (Max D)
Hyperspectral Image Processing Solutionware
Supervised Classification Module
Unsupervised Classification Module
MATLAB Toolbox Parallel and Distributed Computing
Hardware Implementationin FPGA/DSP and GPUs
New: Hyperspectral Coastal Image Analysis Toolbox (HyCIAT)
Collection of methods for analysis of hyperspectralimages over coastal environments.
Collection of functions that extend the capability of the MATLAB® numeric computing environment.Designed in Macintosh and PC-Windows Systems.
Based in algorithms developed by Dr. James Goodman and from research done in Laboratory of Applied Remote Sensing and Image Processing (LARSIP).
HyCIAT: Processing Scheme
HyCIAT: Sample Results
Abundances and Fractional Maps
1. Low-dimensional representations preserve distance throughout manifold.
2. Only local information is required to produce low-dimensional representation.
3. We can infer other interpoint distances.
1. High-computational cost is required.
2. For Hyperspectral Images, Manifold learning algorithms do not take in consideration spatial information.
Manifold Learning for HSI Exploitation
Manifold Learning using GPU's
Algorithm TaskComputational Complexities
CPU/GPU processing
Isomap
Find k-NN per pixel O(n2) GPUCompute approximate geodesic distance O(n2log(n)) CPUFind optimal t dimensional representation O(n3) GPU
Laplacian
Eigenmaps
Find k-NN per pixel O(n2) GPUConstruct diagonal matrix D O(kn) CPUFind the t dimensional representation O(n3) GPU
Locally Linear Embedding
Find k-NN per pixel O(n2) GPUCalculate matrix C O(kn) GPUCalculate matrix M O(n3) GPU
Cuprite scenario
Execution time as a function of image size
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0 20 40 60 80 100 120
x103
x103Number of pixels
Exec
utio
n tim
e (s
)
CPUGPU
Execution time as a function of k in the k-NN search
0
1
2
3
4
5
6
5 10 15 20 50
k
GPU (40000 pixels)CPU(40000 pixels)GPU (60000 pixels)CPU (60000 pixels)
MSD PostersR2C
R2C p3
Leidy Paola Dorado-Munoz / UPRM Amit Mukherjee / RPI
Miguel Velez-Reyes / UPRM Badrinath Roysam / RPI
"Detection of Interest Point for Multispectral and Hyperspectral Images Using Lowe’s Approach and Anisotopic Diffusion"
R2C p4 Andrea Santos- Garcia / UPRM
Miguel Velez-Reyes / UPRM Samuel Rosario / UPRM Jesus D. Chinea / UPRM
"A Comparison of Unmixing Algorithms for Hyperspectral Imagery"
R2C p5 Carolina Pena Ortega / UPRM
Miguel Velez Reyes / UPRM
"Comparison of Basis-Vector Selection Methods for Target Detection"
R2C p6 Nestor J. Diaz G. / UPRM Vidya Manian / UPRM "Hyperspectral Texture Synthesis by
Multiresolution Pyramid Decomposition"
R2C p7 Karin Griffis / BU Maja Bystrom / BU"A Tunable, Multi-scale, Multi-band Segmentation Procedure for Remotely-Sensed Imagery"
R2C p8 Eladio Rodriguez- Diaz / BU
David A. Castanon / BU Irving J. Bigio / BU
"Pattern Recognition Methods for Spectral Classification in ESS Diagnosis of Cancer"
Related PostersR3
Validating TestBEDsSea p2 Carlos J. Solis Ramirez
/ UPRMRaul E. Torres / UPRM
"Hyperspectral Image Registration and Fusion for Underwater Applications"
Sea p3 Carlos J. Solis Ramirez / UPRM
Raul E. Torres / UPRM
"Modification of the SeaBed Autonomous Underwater Vehicle for Hyperspectral Image Acquisition"
R3A p6 Yajaira Gonzalez Gonzalez / UPRM
Nayda G. Santiago / UPRM
"Analyzing the Use of GPUs for Hyperspectral Image Processing"
R3B and Demo p1
Maria Constanza Torres Madronero / UPRM
Miguel Velez Reyes / UPRM
"HyCIAT: Hyperspectral Coastal Image Analysis Toolbox"
R3B and Demo p2
Samuel Rosario / UPRM Miguel Velez Reyes / UPRM
"Speeding Up the Hyperspectral Image Analysis Toolbox"
Questions?
Suggestions, Advice, …