hyperspectral image projector - itea

1Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division

Hyperspectral Image Projector

Joseph P. Rice

Optical Technology DivisionNational Institute of Standards and Technology

Gaithersburg, MD 20899 USA

Collaborators: L. Brandon Shaw, Rafael Gattass, Jas Sanghera, NRLJorge Neira, Steve Brown, David Allen, Allan Smith, NIST

Michael Kehoe, Casey Dodge, Casey Smith, Rand Swanson, Resonon Inc.

This work is funded by the U.S. Department of Defense Test Resource Management Center (TRMC) Test and Evaluation / Science and Technology (T&E/S&T) Program through MIPR8DDAT3A622,

Frank Carlen, Multi-Spectral Test Focus Area Executing Agent


Problem Statement

Test and Evaluation (T&E) NeedMultispectral and Hyperspectral sensor T&E requires scene projectors that can:

Present synchronized, co-registered, spectrally & spatially realistic scenes – in the solar-reflected, visible-near-infrared (VNIR) and short-wave-infrared (SWIR)– in the thermally-emitted, mid-wave-infrared (MWIR) spectral range

Provide high spatial and spectral resolution and large scene areaOperate at real-time speedPresent high-temperature scene contentIncorporate high brightness sources to examine stray light entry & effects

Science and Technology (S&T) ChallengesProject dynamic, 2D spatial images, with pixel-by-pixel programmable spectral contentDevelop a high output, broadband MWIR sourceControl, measure, and correct stray light in the projector and in the reference instruments used to characterize the projected scenes

TRL Start/Stop: 3/6

End State is an integrated and calibrated prototype 2D VNIR/SWIR/MWIR Hyperspectral Image Projector (HIP)


Target Pixel

TrackDirection

Spatial Stray Light

Spectral Stray Light

Spectral and SpatialStray Light

Sensor Focal Plane Array

Sensor Optics

FOV

The HIP Probes Spectral and Spatial Stray Light Paths of the Sensor Under Test

Consider the stray light contributions to thenadir pixel in, for example, the typical pushbroomsensor as depicted here:

•In the real world, most pixels, like the red pixel, provide stray light to the sensor

•This stray light could masquerade as a targetof interest, causing false positives

•Or it could clutter the scene, causing missedidentifications.

•The HIP properly probes such stray light paths, since it includes such pixels in its test


Broadband Fiber Light SourceProvides high brightness in a small spot for high spectral resolutionFused-silica fiber supercontinuum source for VNIR/SWIRChalcogenide fiber supercontinuum source for MWIR

Spectral EngineFiber source light is spectrally dispersed across a digital micromirror device (DMD)Image reflected from on-pixels of DMD is spatially integrated and determines spectrum

Spatial EngineUses another DMD to project spectra into Unit Under Test (UUT) per 2D spatial abundance images, through a collimator, in sync with spectral engine

Reference InstrumentWell-calibrated imaging spectrometer to measure the spectrum of each spatial pixel at the output, relative to NIST reference standards, to provide truth data

HIP Background

Spectral Engine(Spectra Generator)

Reference InstrumentBroadband Fiber Light Source

Spatial Engine(Image Projector) Unit Under Test

Computer

VNIR/SWIR: 450-2500 nm, MWIR: 3000 – 5000 nm


Principle of the Spectral Engine

Dispersing element Spatial Light Modulator(SLM) Recombine the Light

SLM Light Guide

Available High-Resolution Spatial Light Modulators:• Digital micromirror devices (DMD)• Liquid crystal on silicon (LCOS) arrays


Digital Micromirror Device (DMD)

An array of MEMS micromirror elementsThe HIP uses these in the Spectral Engine

and in the Spatial EngineDeveloped by Texas Instruments (TI)

1024 x 768 elements, +/- 12 degree tilt angle Aluminum mirrors13.68 micron pitch< 24 microseconds mechanical switching time

For the prototype HIP DMDs, TI D3000 XGA DMDs: 11 kHz binary frame rate is achievedThe standard glass windows have been replaced by windows optimized for each spectral rangeBrassboard control algorithms use LabVIEW top-level software with a USB interface to a standard PCFor the prototype, we are exploring real-time drive electronics options for these DMDs

MEMS = Micro-Electro-Mechanical System

Mirror -12 degreesMirror +12 degrees


How the DMD is used in a spectrally tunable source: Monochromator Mode

Near IRUV Visible

1024 mirrors

768

mi r

ror s

Wavelength

Inte

nsity

mirror array

Mirrors in "On"Position


Near IRUV Visible

1024 mirrors

768

mir

r ors

Wavelength

Inte

nsity

mirror array

How the DMD is used to create an arbitrarily programmable spectrum


VNIR/SWIR Supercontinuum Fiber Source

Utilizes non-linear effects in a photonic crystal optical fiber to greatly broaden the spectrum of a 1064 nm pump laser

Broadband light is generated in a single-mode (<5 μm core diameter) photonic crystal (holey) optical fiber

No etendue issues as with lamps or blackbodiesIdeally suited for coupling to a spectral engine

High power and high spectral resolution:

3 mW/nm spectral power density from 450 nm to 1700 nm

Presenter

Presentation Notes

One question is how do we get enough flux into the spectrograph? Coupled into the input fiber in the Lambda Commander? Lamp-based sources have a large spatial footprint making it difficult to efficiently couple the flux into an optical fiber (or through a monochromator). We are turning to supercontinuum sources for the input. In supercontinuum sources, broad-band white light is generated in a single-mode optical fiber (5 um core diameter). Since the light is already in an optical fiber (and a single mode one at that), couple issues are solved in principle. And, since it would correspond to an entrance slit of 5 um, these sources offer the possibility of both high flux and high spectral resolution! Current commercial sources offer 3 mW/nm spectral power density from approximately 450 nm to 1700 nm, with radiant flux available out to 2500 nm.


HIP Prototype Concept

NIST ReferenceInstruments

Supercontinuum Fiber Laser

Light SourceVNIR/SWIR

Spatial EngineVNIR/SWIR Sensor

Under Test

Computer

VNIR Spectral Engine

SWIR SpectralEngine

MWIR Spectral Engine

MWIR Spatial Engine

MWIR Sensor Under Test

450-2500 nm

3000 – 5000 nm

Chalcogenide Supercontinuum

Light Source

Acronyms: VNIR = Visible Near Infrared; SWIR = Short Wave Infrared; MWIR = Mid Wave Infrared


HIP Prototype Specifications

Parameter Target Specification

Spectral Range450 - 2500 nm (VNIR-SWIR)3000 – 5000 nm (MWIR)

Spectral Resolution 5 nm VNIR8 nm SWIR8 nm MWIR

VNIR/SWIR/MWIR Sync. Accuracy 1 microsecond

Spatial Format 1024 H 768 V

Projected FOV 7.9 H 5.9 V

Spatial Resolution 0.135 mrad

Average Spectral Radiance VNIR/SWIR 1000 W/m2srμmMWIR: 750 W/m2srμm

Bit Depth/ Frame Rate 12 bits/250 Hz

Contrast Ratio VNIR/SWIR: 1000:1 MWIR: 250:1

Wavelength Accuracy 2 nm

Radiance Accuracy 2%


Design Concept for VNIR/SWIR Spectral Engine

SupercontinuumFiber Light

Source

DMD2

Recombining System

Dispersing Prisms

DMD1

VNIR SWIR

Input

Output toSpatial Engine

SWIR

VNIR

• Splits input into VNIR and SWIR• One DMD (1024x768) for each range• Optical modeling in Zemax and FRED

– Prisms selected– Design meets requirements

Spectral rangeTransmittanceSpectral resolution

• Mechanical modeling in Solid Works

VNIR/SWIR spectral engine fits in 19” rack-mountable chassis

Source Spectrum


HIP VNIR-SWIR Spectral Engine

Inside view from back:Outside view from front:

Supercontinuum Light Source

VNIRSpectralEngine

SWIRSpectralEngine


HIP VNIR-SWIR Spectral Engine


VNIRSpectralEngine

SWIRSpectralEngine

Source Collimator

ElectronicsSection

Output

}}

View from back:

DichroicBeamsplitter

DichroicBeamcombiner


VNIR-SWIR Spatial Engine Optical Design

Sensor


VNIR-SWIR Spatial Engine Mechanical Design

DMD

DMD Driver Electronics PC Boards Cooling Fan

ReplaceableCollimator

Purged, SealedOptics SectionBeam Pipe

For Direct CouplingFor Spectral Engine

ElectricalConnectionsTo SpectralEngine


VNIR-SWIR HIP System

Brassboard

VNIR Spectral Engine

SWIR Spectral Engine

Spatial Engine

Collimator For Projectionto UUT

Super-continuumSource

Power Supply


VNIR-SWIR HIP

Liquid Light Guide:•Couples HIP Spectral Engineto HIP Spatial Engine

HIPSpectral Engine

(rear view)

HIPSpatial Engine

TranslationStageBrassboard


VNIR Spectral Resolution Test Data

Meets the < 5 nm spec

Spectral Resolution of Test Instrumentation is 0.2 nm


SWIR Spectral Resolution Test Data

Meets the < 8 nm spec

Spectral Resolution of Test Instrumentation is 0.5 nm


Spectral Matching Performance

• Image on the Spectral Engine DMD that gives the spectrum above

• Measured and target spectra with supercontinuum fiber light source with the breadboard visible-band spectral engine

• NIST-developed software has been used to automatically calibrate this and match to target spectrum

Wavelength

Inte

nsity


The HIP Can Match Typical Reflected-Solar Radiance Spectra

• HIP has enough light to simulate a bright sunny day outside• Left panel shows the maximum HIP spectral radiance (all DMD micromirrors on)• Left panel shows the Top-of-Atmosphere (TOA) Earth reflected radiance spectrum• Right panel is an expanded scale: Red shows HIP output matched to some typical Earth-reflected spectra in the Visible-Near IR (VNIR)


Hyperspectral Imager Test at the HIP

Used a VNIR pushbroom Hyperspectral Imager (HSI) from collaborators at University of ColoradoInput data was a real scene collected by HSI Projected by the HIP and measured by the HSI.HSI scanned HIP to simulate ground track motion

HIP Projected, HSI Measured:

HSI

With Greg Kopp , Paul Smith, and Joey Espejo of the University of Colorado

Presenter

Presentation Notes

With Greg Kopp, Joey Espejo, and Paul Smith of LASP


MWIR Spectral Engine Design


MWIR Spatial Engine Design


Monochromatic HIP Projection Algorithm

BSQ Cube1 1024x768xM

Load M ImagesOnto Spatial Engine

DMD

Project into Sensor

Issues:•Not light efficient to use monochromatic bands: low brightness•Need to cycle through all M monochromatic bands for each cube: slow

Maximum cube rate is f/M, where f is single image frame rate.

M is typically 100 to 1000

M X

BSQ (Band Sequential): A stack of 2D images, each at one of M different monochromatic wavelengths


Spectral Engine DMD in Monochromator Mode

Near IRUV Visible

1024 mirrors

768

mi r

ror s

Wavelength

Inte

nsity

mirror array

Mirrors in "On"Position


Near IRUV Visible

1024 mirrors

768

mir

r ors

Wavelength

Inte

nsity

mirror array

Spectral Engine DMD in Broadband Mode


HIP Image Cube Projection Concept

Spatial Engine• Projects images with

component spectra into sensor

• Reference instrument characterizes output

ReferenceInstrument

Sensor Under Test


Spectral Engine• Uses light from supercontinuum source• Produces programmable spectra that

match component spectra• Directs these to Spatial Engine

0.0

0.2

0.4

0.6

0.8

1.0

400 450 500 550 600 650 700 750

Pow

er

Wavelength (nm)

Example Spectrum

Input HyperspectralImage Cube

ComponentSpectra

Principle-components-type algorithm reduces image cube to Component Spectraand Abundance Images

AbundanceImages


InputImage Cube

Example: AVIRIS Image Cubeof San Diego Naval Air Station

Given an image cube, ENVI/SMACC (COTS software) is used to find the Component Spectra and their Abundances

ES-3

ES-4

ES-2 ES-5

ES-6

Component Abundances

ES-1

ES-1

Component Spectra

ES-2

ES-3

ES-5ES-4

ES-6

Compressive Projection

Spectrum of each pixel is a linear combination of these component “eigenspectra”.

Then we need only project N = 6 broadband spectra instead of M = 30+ monochromatic spectra.

RGB Image:

Presenter

Presentation Notes

J. Gruninger, A. J. Ratkowski, and M. L. Hoke, “The sequential maximum angle convex cone (SMACC) endmember model,” Proc. SPIE 5425, 1-14 (2004).


Prototype HIP Projection Algorithm

BSQ Cube1 1024x768xM

SMAAC to1024x768xN

BSQ (Band Sequential): A stack of 2D images, each at one of M different monochromatic wavelengthsSMAAC: An algorithm to reduce image cube to N component spectra and corresponding spatial images

Project into Sensor

Match N SpectraDisplay N Spectral

Images ontoSpectral Engine DMD

Display N SpatialImages onto

Spatial Engine DMD

N X

•M is typically between 100 and 1000; N is typically < 10Maximum cube rate is f/N, where f is single image frame rate.

•Spectral image display in sync with spatial image display•All spectral and spatial images displayed during sensor exposure time

N X

Faster and brighter than monochromatic projection


Demonstrated CompatibilityBetween MSP2 and HIP

•MSP2 is real-time scene generator software developed by Signature Research•Interface exercise between Signature Research (MSP2) and NIST (HIP)

VNIR Image cube rendered from MSP2 as 1024 x 768 x 150 bandsSuccessfully read into ENVI (RGB false-color image below added in ENVI)Pre-processed for HIP projection using SMAAC

ENVI: ENvironment for Visualization of Imagery (COTS software for analyzing image cubes)

MSP2 data cube from Denise Talcott and Marshall Weathersby of Signature Research, Inc.


Summary

The Hyperspectral Image Projector (HIP) is being developed by NIST and collaborators to address scene presentation issues for realistic T&E of multispectral and hyperspectral imaging sensorsProjects a 2-D image into the sensor whereby the spectrum of each pixel can be independently controlled, to simulate realityConcept has been demonstrated in the VNIR-SWIRPrototype operating in VNIR/SWIR/MWIR currently under developmentPlanning to demonstrate HIP prototype by end of FY2012To be delivered to U.S. Army Redstone Test Center

This work is funded by the U.S. Department of Defense Test Resource Management Center (TRMC) Test and Evaluation / Science and Technology (T&E/S&T) Program through MIPR8DDAT3A622,

Frank Carlen, Multi-Spectral Test Focus Area Executing Agent

hyperspectral image projector - itea

Documents