spire consortium meeting la palma, oct. 1 – 2 2008 spire fts pipeline trevor fulton blue sky...

SPIRE Consortium MeetingLa Palma, Oct. 1 – 2 2008

SPIRE FTS Pipeline

Trevor Fulton

Blue Sky Spectroscopy, Lethbridge, Canada

SPIRE Consortium Meeting La Palma 01 October 2008

SPIRE FTS Pipeline Trevor Fulton 2

Spectrometer Pipeline

• Point source/sparse map (SOF1/3)– A spectrum of a point source that is well centred on the

central detectors of the FTS arrays and/or simultaneously obtain a sparse map of an area roughly 2'' in diameter. For sparse mapping of larger areas, a raster of point source observations will be made.

• Field mapping (SOF2/4)– To take a spectrum of a region of sky or an extended

source that is within the FOV of the spectrometer – i.e. less than 2.6'' circular. This is achieved by using the beam steering mirror to perform a low-frequency jiggle and observing multiple interferograms at each point of the jiggle pattern. For fully-sampled mapping of larger areas, a raster of multiple jiggle maps will be observed.

AOTs



Generic FTS Observation

Time

...

– The Spectrometer detector arrays are pointed at a target via a movement of the Herschel telescope (Raster) and/or the Beam Steering Mirror (Jiggle).

Pointing Building Block

Observation Building Block

– At each pointing, an FTS observation is performed by scanning the spectrometer mechanism



Spatial Sampling Options

Sparse Intermediate Full

NBSM=1 NBSM=4 NBSM=16



Spectral Sampling Options



Spectrometer Pipeline

• The flow of the SPIRE Spectrometer data processing pipelines has been designed to follow the spectrometer AOTs.

• The overall structure of each pipeline maximizes the benefit of the redundant information that exists within each observation building block.

• The data products that will be made available to observers are derived from the output of specific pipeline modules that are at the logical breaks in the overall pipelines.

Overview



FTS Pipeline

Building Block Pipeline– Modify Timelines– Create Interferograms– Modify Interferograms– Transform Interferograms– Modify Spectra



Signal Timeline Product

– Level-0.5 Product that is the input to the FTS pipeline– This product contains a timeline of the recorded

signal for each detector (Voltage vs. Time).



Modify Timelines

1st-Level Deglitching

Glitch template and

threshold

Vd-RMS(t)

V2(t)

Clipping Correction

V1(t)

V3(t)

Time Domain Phase

Correction

LPF Components

Bolometer Time

Constants

Remove Electrical Crosstalk

Electrical crosstalk

matrix

V5(t)

Non-Linearity Correction

V4(t)

V6(t)

Temperature Drift

Correction

Reference Voltage

Conversion Factors

Thermal Fluctuation

Timeline Vth(t)



– Glitches are identified in the Detector Timelines by way of wavelet analysis (same as for Photometer 1st-Level Deglitching)

– Signal samples that are flagged as glitches are corrected within the wavelet analysis.

First Level Deglitching

V1(t)

1st-Level Deglitching

Glitch template and threshold

Vd-RMS(t)



– Clipped signals are those whose raw ADC value = 0 or 216-1 (65535).

– These signal samples are corrected by way of an 8th order polynomial fit to the neighbouring signal samples.

Clipping Correction

V2(t)

Clipping Correction

V1(t)



– The combined effect of the low pass filters in the readout electronics and the thermal response of the bolometers gives rise to a delay in the measured signal samples.

– The delay is quantified by a combination of the known filter parameters and empirically derived thermal time constants for each bolometer.

– The delay per detector is removed by convolution.

Time Domain Phase Correction

V3(t)


LPF Components V2(t)

Bolometer Time Constants



– Electrical crosstalk assumptions:

– Linear– Effects on primary detector

are negligible– No x-talk between arrays


V4(t)


V3(t)

Electrical crosstalk

matrix

nnnn

n

n

n V

V

V

eee

e

ee

eee

V

V

V

3

23

114

321

13

212

13121

4

24

14

1

1

1

1



– Similar correction as that in the Photometer pipelines.

– No Flux Conversion at this point.


V5(t)


Reference Voltage V4(t)

Conversion Factors

dVVf

VftV

V

Vr

i

14

05

3

21 KV

KK

Vf

Vf

r



V6(t)

Temperature Drift Correction

V5(t)

– Vth(t), is derived from the thermometers (2 per BDA).

Temperature Drift Correction

Correlation Parameters

tVCtVtV thiii 56



FTS Pipeline

Building Block Pipeline– Modify Timelines– Create Interferograms

• Interferogram Creation– Modify Interferograms– Transform Interferograms– Modify Spectra



V6(x)

Interferogram Creation

V6(t)

Create Interferograms

Position of ZPD z(t), P(t)

Obliquity Factor

ii

i ZPDf

OPDMPD

inMPDinSMECn tMPDtz

– The mechanism timeline is divided into a set of individual timelines – one per spectrometer scan for the building block.

– The mechanism timelines are then regularized by way of interpolation and converted from MPD to OPD.



V7(x)

Interferogram Creation

V6(t)

Create Interferograms

Position of ZPD z(t), P(t)

Obliquity Factor

xVOPDVtVtVtV iiOPDiiMPDiii 66666

– The input signal timelines for each detector are then merged with the regularized mechanism timelines to create a set of interferograms (one interferogram per scan per detector).



Create InterferogramsV



Interferogram Product

• Each Interferogram Product contains one interferogram (Voltage vs. Optical Path Difference) per scan per detector.



Modify Interferograms

Telescope/SCAL Correction

Reference Interferograms

V8(x)

Baseline Removal

V7(x)

Second Level Deglitching

Glitch Threshold

V6(x)

Phase Correction

Apodization

Optical Phase

V9(x)

V10(x)

V11(x)



– The contributions to the derived interferograms from the telescope and from SCAL are removed from the measured interferograms by way of subtraction.

Telescope/SCAL Removal

V7(x)

Telescope/SCAL Correction

V6(x)

Reference Interferograms

xVxVxV irefii 67



– The position-dependent baseline for each interferogram is first characterized

– Low order polynomial– Low frequency components of the FT

– The derived baseline is then removed from each interferogram by way of subtraction.

xeVxdVxcVxbVaxV iibaseline4

173

172

177

Baseline Removal

V8(x)

Baseline Removal

V7(x)

xVxVxV ibaselineii 78

1

1

4

071 cm

cmiibaseline xVFTFTxV



Baseline Removal



– The interferograms for each detector are inspected and the statistical outliers are flagged as glitches.

– Outliers are flagged on a positional basis using the MAD method.– Samples that have been deemed to be glitches are replaced by the

average of the clean samples at that position for that detector.


V9(x)


V8(x)

Glitch threshold



– The symmetric portion of each interferogram is first transformed.

– A low-order weighted fit is made to the measured in-band for each spectrum to quantify any phase that remains.

– The phase is removed from the MR/LR spectra by multiplication and from the HR interferograms by convolution. HIGH

LOW

edcbaifit

432

Phase Correction

V10(x)

Phase Correction

V9(x)

Optical Phase

i

i

V

VTan

9

91

Re

Im

xVFTV ii 99



– The phase is removed from the MR/LR spectra by multiplication

– The phase is removed from the HR interferograms by convolution.

Phase Correction

V10(x)

Phase Correction

V9(x)

Optical Phase

ifitii ePCF

iii PCFVV 910

iii PCFFTxVxV 1910



– An apodization function is applied to each interferogram.

– This reduces the effects of the Sinc ILS in the spectral to be

derived.

Apodization

V11(x)

Apodization

V10(x)

xApodxVxV ii 1011



Apodization



FTS Pipeline

Building Block Pipeline– Modify Timelines– Create Interferograms– Modify Interferograms– Transform Interferograms

• Fourier Transform– Modify Spectra



– Each interferogram in the building block is transformed into a spectrum in this module.

– The spectral sampling interval will be fixed – the value of which will depend on the requested resolution

Transform Interferograms

V12(σ)

Fourier Transform

V11(x)

xVFTV ii 1112

L2

1

OPDNyquist

2

1



Transform Interferograms

V12(σ)

Fourier Transform

V11(x)

Observation Type

Spectral Sampling

Interval [cm-1]

Nyquist Frequency [cm-

1]

Low 0.25 200

Medium 0.05 200

High 0.01 200



Spectrum Product

• Each Spectrum Product contains one spectrum (Voltage vs. Wavenumber) per scan per detector.



Modify Spectra

Spectral Response

SpectralRSRF

I14(σ)

Flux Conversion

V13(σ)

Optical Crosstalk Removal

Optical Crosstalk

Matrix

V12(σ)

Spectral Averaging

Conversion Factors

I15(σ)

I16(σ)



– The wavenumber-dependent RSRF for the Telescope→BDA path is removed from each of the measured spectra.

Spectral Response

V13(σ)

Spectral Response

V12(σ)

Spectral RSRF

i

ii RSRF

VV

1213



– Compare the derived spectra with those from a source with a known flux.

– The ratio between the two gives the wavenumber-dependant factor by which the measured spectra must be multiplied to give flux-calibrated quantities.

Flux Calibration

I14(σ)

Flux Calibration

V13(σ)

Flux Conversion Factors

iii fVI 1314



– Now that the signal have been converted to units of optical power, an optical crosstalk correction matrix may be applied.

Remove Optical Crosstalk

I15(σ)

Remove Optical Crosstalk

I14(σ)

Optical crosstalk matrix

nnnnnn

n

n

n I

I

I

oooo

oo

ooo

oooo

I

I

I

14

214

114

321

3313

22212

1312111

15

215

115



– On a detector-by-detector and wavenumber-by-wavenumber basis, the spectra derived for all scans in the building block are averaged.

– Outliers, flagged by MAD clipping, are optionally removed from the average (default setting is to remove outliers).

Spectral Averaging

I15(σ)

Spectral Averaging

I15(σ)

ScansN

nkin

Scanski I

NI

11516

1



Average Spectrum Product

• One Average Spectrum Product per Observation Building Block

• Each product contains one spectrum (Flux vs. Wavenumber) per detector



– The average spectrum products from each building block are merged together to form a single spectral cube.

– The cube is sampled on a regular grid in both spatial dimensions as well as in the spectral dimension.

Create Spectral Cube

Spectral Cube (x, y, σ)

Spatial Regridding

ASP1(σ) ASP2(σ) ASPn(σ)ASP3(σ) …



Create Spectral Cube



Conclusions

• As for Photometer pipelines, the hard work is in producing good calibration files.

• Spectrometer Pipelines are currently undergoing phase one of the Scientific Validation.

– This includes are review of the supporting documentation

• End-to-end pipeline test #4 (November 2008).

• Complete outstanding development/calibration products (Winter 2008/2009).

• Demonstration of the pipeline in its current implementation Thursday afternoon.

spire consortium meeting la palma, oct. 1 – 2 2008 spire fts pipeline trevor fulton blue sky...

Documents