xdgnirs v2.0

XDGNIRS v2.0

Rachel Mason & Omaira Gonzalez-Martın

November 6, 2013

Contents

1 Introduction 2

2 Code, Access and Requirements 4

2.1 The Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.2 The Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

3 Inspecting Your Data 6

4 Running the Pipeline (aka “if you don’t read anything else”) 6

5 The Reduction Steps 8

5.1 Identifying the Input Files (Step=0) . . . . . . . . . . . . . . . . . . . . . . . 8

5.2 Pattern Noise Cleaning (Step=1) . . . . . . . . . . . . . . . . . . . . . . . . 9

5.2.1 Example: NGC 1167 . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

5.3 Preparing the Data (Step=2) . . . . . . . . . . . . . . . . . . . . . . . . . . 10

5.3.1 Example: NGC 1167 . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

5.4 Making the Flatfield (Step=3) . . . . . . . . . . . . . . . . . . . . . . . . . . 15

5.4.1 Example: NGC 1167 . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

5.5 Reducing the Science Target and Standard Star Data (Step=4, 5) . . . . . . 16

5.5.1 Example: NGC1167 . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

5.6 S-distortion Correction and Wavelength Calibration (Step=6) . . . . . . . . 19

5.6.1 Example: NGC1167 . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

– 2 –

5.7 Extracting the Spectra (Step=7) . . . . . . . . . . . . . . . . . . . . . . . . 23

5.7.1 Example: NGC1167 . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

5.8 Telluric Line Cancellation and Flux Calibration (Step=8) . . . . . . . . . . . 26

5.8.1 Finding magnitude and redshift information . . . . . . . . . . . . . . 27

5.8.2 Removing the lines in the standard star . . . . . . . . . . . . . . . . . 28

5.8.3 Correcting for telluric absorption . . . . . . . . . . . . . . . . . . . . 30

5.8.4 Flux calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

5.8.5 Example: NGC1167 . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

5.9 Combining the Orders (Step=9) . . . . . . . . . . . . . . . . . . . . . . . . . 31

5.9.1 Example: NGC1167 . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

5.10 Science Target Data Sheets (Step=10) . . . . . . . . . . . . . . . . . . . . . 34

5.10.1 Example: NGC1167 . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

6 Usage/Syntax Examples 34

7 Troubleshooting and Support 36

8 History 36

1. Introduction

This document describes the reduction of GNIRS cross-dispersed (XD) data with the

“XDGNIRS” pipeline (v2.0). The pipeline, which was created for reduction of spectra of

bright, extended galaxies from the Palomar XD programme, is able to take a list of science

and calibration files, identify the file types, reduce the data, and output a flux-calibrated

1D spectrum, with various degrees of manual intervention as desired by the user. The

Palomar XD data are acquired using GNIRS’ popular 32 l/mm cross-dispersed mode, and

the XDGNIRS pipeline was written to handle this particular setup (Table 1). In addition,

it has been generalised to accept data taken without nodding off the slit to blank sky. The

pipeline may work with other cross-dispersed configurations, but this has not been tested.

Potential users are encouraged to familiarise themselves with the reduction of the test data

– 3 –

set (illustrated in this manual) before attempting to run the pipeline on other data.

More explanation of the principles behind each step in the reduction of XD data,

with example images and specific IRAF task parameters, can be found on this web page:

www.gemini.edu/sciops/instruments/ gnirs/data-format-and-reduction/reducing-xd-spectra

(referred to as the “GNIRS XD DR page” from now on). We recommend reading that page

before, or in conjunction with, this document.

The basic architecture of XDGNIRS was set up by Omaira Gonzalez-Martın, and the

code was completed, tested and documented by Rachel Mason, with input from Emma

Hogan, Tom Geballe, Andrew McNichols, Jen Miller, Alberto Rodrıguez-Ardila and Daniel

Ruschel Dutra. OGM is the author of the “REDCAN” IDL pipeline for reduction of mid-IR

imaging and spectroscopy from CANARICAM on the GTC (Gonzalez-Martın et al. 2013,

A&A, 553, 35). REM claims no programming expertise and cannot be held responsible for

negative emotional states that may result from looking at her code.

Table 1. Observational Setup

Object Camera Grating Prism Slit Dither pattern

Science target SB (0.15′′/pix) 32 l/mm SXD 0.3′′ 3 x ABBA, q = 0′′, 50′′a

Standard star SB (0.15′′/pix) 32 l/mm SXD 0.3′′ 2 x ABBAb , q = 1′′, -2′′

aExtended galaxies; unguided offset to blank sky

bDetermined by star magnitude/adjusted by observer

– 4 –

2. Code, Access and Requirements

2.1. The Code

The main program in XDGNIRS, XDpiped.csh, is a “wrapper” script that calls various

other python and shell scripts. The python scripts in turn call various PyRAF tasks in the

Gemini GNIRS data reduction package. XDGNIRS has been tested with versions 1.11.1

and 1.12beta(2) of the Gemini IRAF package. We have found that the most reliable

performance is achieved using the “Ureka” installations of IRAF, PyRAF and

python. We highly recommend installing this software before using XDGNIRS.

Ureka is an easy-to-install package created by Gemini and STScI, available here:

http://ssb.stsci.edu/ureka/

http://ssb.stsci.edu/ureka/1.0beta5/

Users who decide not to use Ureka and are using Gemini IRAF v1.11 will need to

install the v11.1.1 patch to avoid the code crashing on step 6. See the instructions at

www.gemini.edu/sciops/data-and-results/processing-software/download.

The parameters of the Gemini IRAF tasks have been selected to generally work well on

“normal” XD data sets, but if necessary they could be changed by directly editing the calls

to the tasks in the *py scripts. This manual describes how to run XDGNIRS; for help with

the individual IRAF tasks, please see the GNIRS XD DR page and IRAF’s command-line

help.

The XDGNIRS code and example data can be obtained from Dropbox. Three files are

available:

1) https://dl.dropboxusercontent.com/u/28931879/XDGNIRS v2.0.tar: The code and

this documentation

2) https://dl.dropboxusercontent.com/u/28931879/example data.tar.gz: The example

data set and a reduced spectrum in fits, txt and pdf format. Use this if you want to simply

run the code and reproduce these results.

3) https://dl.dropboxusercontent.com/u/28931879/example reduction.tar.gz: All of the

intermediate steps and reduced products (e.g. rest-frame spectra, various apertures, text and

pdf formats). Download this to follow along with the example in this manual, and for future

troubleshooting etc.

Assuming you already have python/Gemini IRAF/PyRAF available, installing the pipeline

involves simply unpacking the tar file and adding a couple of lines to your .bashrc or .cshrc

– 5 –

file (see the “README.txt” file for instructions). As described on the GNIRS Status and

Availability web page1, observers using Gemini IRAF v1.11.1 to reduce data obtained after

June 2012 need to use certain updated IRAF configuration files. These files are included

with XDGNIRS, are automatically used where appropriate, and need not be downloaded

separately. The “cleanir” python script is also included with XDGNIRS (see §5.2).

2.2. The Data

XDGNIRS expects the following raw data:

• Science target acquisition images

– Target name in headers must be identical to target name in spectroscopy file headers

– Only the final acq. image in the sequence is strictly necessary

• Science target spectroscopy files

– At least two object-sky pairs

– ABBA, ABA, or (probably) ABAB nod pattern

– Nod along slit or off to blank sky

– Must contain a “q” offset (as well as or instead of “p” offset)

• Standard star acquisition images

– Same criteria as for science target acq. images

• Standard star spectroscopy files

– One standard star observation only

– Same criteria as for science target spectroscopy

• One or more arcs

• A set of IR and QH flats

• One or more pinhole flats

All of these are acquired as standard for GNIRS XD queue observations. Note that the

code can currently only deal with one standard star and one science target data set at a time.

If you have an observation of a single target split over >1 epoch (including re-acquisitions on

the same night), you will need to reduce each segment separately. If you have two standard

stars, you can either use only one, or repeat the reduction with both standards and see which

one works better. Combining standards is not supported.

1www.gemini.edu/sciops/instruments/gnirs/status-and-availability; 21 Dec 2012 and 15 Nov 2012 up-

dates

– 6 –

3. Inspecting Your Data

Given a list of science and calibration files, the XDGNIRS pipeline can identify the data

types and perform a full reduction. Some checks are performed along the way, and these are

described in the relevant sections of this document. However, it is advisable to at least do

a quick visual inspection of the data before entrusting the reduction to the pipeline. The

GNIRS XD DR page gives more information and some examples. The Quality Assessment

(QA) state of each file, also described on the above web page, should be noted as well. Files

set to ”USABLE” should not be included in the reduction unless the user is sure he/she

knows what they’re doing. Files set to ”FAIL” almost certainly should not be used.

4. Running the Pipeline (aka “if you don’t read anything else”)

XDGNIRS is run by typing the “xdpiped” command, plus a list of options. At a

minimum, XDGNIRS needs a list containing the names of the files on which it will operate.

It also needs to know if you’re using v1.11.1 of the Gemini IRAF package (as opposed to

v.1.2beta). For example, to run the script in the same directory as the raw data files with

no user interaction, using Gemini IRAF v1.11.1:

ls *fits > inputfiles.lst

xdpiped inputfiles.lst gemiraf_ver=1.11.1 ureka=no

The input file list must contain all the science, acquisition and calibration data needed

for the reduction (§2.2). The files do not need to be sorted according to the data type; the

code will take care of that (§5.1). Sometimes the observer will have adjusted the exposure

time of the standard star to avoid saturation or increase the signal. In this case, only the

files with the appropriate exposure time should be included in the input file list (although

the code will attempt to flag deviant exposure times and exit if any are found).

The available options can be seen by typing

xdpiped -h

At the time of writing, these are (with defaults):

rawdir: (./) [location of raw data]

step: 0-10 (0) [step at which to start the reduction]

– 7 –

stop: 0-10 (10) [step at which to stop the reduction]

cleanir_flag: nfrq (f) [whether/how to run cleanir.py]

nsflat_inter: yes/no (no) [run nsflat interactively?]

tgt_thresh: (20) [threshold for radiation event removal from science target

data]

std_thresh: (50) [threshold for radiation event removal from standard star

data]

nssdist_inter: yes/no (no) [run nssdist interactively?]

nswav_inter: yes/no (no) [run nswavelength interactively?]

nsfit_inter: yes/no (no) [run nsfitcoords interactively?]

nsext_inter: yes/no (no) [run nsextract interactively?]

aperture: 1-40 (6) [extraction aperture, pixels (+/-)]

extras: yes/no (no) [extract full-slit spectrum and in steps along the slit?]

columns: 1-40 (6) [width of each extraction step along the slit (pixels)]

continuum_inter: yes/no (no) [interactive continuum fiting?]

telluric_inter: yes/no (no) [run telluric interactively?]

hlines: vega/linefit_auto/linefit_manual/vega_tweak/linefit_tweak: none (linefit_auto) [technique for removing

H lines from standard star]

offsets: no/manual (no) [enter specplot, edit order scaling?]

shift_to_rest: yes/no (no) [look up redshift and shift to rest frame?]

gemiraf_ver: 1.11.1/1.12beta (1.12beta) [User’s Gemini IRAF version]

ureka: yes/no (yes) [Use Ureka installation of IRAF/PyRAF/python?]

These are explained in more detail in the relevant sections of this document. XDGNIRS

is often able to produce a useful quick-look spectrum with no user intervention. Fairly fre-

quently, however, the user will need to adjust the radiation event detection thresholds before

a sensible result is obtained. Science-quality reductions will normally also involve interactive

removal of intrinsic lines in the telluric standard, and perhaps interactive optimisation of

the telluric line removal. To really get to know your data, try saying “yes” to all interactive

options. Error handling is somewhat ad-hoc and limited; we have attempted to make the

script stop tidily upon encountering problems that we have encountered during testing, but

that may not always be the case.

A full reduction takes in the region of 30 minutes on REM’s Mac Pro (and about 14

minutes on the faster computers that apparently exist in Brazil). The code can be interrupted

at any time with a ctrl-C. Each time this happens a “tail” process will be left running, and

a sufficient number of these can eventually crash your computer. The tail processes can be

killed using “killall tail”, for example.

– 8 –

5. The Reduction Steps

In the rest of this document we’ll use an observation of NGC 1167 (a nearby galaxy

with emission filling the slit) to illustrate the reduction. See §2 for the location of the script

and example data.

It would probably be a good idea to initialise your IRAF uparm directory (i.e., remove

all the files from it) before starting the process. We will then create an input file list and

then run XDGNIRS like this (on your command line, not from within IRAF/PyRAF). We’ll

assume that you’re in some directory in which you want to reduce the data, and the raw

files are in ../example data:

ls ../example_data/*fits > temp

cat temp | sed "s/..\/example_data\///g" > inputfiles.lst

xdpiped inputfiles.lst step=0 rawdir=../example_data gemiraf_ver=1.12beta

shift_to_rest=yes extras=yes

Also – if you are not using Ureka – make sure to set the “gemiraf ver” flag to correspond

to the version you have installed; the script will crash if you do not do this. Because we are

starting with an input file list called “inputfiles.lst”, all the output from the code will be

written into a file called “Status inputfiles.txt”, in the PRODUCTS subdirectory (as well as

being displayed in the terminal).

The impatient reader can run the above command and then look at the resulting

“NGC1167.fits” and/or “NGC1167 data sheet.pdf” files to see the final result...

5.1. Identifying the Input Files (Step=0)

Relevant code: XDpiped.csh, ObsIdentifyXD.py, mklistXD.csh, FindSpectra.py

Relevant options: None

What it does: The first step in the process is to identify the types of files in the user-

supplied list, and make sub-lists of each type of data. This is done by the Obsidentify.py and

mklistXD.csh modules, which create various lists in the PRODUCTS subdirectory (target.lst,

QH.lst, etc.). At this point the code also figures out whether the telescope was nodded along

the slit or off to blank sky, does a few basic checks on the input data (e.g. checks for changes

in exposure time/coadds, non-existent files, etc.), and records some information that will

later be needed in step 9 (§5.10). Some of this relies on the Observation Classes being

– 9 –

correct in the headers, which should be the case if the observations were set up following the

GNIRS OT library or template observations.

What to look for: This usually works fine. If you’re curious or suspect problems, check

that the contents of the .lst files in the PRODUCTS directory are what you would expect,

based on the files you specified.

Most likely things to go wrong: Usually, the code exits with an intelligible error if

problems (missing files, bad headers, ...) are encountered. XDGNIRS expects the OBJECT

keyword to be identical in the acquisition and spectroscopy files for a given object (science

target or standard star). If that is not the case, it will think it doesn’t have the data necessary

continue, so you may have to edit the file headers. If the code exits or crashes for reasons

you don’t understand, maybe you’ve encountered a subtle feature of the headers that would

require the grep (or other) syntax in mklistXD.csh to be changed.

5.2. Pattern Noise Cleaning (Step=1)

Relevant code: XDpiped.csh, cleanir.py

Relevant options:

cleanir_flag: nfrq (f) [whether/how to run cleanir.py]

What it does: By default, all input data are “cleaned” of electronic pattern, with XD-

piped.csh acting as a wrapper for the cleanir.py script (supplied as part of the XDGNIRS

package). The available options at this stage are shown above. The default, “f”, means

that subtraction of the pattern is forced, rather than only being applied if the code detects

an improvement in the rms after the cleaning. See this web page for more information and

an explanation of the “r” and “q” options: www.gemini.edu/sciops/instruments/gnirs/data-

format-and-reduction/cleanir-removing-electronic-pattern-0.

The cleaned files are given the prefix “c” (e.g. cN20111204S0339.fits) and are saved in

the OUTPUTS subdirectory. Setting “cleanir flag=n” causes the code to skip the cleaning

step altogether, but files with the “c” prefix are still written (actually, symbolic links are

created).

What to look for: The user is advised to visually inspect the cleaned data files. A

limitation of XDGNIRS is that it uses the same option on all files. If certain files turn out to

need special treatment, workarounds include (1) excluding those files from the input file list,

or (2) deleting the cleaned file, running cleanir.py on it outside the pipeline, and restarting

XDGNIRS from step 2 (§5.3).

– 10 –

Most likely things to go wrong: The cleaning script generally works well, but can

occasionally give odd results that can affect the rest of the reduction in unexpected ways. In

the subsequent step, image statistics are written into the PRODUCTS/Status *.txt file and

if anomalous values are detected (for instance, if the flats have a very different mean value

before and after the cleaning), a warning is written into the PRODUCTS/Warnings.txt file.

If this happens, the user will need to experiment with cleanir flag until satisfactory results

are obtained.

5.2.1. Example: NGC 1167

Figures 1 and 2 show examples of files with electronic striping before and after running

the cleanir script. The weak pattern in file 343 is removed effectively by the script. The

stronger striping in file 341 is also removed, but the underlying quadrant offsets are not.

If desired, the user could kill the pipeline and rerun it with cleanir flag = fq, to make

XDGNIRS call the cleaning script with the -q (quadrant offset removal) option. In this

case the offsets have little effect on the final, combined file, so we will continue this example

reduction without attempting to remove them.

5.3. Preparing the Data (Step=2)

Relevant code: XDpiped.csh, StatsandPrepare.py


What it does: All files are now “prepared” using the nsprepare task, called by XD-

piped.csh via StatsandPrepare.py. As described on the GNIRS XD DR page, this finds the

MDF shift, flags saturated pixels, etc. The nighttime files (generally science target, stan-

dard star, flats and arcs) and daytime pinhole files are prepared separately. This is because

shifts in the x position of the XD orders have been observed to occur between data taken at

different times (the prism turret is not perfectly reproducible), and preparing the pinholes

separately from the nighttime data allows a different shift to be applied if necessary. If a shift

occurs between nighttime files, this will not be taken into account and the data may not be

reduced properly2. Currently there are no safeguards in place to prevent this in XDGNIRS,

so the user should inspect the data before running the reduction and ensure that the orders

are in the same place in the x direction, to within a few pixels, in all data taken during the

– 11 –

Fig. 1.— File N20111204S0343 contains weak electronic pattern (left) which the cleanir

script effectively removes (right). Display files N20111204S0343 and cN20111204S0343 with

z1=-5 and z2=5 to reproduce this figure and inspect the data.

– 12 –

Fig. 2.— File N20111204S0341 shows stronger electronic pattern and quadrant offsets (z1=-

5, z2=5). Calling cleanir with the -f flag removes the striping, but leaves the quadrant

offsets.

– 13 –

night.

The prepared files are given the prefix “n” (e.g. ncN20111204S0339.fits) and are saved

in the OUTPUTS subdirectory. StatsandPrepare.py then uses nsreduce to “cut” the data,

putting each spectral order into a separate file extension. The output files, which (as usual)

are stored in the OUTPUTS subdirectory, are given the prefix “r” (e.g. rncN20111204S0339.fits).

This module also measures and records some statistics on the input images, recorded

in the “Status inputfiles.txt” file in the PRODUCTS subdirectory. The statistics can be

used to determine whether any images have unusually low or high counts. For example,

the first flatfield of a sequence can sometimes be affected by the tertiary mirror not being

properly in position, resulting in low counts in that flat. The code looks for flats that

deviate by >10% from the mean of all the flats, and also raises a warning if the mean of the

flats differs substantially from the expected value (although those thresholds are currently

just a rough guess at what should be acceptable). Any warnings are summarised in the

PRODUCTS/Warnings.txt file.

What to look for: It is a good idea for the user to display one of the cut files for each

type of data using the display or nxdisplay commands, to ensure that the correct region of

the array has been placed into each extension. See the GNIRS XD DR page for examples

of good and bad cutting. All files are cut at this stage. While this is not strictly necessary

(the science target and standard star observations are cut by nsreduce later in the reduction;

§5.5), it does allow all of the cut data to be inspected in one go if desired. Also, check the

Warnings.txt file (and/or the statistics in the Status inputfiles.txt file).

Most likely things to go wrong: This usually works OK.


Figure 3 shows extension 1 (order 3) of an on-source galaxy file and a quartz halogen

flat file, after the orders have been cut into separate extensions. The slit and MDF match

up well in both cases, meaning that the correct shift of the MDF relative to the data has

been found by nsprepare.

The PRODUCTS/Status inputfiles.txt file now contains the following information:

2This occasionally happened with GNIRS at Gemini North before mid-2012, when the instrument was

opened and the grating turret worked on.

– 14 –

Fig. 3.— Left: file rncN20111204S0339[sci,1]. This is an on-source galaxy spectrum after

the orders have been “cut” into separate extensions by nsreduce. Extension 1/order 3 is

shown. Right: rncN20111204S0359[sci,1], a quartz halogen flat file after the cutting.

------------------------------------------------------------------------------

# IMAGE NPIX MEAN STDDEV MIN MAX

cN20111204S0359[1] 1046528 414.3 928.9 -360.3 7375.

cN20111204S0360[1] 1046528 419.4 941.4 -335.8 8156.

cN20111204S0361[1] 1046528 420. 942.8 -337.6 8613.

cN20111204S0362[1] 1046528 420.2 943.5 -332.8 4206.

cN20111204S0363[1] 1046528 420.3 943.9 -339. 4226.

cN20111204S0364[1] 1046528 420.6 944.4 -330.6 4234.

cN20111204S0365[1] 1046528 420.5 944.5 -335.8 4232.

cN20111204S0366[1] 1046528 420.6 944.7 -344. 8569.

cN20111204S0367[1] 1046528 420.9 945.1 -345.7 4219.

cN20111204S0368[1] 1046528 420.9 945.5 -1074. 8382.

cN20111204S0353[1] 1046528 193.3 765.1 -307.4 9009.

cN20111204S0354[1] 1046528 192.4 761.4 -293.1 9006.

cN20111204S0355[1] 1046528 190.9 754.7 -302.7 7923.

cN20111204S0356[1] 1046528 189.7 750.6 -297.7 9249.

– 15 –

cN20111204S0357[1] 1046528 188.9 748. -306.9 8469.

cN20111204S0358[1] 1046528 188.5 746.4 -299. 8485.

cN20111204S0351[1] 1046528 4.998 93.73 -363. 4763.

cN20111204S0352[1] 1046528 5.44 95.7 -355.6 8679.

cN20111204S0374[1] 1046528 42.47 294. -1353. 16415.

cN20111204S0375[1] 1046528 46.18 319.5 -1198. 16365.

cN20111204S0376[1] 1046528 42.85 307.3 -1020. 16602.

cN20111204S0377[1] 1046528 51.4 354.8 -830.3 16489.

cN20111204S0339[1] 1046528 5.27 87.73 -84.24 8016.

cN20111204S0340[1] 1046528 4.75 89.36 -89.1 7970.

cN20111204S0341[1] 1046528 2.91 95.01 -96.97 8279.

cN20111204S0342[1] 1046528 6.496 92.74 -813.2 8091.

cN20111204S0343[1] 1046528 6.306 88.48 -558.8 8118.

cN20111204S0344[1] 1046528 4.482 92.96 -318.5 7860.

cN20111204S0345[1] 1046528 4.94 93.48 -99.11 8409.

cN20111204S0346[1] 1046528 7.381 99.38 -82.71 8300.

cN20111204S0347[1] 1046528 7.949 86.54 -755.2 8339.

cN20111204S0348[1] 1046528 7.629 93.36 -359.6 8033.

cN20111204S0349[1] 1046528 5.691 85.09 -147.2 8097.

cN20111204S0350[1] 1046528 7.609 98.29 -424.9 8109.

-------------------------------------------------------------------------------

Files 359 - 368 are the quartz halogen flats, files 353 - 358 the IR flats, and 351 - 352

the arcs. All files of the same type have similar mean counts, so none would have needed to

be rejected (and the cleaning has not caused any problems that show up in the statistics).

The counts in the standard star (374-377) and galaxy (339-350) are a bit more variable, but

that is not unexpected for a poor weather programme like the one from which these data

were taken.

5.4. Making the Flatfield (Step=3)

Relevant code: XDpiped.csh, FlatFieldingXD.py

Relevant options:

nsflat_inter: yes/no [run nsflat interactively?]

What it does: The next step is to create the flatfield. As described on the GNIRS XD

DR page, two types of flatfield are obtained for XD data taken in this mode, and several

– 16 –

individual spectra are acquired for each type of flat. FlatfieldingXD.py calls nsflat which

combines the data, fits a polynomial to each order, and produces a single, normalised flatfield

file. This is done for each set of flats, then extension 1 of the IR flat and extensions 2-6 of

the QH flat are combined into a single file called “MasterFlat.fits”, saved in the OUTPUTS

directory.

It is not usually necessary to create the flats interactively, but this can be done by

setting nsflat inter=yes if desired.

What to look for: Just display the final flat and make sure it looks OK, if you like.

Most likely things to go wrong: Sometimes, for reasons best known to itself, nsflat

crashes with a fixpix floating point error. This can usually be worked round by adjusting

the value of the lthreshold parameter in nsflat. In the case of a crash, XDGNIRS reruns

nsflat with a couple of different values of that parameter. This is usually successful but if

XDGNIRS cannot get nsflat to work after three attempts, the code exits and returns an

error. We have not yet identified the underlying cause of the error, and do not currently

have a fix for this. However, nsflat seems to work more reliably when the Ureka installation

of IRAF, PyRAF and python are used (§2).


The flat created for NGC 1167 is shown in Figure 4. Running imstat on this file would

give results like this:

--> imstat MasterFlat[sci,1]

# IMAGE NPIX MEAN STDDEV MIN MAX

MasterFlat[sci,1] 183960 0.9994 0.05205 0.3507 1.147

5.5. Reducing the Science Target and Standard Star Data (Step=4, 5)

Relevant code: XDpiped.csh, ReducingXD.py

Relevant options:

tgt_thresh: (20) [threshold for radiation event removal from science target

data]

std_thresh: (50) [threshold for radiation event removal from standard star

data]

– 17 –

Fig. 4.— MasterFlat.fits, displayed using the nxdisplay task.

What it does: At this stage, the following things take place:

- The science target/standard star data are “cleaned” of radiation events

- The cleaned data are cut, flatfielded and sky-subtracted

These steps are performed by the ReducingXD.py module. The algorithm used to

identify and remove the radiation events is described on the GNIRS XD DR page. Briefly,

a “minimum” image is created from all the individual input files at each offset position, and

pixels deviating by some multiple of the noise above the minimum image are identified as

radiation events. The events are “grown” to include their haloes as well as the central pixels

and then interpolated over using IRAF’s fixpix task. The output files have the prefix “l”

(e.g. lncN20111204S0339.fits). This “despiking” is much more critical for data taken prior to

June 2012, when one of the lenses with radioactive coatings in the SB camera was replaced,

but later data do tend to contain a few spikes that can benefit from this procedure as well.

The nsreduce task is then used to cut the data, apply the flat, and do the sky sub-

traction. The resulting files have the prefix “r” (e.g. rlncN20111204S0339.fits). Sky frames

are not explicitly supplied to nsreduce, so the task attempts to identify a sky frame for each

input file based on the nod position and separation in time between the spectra.

– 18 –

The above steps are then repeated on the standard star (step=5).

What to look for: It is very important to inspect the mask files at this stage, to check that

real signal hasn’t been identified as a series of radiation events. Problems are more likely to

be encountered with bright objects and when varying weather conditions caused the signal

to vary greatly among the individual files. The masks have the prefix “gmsk” and can be

found in the OUTPUTS directory. If bad masks are found, rerun the code from step=4 (or

5) and set the tgt thresh (or std thresh) parameter to a value higher than the default (20

or 50). The default threshold is simply a value that has found to work reasonably well on a

number of test data sets, and users may wish to experiment with this parameter to find the

best “despiking” of their data. If unacceptable radiation events are found to remain after

this process, the bgmsk*pl files (or final, combined files) could be edited using the imedit

task. (Sky subtraction does remove many lower-level spikes; see §5.5.1.)

Most likely things to go wrong: The sky-object pairing algorithm in nsreduce works

well except in cases where the files are irregularly spaced, which can happen if the observer

paused for clouds to pass, for example. If nsreduce can’t identify sky files, XDGNIRS will

exit with an error. If the offending files happen to be at the start or end of the sequence,

they could simply be omitted from the input list. Otherwise the workaround would be to

run nsreduce outside the pipeline, supplying a list of sky files via the “skyimages” parameter

(see the nsreduce help). As long as the output files follow the naming convention used by

nsreduce/XDGNIRS, the code could then be restarted at the next step. The name of the

file used for sky subtraction is written into the header of each resulting data file, so this can

be checked if problems are suspected.

5.5.1. Example: NGC1167

Figure 5 shows one of the on-source galaxy spectra before and after the radiation event

removal. Most of the prominent spikes are removed by this process. As Figure 6 shows,

no real signal was flagged when the default value of the tgt thresh parameter was used.

Setting the value of that parameter to lower values causes background emission at the long-

wavelength end of the K band to be identified as a series of radiation events. More commonly,

spurious detections will extend along the target spectrum itself, or horizontally along sky

emission lines. In any case, interpolating over genuine signal will make a mess of the final

spectrum and should be avoided.

As well as the bright, obvious radiation events, a number of less prominent artefacts are

also visible in the data. These can be seen in Figure 7, which shows a close-up of the cleaned

– 19 –

Fig. 5.— On-source galaxy spectrum, before (left, ncN20111204S0339) and after (right,

lncN20111204S0339) radiation event “cleaning”. z1=-5, z2=100. The green circle identifies

an event that is interpolated over during the cleaning process.

data in Figure 5, before and after sky subtraction. Many of these disappear when the data

are sky-subtracted, and others will decrease in magnitude when the files are combined (§5.6).

Nonetheless, if users find ways of improving the process, feedback would be very welcome.

5.6. S-distortion Correction and Wavelength Calibration (Step=6)

Relevant code: XDpiped.csh, SdistorAndWLcalibXD.py

Relevant options:

nssdist_inter: yes/no (no) [run nssdist interactively?]

nswav_inter: yes/no (no) [run nswavelength interactively?]

nsfit_inter: yes/no (no) [run nsfitcoords interactively?]

What it does: XDpiped.csh now calls the SdistorAndWLcalibXD.py module, which uses

the nssdist, nswavelength, nsfitcoords and nstransform tasks to rectify and wavelength-

calibrate the data. The process that occurs here is as follows:

– 20 –

Fig. 6.— The radiation event mask, “bgmskN20111204S0339.pl” created for file 339 using

the default value of tgt thresh (left), and a lower value. Note the background signal that

has been erroneously flagged in the right-hand image.

1) nssdist is used to find and trace the pinholes in the daytime pinhole flat

2) nsfitcoords is run, using the results from nssdist, to provide the information that

nstransform needs to straighten the spectral orders

3) nstransform is run on the combined arc spectra. After this step the spectral orders

are vertical, but the slight tilt of the detector means that the spatial direction is still not

exactly parallel to the detector rows.

4) nswavelength is run on the straightened arc, to obtain the pixel-wavelength relation.

Nswavelength is called with fl median-, which causes it to call nsfitcoords and nstransform,

and (temporarily) produce an arc with horizontal arc lines.

5) nsfitcoords and nstransform are run on the straightened arc

This series of events provides the information necessary to rectify the orders in the

science target and standard star data, and then de-tilt the data so that the spatial direction

is along the detector rows. Normally, nsfitcoords would be run on each individual file to

– 21 –

Fig. 7.— Left: close-up of the data shown in Figure 5, after removal of radiation events

(z1=-5, z2=10). Right: extension 6/order 8 of the same file after sky subtraction (rl-

ncN20111204S0339). The green circle identifies the same area of the image in each case,

for orientation.

provide the information necessary for nstransform to rectify the data. However, the output

from nsfitcoords is related only to the arc and pinhole data, which are the same for every

on-sky data file. XDGNIRS therefore contains a hack to edit the headers so as to link the

data to the nsfitcoords solution from steps 2 and 5 in the database files, eliminating the need

to run the task multiple times. Nstransform is still run twice on each file, first to straighten

the orders and then to perform the slight de-tilting. This is a fairly complex process, and

the curious user can inspect the tf* and ttf* files to see how the data change at each step.

Once the individual files have been rectified, the nscombine task is used to combine

the data into a single, average file. If the telescope was nodded along the slit (rather than off

to sky), the “B” beam spectra are shifted before being combined with the “A” beam data.

The standard star is assumed to be nodded along the slit, so the “B” beam spectra are by

default shifted and combined with the “A” beam spectra. Nscombine obtains the shifts from

the offsets in the headers rather than by cross-correlation (fl cross- in nscombine). The files

generated in this step, and written to the OUTPUTS directory, are “target comb.fits” and

– 22 –

“standard comb.fits”.

What to look for: Check that the final, combined files (e.g standard comb.fits) look

reasonable. It is usually quite obvious when things have gone wrong, but sometimes more

subtle problems are present, such as the slit appearing to change length by a few pixels

along the order. A 1D, wavelength-calibrated arc spectrum is also generated at this stage

(calibrated arc.fits). This can serve as a useful sanity check of the wavelength calibration.

Most likely things to go wrong: Sometimes a wavelength solution is not found, (hope-

fully) causing the code to exit with an error. This often means that one or more of the orders

has not been transformed properly. If that is the case, that order in the transformed files

(ttf*fits), and/or final, combined files (target comb.fits, standard comb.fits) will look odd.

The fix is to restart from this step with nssdist inter=yes and/or nsfit inter=yes, as it may

be necessary to delete deviant points in nsfitcoords (as described on the GNIRS XD web

page).

If the transformed data look good, it may be that nswavelength is unable to auto-

matically find a wavelength solution. Use nswav inter=yes to see if there is an obvious

problem. Sometimes, using cleanir flag=q in step 1 (§5.2) can introduce spurious offsets

between quadrants in the arcs, which can upset nswavelength. Or maybe you just need to

manually identify some lines, for no obvious reason.


Two orders of the NGC 1167 spectra are shown in Figures 8 and 9, before and after

running nstransform on the science data. The wavelength initially increases from top to

bottom, the orders are tilted, and the sky lines are not exactly parallel to the detector rows.

In the transformed data set the wavelength scale is flipped, the orders are vertical and the

sky lines are horizontal to within 0.2 pixels. The short-wavelength end of order 8 has not

been well-rectified, but this is because the throughput in that region is low enough that even

the pinhole flats have little signal. There are no useful science data in that region, so the

odd shape is not a problem. The same is true for smaller portions of orders 6 and 7.

The final, combined data sets are shown in Figure 10. Because the telescope was nodded

off the slit for the extended galaxy observations, the B beam files were not shifted and

combined with the A beam files, and only a single positive trace is visible in the combined

2If gemiraf ver=1.11.1, XDGNIRS calls the version of nssdist that was released with v1.12beta, which is

more robust than the previous version. The nssdist.cl code is included with the XDGNIRS files.

– 23 –

Fig. 8.— Extension1/order3 before and after transforming (files rlncN20111204S0339, ttfrl-

ncN20111204S0339; z1=-5, z2=20).

file. A much smaller nod was used for the standard star, and the positive spectrum is flanked

by a pair of negative ones. As the B beam spectra were shifted before the files were combined,

the positive spectrum contains all the useful science data and only a single spectrum will be

extracted (§5.7).

5.7. Extracting the Spectra (Step=7)

Relevant code: XDpiped.csh, Extraction.py

Relevant options:

nsext_inter: yes/no (no) [run nsextract interactively?]

aperture: 1-40 (6) [extraction aperture (+/-)]


columns: 1-40 (6) [width of each extraction step along the slit (pixels)]

What this does: Next, 1D spectra are extracted from each order. By default an aper-

– 24 –

Fig. 9.— As for Figure 8, but for extension 6/order 8.

ture of ±6 pixels (±0.9′′) is used; this can be changed on the command line. The standard

star and science target spectra extracted in this way are given the prefix “v” (e.g. “vtar-

get comb.fits”). If “extras=yes”, a spectrum is also extracted in an aperture of ±23 pixels

(the entire slit), and in steps along the slit3. These spectra have the prefix “a” and “s”,

respectively (e.g. atarget comb.fits, s1target comb.fits, s2target comb.fits; numbers indicate

the step along the slit starting from lower pixel numbers).The full-slit extraction may fail if

the target is not located near the centre of the slit, and the full-slit extraction is intended

for extended objects in which the “B” beam was nodded off the slit. If these conditions do

not apply to your data set, using “extras=no” is highly recommended. If “extras=yes”, the

full-slit and stepwise extractions are only done for the science target, not the standard star.

As the spectra are now vertical, nsextract simply locates the peak and extracts the

number of columns corresponding to the “aperture” parameter. IRAF’s “apall” task is not

called and no spectral tracing is performed. This works well enough for bright objects,

but faint objects will require nsext inter=yes, so the aperture location can be verified and

adjusted if necessary.

Before the step-by-step extraction is performed (if extras=yes), the science target spec-

trum is traced. This is because the spatial structure of some science targets can vary with

– 25 –

Fig. 10.— Extension 1/order 3 of the combined galaxy and standard star data sets.

wavelength. When nswavelength searches for the peak of the spectrum in the spatial direc-

tion, it looks near the centre of the (useful) wavelength range in each order. By tracing the

spectrum along the slit, we ensure that the end of each order “lines up” with the start of

the next one, avoiding potential mismatches in flux between orders. The trace determined

at this point is then used as a reference for the step-by-step extraction. This may not be

appropriate for all intended uses of the data.

What to look for: You might like to look at the extracted orders of the science target and

standard star, the vtarget comb and vstandard comb spectra.

Most likely things to go wrong: This step relies on the automatic peak-finding algo-

rithm in nsextract. This will not work for faint sources or if for some reason there are

spikes/artefacts in inconvenient places. If you suspect that the wrong columns have been

extracted, try nsext inter=yes to display the cross-cut and adjust the apertures as appropri-

ate. The full-slit and stepwise extractions have not been extensively tested. If they appear

3If gemiraf ver=1.11.1, XDGNIRS calls the version of nsextract that was released with v1.12beta. This

version fixes an undesirable feature in the interpretation of the “upper” and “lower” parameters in apall that

previously prevented extracting all the way along the slit.

– 26 –

Fig. 11.— Spectra extracted from extension 1/order3 of the standard star (above, vstan-

dard comb.fits) and galaxy (vtarget comb.fits).

to be causing problems, try extras=no.


The spectra extracted from extension 1/order 3 of the galaxy and standard star data

are shown in Figure 11. Further examples, including the remaining orders, can be found on

the GNIRS XD DR page.

5.8. Telluric Line Cancellation and Flux Calibration (Step=8)

Relevant code: XDpiped.csh, mag2mass.py, FluxCalibXD.py

Relevant options:


continuum_inter: yes/no (no) [interactive continuum fitting?]

telluric_inter: yes/no (no) [run telluric interactively?]

– 27 –

hlines: vega/linefit_auto/linefit_manual/vega_tweak/linefit_tweak

none (linefit_auto) [technique for removing H lines from standard star]

What it does: A number of things are done in this step, and this is where user interaction

is most likely to be needed. The code:

1) Queries SIMBAD and the 2MASS catalogue to find the spectral type and IR mag-

nitudes of the standard star. If no IR magnitudes are found, only a relative flux calibration

will be performed.

2) Removes intrinsic H absorption (or other) lines from the spectrum of the standard

star

3) Corrects the science target for telluric absorption, using the line-free standard star

spectrum

4) Multiplies the telluric-corrected orders by a blackbody spectrum appropriate for

the spectral type of the standard star, and applies a rough absolute flux scaling if the K

magnitude of the standard star is known

If extras=yes, steps 4 is carried out for the full-slit and step-by-step extractions (§5.7)

as well as the “standard” aperture extraction. The process is discussed in more detail in the

following sections.

5.8.1. Finding magnitude and redshift information

XDGNIRS looks for a file called OUTPUTS/std star.txt, containing the spectral type

and IR magnitudes of the standard star. The temperature corresponding to the spectral

type of the standard star is then looked up in the “starstemp.txt” file supplied with the

code4. If OUTPUTS/std star.txt does not exist, SIMBAD and 2MASS are queried for this

information. This implies that the user must have an internet connection the first time the

code is run for a given data set. OUTPUTS/std star.txt is not deleted when the code is rerun,

though, so subsequent runs of the code can be done when the user is offline. Alternatively,

if this information is already known, the user can create the file following the example of the

file created for NGC 1167:

k K 8.005 9701

h H 7.981 9701

j J 7.887 9701

j J 7.887 9701

– 28 –

i J 7.887 9701

i J 7.887 9701

Similarly, if shift to rest=yes, XDGNIRS queries NED for the science target’s redshift.

The redshift is written to OUTPUTS/redshift.txt, like this:

0.01644

Again, the file can be created by the user if necessary.

5.8.2. Removing the lines in the standard star

XDGNIRS provides a few options for removing intrinsic lines in the telluric standard

star. The automatic options assume that an early A-type star was observed, and attempt

to remove the H absorption lines in the spectrum.

• Vega method: if hlines=vega, the telluric task is used to shift and scale a model Vega

spectrum (Figure 12). Whether or not this is done interactively depends on the telluric inter

flag. In either case the shift and scale values are written to PRODUCTS/telluric hlines.txt,

so the user can later reproduce what they did manually, if desired.

• Line fitting method: if hlines=linefit auto, the bplot task is used, together with a

list of start and end wavelengths of the H lines, to fit and subtract Lorentz profiles. The

line wavelengths are given by the cur*txt files supplied with the code. They were roughly

derived by eye, and the user could edit them if desired. To interactively fit and subtract

Gaussian, Lorentz or multiple profiles, set hlines=linefit manual. This will call splot rather

than bplot, so the user can use the “k”, “l” and “-” keys to fit and subtract a Lorentz profile,

for example. The code expects the user to use the “i” key to write out the lineless spectrum

created for each order. As well as being useful for optimising the Balmer line removal, this

may be a helpful option if the telluric standard wasn’t an A-type star. And this can also be

an opportunity to interpolate over obvious spikes, if appropriate.

Example of using the manual line fitting method:

- hit “k” at the left-hand side of the line

- hit “l” at the right-hand side of the line to fit a Lorentz profile

4From www.pas.rochester.edu/∼emamajek/EEM dwarf UBVIJHK colors Teff.dat

– 29 –

Fig. 12.— Vega model spectrum, orders 3-8.

- hit “-” at the right-hand side

- hit “-” at the left-hand side to subtract the Lorentz profile

- hit “r” to see the spectrum minus the fit

- hit “i” write the corrected spectrum to a file for later use by XDGNIRS

- hit “q” to quit and move on to the next order

Figure 12, which shows the Vega model spectrum used if hlines=vega, may help the

user distinguish between H lines and telluric absorption lines in their data.

The behaviour of splot can be kind of a pain. If you think the code is hung and isn’t

doing anything, make sure to click in the grey box at the bottom of the splot window, making

sure that the cursor is active, before typing commands there. Splot should not prompt for

the name of an output spectrum. If it does, you’ve done something wrong.

• Vega tweak, linefit tweak: With these options, the Vega or Lorentz fitting algorithm

is run first, then the code enters splot so the user can inspect and edit each order. If no

changes are necessary, simply hit “q” to quit and the next order will be shown. To edit the

spectra, take a look at the instructions in the previous bullet.

• None: if hlines=none, the code does not attempt to remove any intrinsic lines from

the standard star

– 30 –

5.8.3. Correcting for telluric absorption

The first step in the telluric line removal is to fit a continuum to each order of the line-

free standard star spectrum that was created in the previous step, and divide the spectra by

the fits. This appears to give better results than using the spectra directly. The continuum

fitting seems to work well without intervention, but the continuum inter=yes option can be

used to do it interactively if desired.

The telluric task then takes the continuum-divided spectra of the standard star, and

the science target spectra, and finds the shift and scaling that (hopefully) best take out the

telluric lines in the science target. This can be done interactively by setting telluric inter=yes

(note that, if telluric inter=yes and hlines=vega, both calls to telluric will be interactive;

see §5.8.2). The shift and scale values are written to PRODUCTS/telluric telluric.txt so

that any manual adjustment of them can be reproduced later, if desired.

At this stage, the telluric-corrected science target spectrum has some arbitrary overall

shape dictated by the continuum fit to the standard star, and the telluric task has introduced

a normalisation factor that differs from order to order. The normalisation is taken out and

the science target spectra are divided by the standard star continuum fit.

5.8.4. Flux calibration

Each order is then multiplied by a blackbody function of the temperature specified in

the std star.txt file (§5.8.1). If the K band magnitude of the standard star is known, the

blackbody function for extension 1/order 3 is scaled to that value (converted to Fλ), providing

a very rough absolute flux calibration. If not, only relative flux calibration is performed.

What to look for: You probably just want to see whether the H line residuals (if using an

A-type standard) and telluric line cancellation are to your satisfaction. Look at the flam*fits

files (or wait until the end and inspect the final spectrum).

Most likely things to go wrong: Sometimes, if the code is interrupted at just the wrong

time (or for other reasons we don’t understand), the OUTPUTS/std star.txt can exist but

not contain the right (or any) information. If the code crashes at or after step 8, try deleting

that file and rerunning from step 8. Also, the mag2mass.py code can cause errors if the

html returned by the SIMBAD/2MASS queries isn’t as expected. We have implemented

some protection against that, but probably haven’t thought of everything. So if you see

something like ”numi: variable not defined” in the terminal, that could be the problem.

– 31 –

Fig. 13.— Rest-frame spectra of NGC 1167. The red spectrum shows the spectrum generated

using the default automatic Lorentz profile line removal method, while the white spectrum

shows the spectrum obtained using the automatic Vega model method.


Figure 13 compares the results of two methods of removing the H absorption lines in the

standard star, as described in §5.8.2. Different residual lines can be seen in each case. Both

methods result in some improvement, but better results could be obtained by repeating this

step interactively. The second spectrum was obtained by running the following command

once the initial reduction was complete:

xdpiped inputfiles.lst step=8 rawdir=../example_data shift_to_rest=yes

extras=yes hlines=vega_auto

Finally, comparing Figure 13 with Figure 11 shows that the automatic telluric correction

has worked quite well, at least in the K band.

5.9. Combining the Orders (Step=9)

Relevant code: XDpiped.csh, redshift.py, CombineOrders.py

Relevant options:

offsets: no/manual (no) [enter specplot, edit order scaling?]

– 32 –

shift_to_rest: yes/no (no) [look up redshift and shift to rest frame?]

What it does: The individual orders are combined using odcombine. A plot of the indi-

vidual orders, OUTPUTS/Orders.pdf, is generated so the user can look for offsets between

orders. If significant offsets are found, the code can be restarted at this step using the off-

sets=manual option. This takes the user into specplot, so the scaling of individual orders

can be adjusted. We suggest using this option in the following way:

1) First, set the colour of each order using commands like :color[2] 2 (to make the second

spectrum red, for example). You’ll need to hit ”r” to redraw the plot and see the new colour

scheme. This makes it easier to see if any orders are offset in y.

2) Next, put the cursor near any order you want to scale, and type :scale 0.9 (for

example). It’s important to know that specplot applies whatever command you give it to

the spectrum nearest the cursor.

3) Repeat the previous step until you’re satisfied, then hit “q” to quit. There’s no need

to write out the edited spectra, the final scaling is written into a log file by specplot and the

code uses that information to scale the orders.

Especially in the higher orders, the throughput is very low in certain regions of the

spectrum. The code deals with this by only passing restricted ranges of data to odcombine

for combining. These pieces are specified by a file that is written into the OUTPUTS

directory the first time the code is run. If the orders.pdf file suggests that it would be better

to exclude (or include) more of any order, the user can proceed like this:

1) Look for the “goodbits.txt” file in the OUTPUTS directory. That file contains the

regions of each order used in the final, combined spectrum, starting with order 3/extension

1. Make a guess at the region you would like to use, and edit and save the file.

2) Run the code again, and see if you like the orders.pdf file better

3) Repeat until satisfied. To revert to the original values, simply delete the OUT-

PUTS/goodbits file and rerun the code from this step.

The final spectrum is given a name based on the target name in the original file headers.

If shift to rest=yes, a rest-frame spectrum is generated based on the redshift obtained by

the redshift.py code, which queries NED. The spectra are also written to .txt files at this

stage.

Most likely things to go wrong: Special characters in the filename can cause the code

to fail. Currently the only protection against this is for the “/” character, which is replaced

with “ ”. Interorder offsets are most common between orders 7/8 (the shortest-wavelength

– 33 –

1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4µm, observed

0.0

0.2

0.4

0.6

0.8

1.0

F λ

1e 15

Fig. 14.— The “orders.pdf” file generated for NGC 1167.

orders) and are usually of the order of 10% or less. Large interorder offsets usually indicate

that something has gone wrong; probably one of the orders has been badly rectified, or

maybe the correct columns weren’t extracted. See sections 5.6 and 5.7 for what to do if this

happens.


The “orders.pdf” plot generated for NGC 1167 is shown in Figure 14; evidently there

are no large offsets between orders in this reduction.

– 34 –

5.10. Science Target Data Sheets (Step=10)

Relevant code: XDpiped.csh, GettingInfoXD.py, SavingInfoXD.py


What it does: This part of the code makes a pdf file showing the final spectrum and

recording some information such as S/N, airmass, etc.

Most likely things to go wrong: This step is intended to help the Palomar XD group

interpret the reduced data and may not work well on other data sets. If this step causes

problems, simply run the code with “stop=9” and use your own routines to plot the .txt files

output in the previous step. The total counts and FWHM recorded in the data sheet may

not be correct for all data sets.


Figure 15 shows the final data sheet created for this galaxy. This spectrum was obtained

with no manual intervention (except, arguably, inspecting the radiation event masks as

described in §5.5). Various aspects of the reduction could be improved by re-running steps

interactively (notably the standard star H line removal), but the overall result is a useful

quick-look spectrum that could be used for planning future observations, for example.

6. Usage/Syntax Examples

These examples assume that the initial file list is called “inputfiles.lst” and that Ureka

is used.

• Reduce a data set as far as flatfielding and sky subtraction, inspect the radiation events,

and then restart with a different threshold value:

xdpiped inputfiles.lst step=0 stop=5 rawdir=../example_data

[display and assess the bgmsk*pl files, decide on new thresholds]

xdpiped inputfiles.lst step=4 stop=5 rawdir=../example_data tgt_thresh=30

std_thresh=60

[repeat until masks are satisfactory, then restart]

xdpiped inputfiles.lst step=6 rawdir=../example_data

– 35 –

11B-Q-111 Total counts, K FWHM ("), K S/N, 2.1-2.2 um Airmass HA

NGC1167 271 1.04 36 1.38 +03:07hip17971(A0V) 9396 0.46 61 1.35 +03:04

11B-Q-111 Slit, Par. Ang Diff IQ CC WV SB

NGC1167 97 ◦ , -81 ◦ 178 ◦ Any 80 Any 50

hip17971(A0V) 97 ◦ , -86 ◦ 183 ◦ Any 80 Any 20

XDGNIRS, v1.5, Mar 26 16:51:22 HST 2013

1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4Rest wavelength, µm

0.0

0.2

0.4

0.6

0.8

10−

15ergcm

−2s−

1◦ A−

1 NGC1167

Fig. 15.— The “data sheet” generated for NGC 1167. The dashed blue line is a model

Vega spectrum, intended to point out the locations where imperfectly removed lines in the

standard star could be mistaken for real features. The “messy” regions around 1.4 and 1.8

µm correspond to areas of of low atmospheric transmission.

• Fully reduce a data set, then improve the standard star line removal step using the semi-

interactive line fitting method:

xdpiped inputfiles.lst step=0 rawdir=../example_data

xdpiped inputfiles.lst step=8 rawdir=../example_data hlines=linefit_tweak

– 36 –

7. Troubleshooting and Support

XDGNIRS is not officially supported by Gemini. File a helpdesk request for general

questions about GNIRS data and reduction. XDGNIRS-specific questions can be addressed

to Rachel Mason, who will attempt to answer on a best-efforts basis, other commitments

permitting.

8. History

v2.0 - first “public” release?

xdgnirs v2.0

Documents