new static dq masks for nicmos

TIPS, Mar 19, 2009 - Tomas Dahlen 1

New static DQ masks for NICMOS

Tomas Dahlen & Elizabeth Barker

TIPS 3/19/2009

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.


NICMOS Data Quality (DQ) extension of calibrated images (*_cal.fits or *_ima.fits) contains information about which pixels may be problematic and should/may be excluded when combining dithered images.


DQ Meaning0 No known problems1 Reed-Solomon decoding error in telemetry2 Poor or uncertain Linearity correction4 Poor or uncertain Dark correction8 Poor or uncertain Flat Field correction16 Pixel affected by "grot" on the detector32 Defective (hot or cold) pixel64 Saturated pixel128 Missing data in telemetry256 Bad pixel determined by calibration512 Pixel contains Cosmic Ray1024 Pixel contains source (see Section 3.4)2048 Pixel has signal in 0th read4096 CR detected by Multidrizzle8192 User flag16384 Curvature in detector response

• Most flag values are dynamic, i.e., are set during observation/calibration.• Static flag values are DQ=16 (“grot” affected pixels) and DQ=32 (“bad” pixels).• Each NICMOS camera has its own static mask given in the header keyword MASKFILE.


• The current “post-NCS” static masks were created 2002.

• We now have a large set of calibration images obtained during NICMOS monitoring programs 2002-2008.

• We have used this data to create new static masks for the three NICMOS cameras based on better statistics.

• The temporal coverage also allows us to investigate any changes in the masks with time or detector temperature.


New static DQ masks for NICMOS


• We use long dark exposures (>1000s) to identify “Bad” pixels as pixels with a dark current deviating from what is expected.• Hot pixels are defined as pixels with excessive charge compared to surrounding pixels.• Cold pixels have extremely low(/zero) dark current (i.e., “dead” pixels).• To quantify the number of bad pixels we use the recipe in Sosey (2002, NICMOS ISR 2002-001 “ Updating the NICMOS Static Bad Pixel Masks”).


DQ=32 “Bad” pixels

1. A median dark image is made using available monitoring dark data after excluding CR affected pixels

2. A smoothed image is made and subtracted from the median image

3. The resulting image is rescaled in units of its rms4. Bad pixels are defined as pixels outside 5

Sosey (2002)

5

Since we have monitored the darks continuously from 2002 to 2008, we can look for changes with time/temperature.


DQ=32 “Bad” pixels

NIC 1 NIC 2 NIC 3

Old DQ=32 193 (0.29%) 656 (1.0%) 446 (0.68%)

2002 46 17 16

2003 49 21 21

2004 70 28 28

2005 69 24 30

2006 70 24 29

2007 66 29 34

2008 70 31 36

All new 88 (+0.13%) 40 (+0.06%) 42 (+0.06%)

Old DQ=32: Number of DQ=32 pixels in existing mask.For each year the number of additional hot pixels is tabulated.All new: Total number of different pixels flagged as hot during at least one year.

Black dots: number of DQ=32 pixels in existing mask.Red dots: number of DQ=32 after adding new bad pixels

Num

ber o

f bad

pix

els

6


DQ=32 “Bad” pixels Plots show pixels that are not flagged in existing DQ mask, but pass the criteria for being bad in this investigation.

Number of pixels that “suddenly” turn bad and can be “saved” by using multiple DQ masks is ~5 in each camera. (In addition a few bad pixels drop below the selection with time.)

We find that having multiple DQ masks to “save” at the most a few pixels per camera is not “worthwhile” and

only a single mask is created for each camera.

7


DQ=16 “grot” pixels

• The grot consists of flecks of anti-reflective paint on the detectors (that were scraped off the optical baffles between the dewars due to the expansion of the solid nitrogen in Cycle 7).

• Grot leads to areas with reduced sensitivity.

• Size of grot ranges from 25m to over 100m and since NICMOS pixels are 40m on a side, grot can affect regions of several pixels, as well as a fraction of a pixel.

The largest example of a grot region is the "battleship" feature in NIC 1 which affects approximately 35 pixels.

Flat-field image DQ mask

8


DQ=16 “grot” pixels

• The grot affects the incoming light onto the detectors.• Flats are ideal to estimate the effects of grot due to their high “uniform” counts over the whole detector.• To quantify the grot we use the recipe in NICMOS ISR 2003-003 (Schultz et al.).

1. We use a well sampled non-inverted flat-field (F160W filter).2. A smoothed flat image is subtracted from this flat.3. Grot pixels are defined as a pixels deviating more than 4 in the

subtracted image (excluding bad DQ=32 pixels).

9

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.DQ=16 “grot” pixels

Existing New

NIC1 180 170

NIC2 243 123

NIC3 249 113

Table: The number of grot pixels. The number found here is significantly lower than in the existing mask for nic2 and nic3.

To investigate this we look at the relative DQE response of the flagged pixels - this is the pixel value of a ratio image created by dividing the flat image with a smoothed flat image. “Normal” pixel has response=1.

Q: what are the characteristics of the pixels not selected using the new selection?

10


Existing New

NIC1 180 170

NIC2 243 123

NIC3 249 113

The new selection suggest that grot pixels have response ~< 0.8.The pixels that are flagged in the existing mask and NOT flagged in the new selection have a fairly normal response in the range 0.8-1.2.

Existing New Existing - New

Relative DQE response

11


Existing New

NIC1 180 170

NIC2 243 123

NIC3 249 113

The new selection suggest that grot pixels have response ~< 0.8.The pixels that are flagged in the existing mask and NOT flagged in the new selection have a fairly normal response in the range 0.8-1.2.

Furthermore, the rms of the previously flagged pixels look normal

Previously flagged

Relative DQE response

Black dots: rms of non-flagged normal pixels

Green dots: rms of previously flagged pixels

Rms look similar, green dots should be OK

We should therefore be able to “de-flag” these

Rms vs. response

12


Q: Does the number of grot pixels change with time?

A: No.But there are somewhat more flagged pixels in NIC2-2008, but these have fairly normal response 0.7-0.9. We will not include a time dependence in the grot mask (i.e., not create multiple masks).

Existing New 2002 2008

NIC1 180 170 172 174

NIC2 243 123 119 139

NIC3 249 113 113 112

As a final selection, we include all “4 pixels” from above as grot pixels, except those that have a response >0.8 and a normal signal-to-noise.

The new MASKFILEs for the post NCS era will be delivered by end of March -09.

FINAL SELECTION

13

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.New static DQ masks for NICMOS

Summary:Grot, DQ=16 Bad, DQ=32

Existing New Existing New

NIC 1 180 163 193 281

NIC 2 243 119 656 696

NIC 3 249 96 446 448

Existing:

New:

NIC 1 NIC 2 NIC 3

DQ=16, grot pixels

new static dq masks for nicmos

Documents

number of pixels

bad pixels old dq

cold pixels

bad pixels plots

surrounding pixels

rmsbad pixels

dead pixels

number of dq