nicer data introduc0on 2017-08-23 craig markwardt
TRANSCRIPT
![Page 1: NICER Data Introduc0on 2017-08-23 Craig Markwardt](https://reader031.vdocuments.site/reader031/viewer/2022020622/61edf0b46bea0d4550337c2b/html5/thumbnails/1.jpg)
1
NICERDataIntroduc0on
2017-08-23CraigMarkwardt
GSFC
![Page 2: NICER Data Introduc0on 2017-08-23 Craig Markwardt](https://reader031.vdocuments.site/reader031/viewer/2022020622/61edf0b46bea0d4550337c2b/html5/thumbnails/2.jpg)
2
• NICERprojectdeliveringfirst"unvalidated"datasetstoteamforevalua=onandvalida=on
• Morethan1600datasetsinthearchivedelivered
• Contentsofthefilearedescribed• Caveats
![Page 3: NICER Data Introduc0on 2017-08-23 Craig Markwardt](https://reader031.vdocuments.site/reader031/viewer/2022020622/61edf0b46bea0d4550337c2b/html5/thumbnails/3.jpg)
3
• HEASoGversion6.22hasbeenreleased.NICERsoGwareisincluded• NICERcalibra=ondataisnowpartofHEASARCCALDB• Changesto“Ini=alReleaseData”
– Namingofdatasetsnolonghastargetnameindirectoryname– The“UFA”eventfile,nominallyunfiltered,nowhasbeenlightly
filteredfordiskspaceandperformancereasons.(Undershootsandovershootsremoved)
– Filterfile(auxil/ni*.mkf)hasmanynewcolumns.See‘[elpniprefilter’fordescrip=ons.Newtask‘nimake=me’canbeusedforstandard=mescreening.
– A]tudeprocessingisnowimproved.Bothstartracker(ST)andinstrument(INST)a]tudesarepreserved,butSTa]tudeisnowusedforprocessing.
– OrbitfilenowincludesSinglePointSolu=onGPSsolu=onforrarecaseswhenfilteredGEONSsolu=onisnotavailable.ThisextensionisORB
– Metadatakeywordsarepopulated– Processinglogfileiscomplete
![Page 4: NICER Data Introduc0on 2017-08-23 Craig Markwardt](https://reader031.vdocuments.site/reader031/viewer/2022020622/61edf0b46bea0d4550337c2b/html5/thumbnails/4.jpg)
4
• ChangestoNICEReventdata– BothPIandPI_FASTpulseheightsarecalibrated– NeweventfilecolumnMPU_UNDER_COUNTusedforcalibra=onpurposes
– Merged-MPUeventfileGTIiscombinedwith“AND”logic(intersec=onofindividualMPUGTIs)
– IndividualMPUGTIsarepreservedinthemergedfileasGTI_MPUnextensions
– Incleanedeventfile,thereisaGTI_SELextensionwhichistheinput=meselec=onGTIusedforfinalscreening,forreproducibilityreasons
![Page 5: NICER Data Introduc0on 2017-08-23 Craig Markwardt](https://reader031.vdocuments.site/reader031/viewer/2022020622/61edf0b46bea0d4550337c2b/html5/thumbnails/5.jpg)
5
• Each“observa=onsegment”willbeinaseparatedirectoryiden=fiedbyits10-digitobserva=onID– Observa=onIDisuniquetoasegment– Scienceopsmaysplitsegmentsat~dailyboundaries– Anobserva=onsegmentmaycontainmorethanoneNICERorbit
• Capturedineventfilegood=meintervals(GTIs)• Directorylayout
– NNNNNNNNNN/(observa=ondirectory)• x=/ -X-rayTimingInstrumentdata
– event_uf/ -unfilteredX-rayevents– event_cl/ -cleaned&calibratedX-rayevents– hk/ -XTIhousekeepingfiles
• auxil/ “Mission-level”orbit,a]tude&HK• log/ Processinglogfiles
![Page 6: NICER Data Introduc0on 2017-08-23 Craig Markwardt](https://reader031.vdocuments.site/reader031/viewer/2022020622/61edf0b46bea0d4550337c2b/html5/thumbnails/6.jpg)
6
• FundamentalscienceproductisX-rayeventdata– Level1uncalibratedunfilteredeventdatastoredin
x=/event_uf– Level2calibratedandcalibrated+screeneddataarestoredin
x=/event_cl– Level1&higherdataisdeliveredwitheveryobserva=on
• Theeventfilenamingschemeis,– niNNNNNNNNNN_0mpuN_UU.evt
• NisMPUnumber0-6individually• N=7indicatesallMPUscombined• UUindicateslevel
– uf=unfiltered– ufa=calibratedunfiltered(“lightlyfiltered”toremoveoverandundershoots)– cl=calibratedandfiltered(cleaned)
– Example:ni1707030136_0mpu7_cl.evt• Obsid1707030136,cleaneddata,allMPUscombined
![Page 7: NICER Data Introduc0on 2017-08-23 Craig Markwardt](https://reader031.vdocuments.site/reader031/viewer/2022020622/61edf0b46bea0d4550337c2b/html5/thumbnails/7.jpg)
7
• Therearetwoextensionsofinterest– Extension1:EVENTS–X-rayeventlist– Extension2:GTI–GoodTimeIntervaltable
• NICERfollowsstandardizedformatsforeventlistsandGTIs,soX-rayastronomersshouldbefamiliarwiththem.(Seenextpage)
• Level1“uf”filesareoriginaldatastreamfromeachMPU– Therearesevenfiles,oneforeachMPU
• Level2“ufa”and“cl”fileshaveall7MPUsmergedintoonefile,calibra=onsapplied
![Page 8: NICER Data Introduc0on 2017-08-23 Craig Markwardt](https://reader031.vdocuments.site/reader031/viewer/2022020622/61edf0b46bea0d4550337c2b/html5/thumbnails/8.jpg)
8
HDU 2 EVENTS BinTable 9 cols x 16744802 rows
Col Name Format[Units](Range) Comment 1 TIME 1D [s] Time of events 2 RAWX 1B [pixel] (0:7) Event X position RAW coordinates 3 RAWY 1B [pixel] (0:6) Event Y position RAW coordinates 4 PHA 1I [chan] (0:4095) Slow Pulse Height Analyzer 5 PHA_FAST 1I [chan] (0:4095) Fast Pulse Height Analyzer 6 DET_ID 1B Detector ID number - 10*MPU+FPM 7 DEADTIME 1B [s] Event dead time 8 EVENT_FLAGS 8X MPU Event Flags 9 TICK 1K MPU tick count of event 10 PI 1I Slow Pulse Invariant 11 PI_FAST 1I Fast Pulse Invariant 12 MPU_A_TEMP 1E MPU Analog Temperature 13 MPU_UNDER_COUNT 1J MPU undershoot rate 14 PI_RATIO 1E Ratio PI/PI_FAST
HDU 3 GTI BinTable 2 cols x 13 rows
Col Name Format[Units](Range) Comment 1 START 1D [s] GTI start time 2 STOP 1D [s] GTI stop time
HDU 4 PPS_TREND BinTable 3 cols x 11500 rows
Col Name Format[Units](Range) Comment 1 TIME 1D [s] GPS reported time at 1-PPS 2 TICK 1K MPU tick count at 1-PPS 3 PKT_TICK 1K MPU tick count at time of X-ray packet accumula
GREEN=Level1→2Addi=ons
![Page 9: NICER Data Introduc0on 2017-08-23 Craig Markwardt](https://reader031.vdocuments.site/reader031/viewer/2022020622/61edf0b46bea0d4550337c2b/html5/thumbnails/9.jpg)
9
• Thereare56modulesarrangedasfollows(seefollowingpage):– sevenMPUslabeled0-6
– eightFPMsperMPU,labeled0-7• ThemodulesarelabeledwithasingleintegerDET_IDintheeventlist:
– DET_ID=10xMPU+1xFPM– Example:27meansMPU2,FPM7
![Page 10: NICER Data Introduc0on 2017-08-23 Craig Markwardt](https://reader031.vdocuments.site/reader031/viewer/2022020622/61edf0b46bea0d4550337c2b/html5/thumbnails/10.jpg)
10
1-100
1-201
1-302
1-403
1-504
1-706
1-605
2-110
2-211
2-312
2-413
2-514
1-807
2-615
3-120
3-221
3-322
3-423
3-524
2-716
3-625
4-130
4-231
4-332
4-433
4-534
2-817
3-726
5-140
5-241
5-3 42
5-443
5-544
3-827
4-635
6-150
6-251
6-352
6-453
5-645
4-837
4-736
7-160
7-362
7-564
7-766
6-554
5-847
5-746
7-261
7-463
7-665
7-867
6-655
6-857
6-756
+Y
+X
1-7
06
MechanicalDrawingPosi=on
ScienceDetectorID
MPU(0-6)
FPM(0-7)
![Page 11: NICER Data Introduc0on 2017-08-23 Craig Markwardt](https://reader031.vdocuments.site/reader031/viewer/2022020622/61edf0b46bea0d4550337c2b/html5/thumbnails/11.jpg)
11
• TimeisreportedintheTIMEcolumn• TIMEiselapsedTTsecondssincetheepoch
2014-01-01T00:00:00UTC• TSTARTandTSTOPreportthestartandstopofgood=me• EXPOSUREistotallive=me• ConversionofNICER=mestampstoabsolute=meinMJD
canbedonewiththefollowing:– MJD(TT)=MJDREFI+MJDREFF+(TIMEZERO+TIME)/86400
– MJD(UTC)=MJD(TT)+LEAP_INIT• BUT:SeetheCaveatspagesforimportantlimita0ons!
CurrentlyLEAP_INITisnotproperlyfilledsoyouwillneedtouseLEAP_INIT=2manually.
![Page 12: NICER Data Introduc0on 2017-08-23 Craig Markwardt](https://reader031.vdocuments.site/reader031/viewer/2022020622/61edf0b46bea0d4550337c2b/html5/thumbnails/12.jpg)
12
• UncalibratedLevel1incolumnsPHAandPHA_FAST– EachMPUandFPMhasdifferentPHAscale!– Thus,useofuncalibratedPHAisoflimitedusebyscien=sts
• CalibratedLevel2pulseheightinPIandPI_FAST(“ufa”and“cl”files)– Everymoduleisplacedonsameenergyscale!
• PI=(ENERGY/10eV)– Example:PI=110correspondsto1.10keV
• NICERcalibrated“ufa”andcleaned“cl”fileshavethepulseheightcalibra=onappliedforbothPIandPI_FAST
• NOTE:theNICERresponsebelow~250eVcanbecomplicated;thestandardscreeningremoveseventsbelow200eV.
![Page 13: NICER Data Introduc0on 2017-08-23 Craig Markwardt](https://reader031.vdocuments.site/reader031/viewer/2022020622/61edf0b46bea0d4550337c2b/html5/thumbnails/13.jpg)
13
• TheNICERteamhasnowclearlyiden=fiedthedifferencebetweenPIandPI_FASTasadiscriminatorofbackgroundevents– nonX-raysthatinteractinouterregionsofthedetectorscreatedifferentchargecloudsthanconcentratedX-rays,whichresultindifferentpulseheightsinslowandfastchains
• AcolumnnamedPI_RATIOisdefined PI_RATIO=PI/PI_FAST
• TheNICERteamwillprovideimprovedguidancehowtousethiscolumnforbackgroundfiltering
![Page 14: NICER Data Introduc0on 2017-08-23 Craig Markwardt](https://reader031.vdocuments.site/reader031/viewer/2022020622/61edf0b46bea0d4550337c2b/html5/thumbnails/14.jpg)
14
• Gaincalibra=onofbothchainsallowstocalculatePI_RATIO=PI/PI_FAST,whichisanindicatorofbackgroundevents
• Blackcutabove,includedinnicermergeclean,willexcludebackgroundevents
PI_R
ATIO
PI=Energy/10eV
Background
RealX-rayEvents
![Page 15: NICER Data Introduc0on 2017-08-23 Craig Markwardt](https://reader031.vdocuments.site/reader031/viewer/2022020622/61edf0b46bea0d4550337c2b/html5/thumbnails/15.jpg)
15
• ThereareseveralNON-X-rayeventtypes.TheseareindicatedintheEVENT_FLAGSbitcolumn…(‘x’meansdon’tcare)
– EVENT_FLAGS==xxxxx1:“undershoot”reset– EVENT_FLAGS==xxxx1x:“overshoot”reset– EVENT_FLAGS==xxx1xx:soGwaresample
– EVENT_FLAGS==xx1xxx:fastsignalchaintriggered– EVENT_FLAGS==x1xxxx:slowsignalchaintriggered– EVENT_FLAGS==1xxxxx:firsteventinMPUpacket
• TherecanbeMANY,MANYmorenonX-raysthanX-rays!
![Page 16: NICER Data Introduc0on 2017-08-23 Craig Markwardt](https://reader031.vdocuments.site/reader031/viewer/2022020622/61edf0b46bea0d4550337c2b/html5/thumbnails/16.jpg)
16
• ToretrievevalidX-rays,usethefollowingfilter:– EVENT_FLAGS=x1x000
(requireslowchain,don’tcarefastchain,rejectresets)
– Thisscreeningcriteriahasbeenappliedtogeneratethedeliveredcleaned“cl”file.
• ToretrieveFAST+SLOWevents,usethefollowingfilter:– EVENT_FLAGS=x11000
(requirefast+slowchain,rejectresets)
• Example:– niextract-events ‘input.evt[EVENT_FLAGS=bx11000]’ output.evt gti=GTI
![Page 17: NICER Data Introduc0on 2017-08-23 Craig Markwardt](https://reader031.vdocuments.site/reader031/viewer/2022020622/61edf0b46bea0d4550337c2b/html5/thumbnails/17.jpg)
17
• Wecurrentlyrecommendthatanalystsusethetoolnicermergecleantoscreendata.Thistaskwillautoma=callyremovenonX-rays.
• Thetaskwillautoma=callyscreenoutdatabelow200eV.
• WearecurrentlyrecommendingthatuserswillprobablyneedtorejectcountswithPI<250eV.
PSRJ0437Spectrum
TargetSpectrum
~250eV
![Page 18: NICER Data Introduc0on 2017-08-23 Craig Markwardt](https://reader031.vdocuments.site/reader031/viewer/2022020622/61edf0b46bea0d4550337c2b/html5/thumbnails/18.jpg)
18
• “Image”columnsRAWX/RAWYcanbeusedtomakearudimentarydetectorplaneimage(use“fv”)
• DEADTIMEisinstrumentdead=measrecordedbytheMPUforeachevent.– EachMPUoperatesindependentlyfordead-=mepurposes
• TICKistheuncalibratedinstrument-recordedevent=meinunitsof40nanoseconds.– NOTEthateachMPU=cksatadifferentand=me-variablerate.UsetheTIMEcolumn.
• ThePPS_TRENDextensionishousekeepingdata
![Page 19: NICER Data Introduc0on 2017-08-23 Craig Markwardt](https://reader031.vdocuments.site/reader031/viewer/2022020622/61edf0b46bea0d4550337c2b/html5/thumbnails/19.jpg)
19
• AnysoGwarethatunderstandsbasicX-rayeventlistsshouldworkwithNICERdata.
• Youwillprobablyhavetodosomescreeningoutsideofxselecttoremovenon-X-rays.
• Youcanusexselectandextractor.– Examplexselectsession:
> read events ./ni1707030136_0mpu7_cl.evt> set binsize 1> extract curve> plot curve> set phaname PI> extract spectrum> plot spectrum
• Scien=ststypicallyusetheirown=mingsoGware!
![Page 20: NICER Data Introduc0on 2017-08-23 Craig Markwardt](https://reader031.vdocuments.site/reader031/viewer/2022020622/61edf0b46bea0d4550337c2b/html5/thumbnails/20.jpg)
20
• Thesedataares=llconsideredunvalidated,andscien=stsshouldbeonthelook-outforoddoroff-nominalbehaviors.
• Thenextpagesdescribecertainknownissuesthatdonotneedtobereported
![Page 21: NICER Data Introduc0on 2017-08-23 Craig Markwardt](https://reader031.vdocuments.site/reader031/viewer/2022020622/61edf0b46bea0d4550337c2b/html5/thumbnails/21.jpg)
21
• Ithasbeenreportedthattheremaybea1secondabsolute=mingoffsetofNICER=mestamps
• ThisoffsetwouldmanifestitselfwhencomparingNICERdatatootherobservatoriesanda1-seconddifference.
• TheNICERteamisawareofthisissueandisinves=ga=ng,butwantedtogetdataouttotheteamwithoutdelay,forthemanyscien=stswhoarenotsensi=veto1second=mingoffsets.
• Iftheissueisresolvedanda=mecorrec=onisneeded,theteamwillprovideasmalltoolneededtoadjustdatasetsin-place.
![Page 22: NICER Data Introduc0on 2017-08-23 Craig Markwardt](https://reader031.vdocuments.site/reader031/viewer/2022020622/61edf0b46bea0d4550337c2b/html5/thumbnails/22.jpg)
22
• Duringpre-launchtes=ngandflight,somedetectorscanproduceasingleeventcountwithinafewmicrosecondsofthe1secondGPS=mingpulse.
• ThisisknowntoMITas"PPSNoise,"althoughformallyitisnotnoise.
• ThereareanumberofwaysthisPPSnoisemaymanifestitself– falseX-raycountsatexactcorrelatedwith1-secondrollover
– correlateddead-=meatexact1-secondrollover– BEWAREofexact1-secondpulsarsorQPOs
• Teamisinves=ga=nghowtoscreenthese,buttherewillalwaysatleastbedead=meat1-secondrolloverboundaries
![Page 23: NICER Data Introduc0on 2017-08-23 Craig Markwardt](https://reader031.vdocuments.site/reader031/viewer/2022020622/61edf0b46bea0d4550337c2b/html5/thumbnails/23.jpg)
23
• MostofNICERanalysisisgearedtowardscreeningoutundesirableeventsforspectralor=mingprocessing.
• Formostsensi=veanalysishowever,dead-=memaybeimportant.
• Consideringdead=me,– EachMPUtriggersindependently– AllMPUeventsshouldbeconsideredtoes=matedead=me,notjustthecleanedeventlist
– EachMPUes=matesitsamountofdead=meonaper-eventbasis
• Therefore,scien=stswishingtocontempla=ngdetaileddead=meanalysisshouldgobacktotheper-MPU"uf"filesandtallydead=mefromthoseevents.
![Page 24: NICER Data Introduc0on 2017-08-23 Craig Markwardt](https://reader031.vdocuments.site/reader031/viewer/2022020622/61edf0b46bea0d4550337c2b/html5/thumbnails/24.jpg)
24
• Almostallofthemetadatakeywordsarefilledwithproperinforma=on,buttheLEAP_INITkeywordisnot.
• Thiskeywordisintendedtoaidscien=ststoconvertbetweentheTT=mesystemandUTC(seepreviousslides)
• CurrentlyLEAP_INITisfilledwithzero,whichiserroneous– ForallNICERobserva=onstodate,LEAP_INIT=2
• Scien=stsshouldtakecareun=lthisisfixed.