EHTC Processing(Event Horizon Telescope Collaboration)

An update for the DiFX Meeting

Sept 4, 2018 in Bad Kötzting, Germany

Geoff Crew

MIT Haystack Observatory

Context

• The Event Horizon Telescope Collaboration is the organization working to image the supermassive black holes M87 and SgrA*

• (Sorry, no black hole images today, and can’t say much…)• Other than bandwidth (32 Gbps in 2017, 64 Gbps 2018++) the main

technical challenges are:• 2 correlators working collaboratively (Haystack and Bonn are comparable)• ALMA is has linear (not circular) polarizations• Correlation products to support both AIPS and HOPS pipelines

• The primary data processing pipeline for the collaboration is HOPS-based• AIPS is still in some use; and extensive consistency checks have been made

• As for the organizational challenges…

2

Institutions on the EHT Board 

Academia Sinica Institute of Astronomy and Astrophysics

University of Arizona

University of Chicago 

East Asian Observatory

Goethe­Universität

Institut de Radioastronomie Millimétrique

Large Millimeter Telescope 

Max Planck Institute for Radioastronomy

MIT Haystack Observatory

National Astronomy Observatory of Japan

Perimeter Institute for Theoretical Physics

Radboud University

Smithsonian Astrophysical Observatory

3

Affiliated Institutions

Aalto UniversityUniversiteit van AmsterdamArizona Radio ObservatoryInstituto de Astrofísica de AndalucíaInstituto Nacional de Astrofísica, Óptica y ElectrónicaInstitute for Astrophysical ResearchBoston UniversityBrandeis UniversityUniversity of California, BerkeleyCalifornia Institute of TechnologyChinese Academy of SciencesCologne UniversityUniversidad de ConcepciónCornell UniversityInstitute of High Energy PhysicsHuazhong University of Science & TechnologyUniversity of IllinoisJoint Institute for VLBI ERIC

Kavli Institute for Astronomy and AstrophysicsKorea Astronomy and Space Science InstituteLeiden UniversityUniversity of MarylandUniversity of Massachusetts AmherstMax Planck Institute for Extraterrestrial PhysicsNanjing UniversityNational Astronomical Observatories of ChinaOnsala Space ObservatoryPeking UniversityPurple Mountain ObservatoryUniversity of Science and TechnologyUniversity of Science and Technology of ChinaSeoul National UniversityShanghai Astronomical ObservatoryInstitute of Statistical MathematicsUniversity of WaterlooYunnan Observatory

4

EHTC Funding Support

5

Event Horizon Telescope in 2017, 2018, 2020 

SMA/JCMT

SMT

SMT

IRAM 30m

IRAM 30m

GLT

GLT

GLT

ALMA

ALMA/APEX

ALMA/APEX

LMT

LMTLMT

D. M

arro

ne/U

ofA

SPTSPT

SPT

SMA/JCMT

SMA/JCMT

NOEMA

EHT2017: A Large Number of Highly Significant Detections

Cal

ibra

tion 

& E

rror

 Ana

lysi

s W

G

EHT2017: J1733­1304 LMT

ALMA

SPT

Cal

ibra

tion 

& E

rror

 Ana

lysi

s W

G

EHT2017: 3C273 

LMT

ALMA

SMT

Cal

ibra

tion 

& E

rror

 Ana

lysi

s W

G

Overview of Operations with PolConvert

• Conversion tool works with both ALMA bands and could be used with other telescopes

• Full validation requires >3 hour observing runs for good parallactic angle coverage

• Operational scenario requires ALMA data analysts to reduce ALMA data for PolConvert inputs and provide them to the correlators

10

L/R

Observation ALMA Archive

VLBI Correlation Project PIPolConvert

PolConvertInputs

VLBI DataProducts

ALMA Data Products

X/Y

Polarization CalibrationReduction (QA2)

Martí-Vidal, Goddiand JAO staf

Rev 1,3,5

PolConvert 0, (ALMA QA2)

• Performed by ALMA data analysts (Ivan Martí-Vidal, Ciriaco Goddi & two JAO “friends”) who do QA2 for normal observations.

• A complication is that normal ALMA calibrations are un-phased, whereas VLBI scans are phased, and the delay handling is different so scans cannot be mixed in the analysis within CASA.

• Another complication (solved) is transfer of calibration from one project to another as required.• Flagging of data is usually necessary to keep CASA tasks from generating garbage

• QA2 script(s) (see applications/polconvert/trunk/src/QA2) are used to generate several calibration tables (provided to the correlators as a tarball):

• Antenna table, CalAppPhase table, Bandpass correction table, D-term correction table• Amplitude calibration table (Jy/K), Phase calibration table, XY phase calibration table

• These are delivered to the correlators (Bonn & Haystack) and can be used by operators there without needing to become CASA experts. Additionally, QA2 reduction products go to the PI.

11

PolConvert, 1 (the rationale)

• PolConvert which was developed by Ivan Martí-Vidal (with MPIfR) to solve the linear to circular polarization problem posed by the ALMA Phasing Project (per white-paper study by Alan Roy for the APP)

• The (easiest/least effort) recommendation was to have DiFX perform a mixed-polarization correlation and insert a conversion step afterwards that would take the result into what would have been produced if ALMA was actually used circular polarizations. Needs:

• Calibration information (mainly X-Y phase offset) from the ALMA observation• A converter for the SWIN output (do the math, Luke)• A rewriting of DiFX setup files for difx2mark4 & difx2fits

12

PolConvert, 2 (the implementation)

• For full details see: “Calibration of mixed-polarization interferometric observations: Tools for the reduction of interferometric data from elements with linear and circular polarization receivers”, I. Marti-Vidal, A. Roy, J. Conway, and A. J. Zensus, A&A 587 (2016) A143 (doi: http://dx.doi.org/10.1051/0004-6361/201526063).

• Implemented as a CASA task (to allow ease of use with ALMA observational data) and at the moment only supports ALMA (B3&B6).

• Source code in DiFX tree: applications/polconvert/trunk• Used successfully for the 9 ALMA Cycle 4 (2017) projects:

• http://eventhorizontelescope.org• https://aas.org/posts/news/2017/09/update-event-horizon- telescope• Starting 2018 soon

13

E.G. XY Phase Calibration QA2 Product

14

(From one of the commissioning data sets)

PolConvert Also Calculates Phasing Efficiency

15

(From one of the commissioning data sets)

Some Details of the PolConvert Implementation

• PolConvert integrated within DiFX trunk using automake, &c.• Polconvert is mostly written in C++ and available to Python via

shared object (_PolConvert.so) used by the CASA task machinery:• mytasks.py• polconvert_cli.py• polconvert.py• polconvert.xml• runpolconvert.py• task_polconvert.py

• Current defaults are for ALMA, but EVN support has been started.

16

One-Time Setup for Each User

• source setup-difx.sh• $DIFXCASAPATH/casa

• quit• y # yes, dammit, I really do want to quit.

• Edit $HOME/.casa/init.py to include the following so that PolConvert will be automatically available within CASA:

print 'loading polconvert’try: from os import environ execfile(environ['DIFXROOT'] + '/share/polconvert/mytasks.py')except: print 'unable to load $DIFXROOT/share/polconvert/mytasks.py’

17

Driver Scripts for Use by Correlator Operators

• Setup correlation and run (more-or-less) as normal.• ALMA products end up as mixed pol (XL, XR, YL, YR)• ALMA uses CALC, so atmospheric calculations need to be turned off for ALMA: CalcifMixed

• Three Python scripts (applications/polconvert/trunk/src/PP), use –help for some details… • prepolconvert.py

• Script to import/edit correlation inputs and outputs to a working directory• Usage: prepolconvert.py –v –s <source-directory> $jobs

• runpolconvert.py• Script actually execfile()’d within CASA• For usage, see drivepolconvert.py

• drivepolconvert.py• Script that builds (Python) command inputs for CASA that can be used to invoke CASA in a batch operational mode (one CASA invocation per VLBI

scan).• Usage: drivepolconvert.py –v –l <qa2label> $opts $jobs

• Continue with difx2fits or difx2mark4 as normal• Additional scripts to manage the entire suite are also in the DiFX tree (sites/Haystack/ehtc)

• ehtc-tarballs.sh – driver script that manufactures FITS, HOPS and fringe tarballs• ehtc-jsgrind.sh – driver script that runs the polconvert stages and make the tarballs

18

PolConvert On-the-Fly Per-Sub-Band Fringes

19

• ALMA observes with LINEAR polarizations

• PolConvert converts to CIRCULAR polarizations

• Band 6, April 8 2016 commissioning session VLBI calibrator (a.k.a. boring quasar) J1337-1257

• 1.3 Jy at 233 GHz• PolConvert “quick fringe”

analysis on single channels (no band-pass corrections) for “runtime” look at polarization processing

• Current work is to establish level of polarization purity in detailed analysis.

Original, Mixed Polarizationsas generated by DiFX

Converted, Circular Polarizations aswould have been generated by DiFX

|226.1 MHz FREQUENCY 228.1| |226.1 MHz FREQUENCY 228.1|

(From one of the commissioning data sets)✔ “Ok so far”

Corresponding Fourfit Fringes (parallel hands)

20

Fringes on first 60s (of 240s recording)

(From one of the commissioning data sets)

Corresponding Fourfit Fringes (cross hands)

21

Fringes on first 60s (of 240s recording)

(From one of the commissioning data sets)

Summary

• All components are currently in the DiFX trunk (=> 2.5.3)• EHTC hard at work imaging “Rev3” (which includes SPT

data).• The 2017 processing has taken much longer than any would

like due to the need to shake out all the details (not just with PolConvert or QA2 but also with the array calibration). Rev5 just delivered.

• “Publication early next year.”• 2018 data processing should begin soon, and• 2019 observation planning has started• Also starting to look at 345 GHz….

22

Some slides from last year on non-ALMA usage

23

On-the-fly Fringing (per sub-band)

24

XR ~ XL ~ YR ~ YL LL ~ RR >> LR ~ RL

(One of the commissioning data sets)

25

26

27

28

29

30