e-VLBI Development at Haystack Observatory

5th Annual e-VLBI WorkshopHaystack Observatory

20 September 2006

Alan R. WhitneyKevin Dudevoir

Chester RuszczykJason SooHoo

Current Projects at Haystack Observatory

• Network interfacing equipment for e-VLBI– Mark 5 VLBI data system

• Standardization (VSI-E)

• Intelligent Applications– Automation of e-VLBI transfers an ongoing process

– Development of optimization-based algorithms for intelligent applications ongoing (EGAE)

– Intelligent optically-switched networks (DRAGON)

• e-VLBI test experiments

• Production e-VLBI– Put e-VLBI into routine use – progressing well in limited venues

• Support e-VLBI development for VLBI2010 initiative

Scorecard of Antenna/Correlator Connectivity• JIVE Correlator (6 x 1 Gbps)• Haystack (2.5 Gbps; plans to expand to 10Gbps)• Westford, MA (10 Gbps to Haystack; 1 Gbps to outside world)• Kashima, Japan (2.5 Gbps)• Usuda, Japan (2.5 Gbps)• Nobeyama, Japan (2.5 Gbps)• Koganei, Japan (2.5 Gbps)• Metsahovi, Finland (1 Gbps)• MPI (1Gbps)• Tsukuba, Japan (2.5 Gbps)• GGAO, MD (1 Gbps)• Onsala, Sweden (1 Gbps)• Torun, Poland (1 Gbps)• Westerbork, The Netherlands (1 Gbps)• Medicina (1 Gbps)• NyAlesund – (~80 Mbps)• Jodrell Bank (1 Gbps)• Arecibo, PR (155 Mbps)• Wettzell, Germany (~30 Mbps)• Kokee Park, HA (nominally ~30 Mbps, but currently disconnected)• TIGO (~2 Mbps)• Svetloe (speed unknown)

In progress:• Hobart – expect fiber to station by Feb 2007 • Forteleza – funds secured for fiber connection at 2.5Gbps;

contract has been signed; expect completion Feb 2007• Zelenchukskaya – details unknown• Badary – details unknown

VSI-E

• Goals:– Efficient transport mechanism– Standard protocols– Internet-friendly transport– Scalable to 100Gbps– Suitable for real-time, quasi-real-time, and post-real-time data

transmission– Ability to make use of multicasting to transport data and/or

control information in an efficient manner• could be used in the future for support of distributed correlation

VSI-E Status

• Beta version of VSI-E is being tested in transfers from Kashima to Haystack and NyAlesund to Haystack (has been somewhat delayed by network availability at Haystack); expect result soon

• Awaiting successful demonstration before final ratification; plan submission to Internet Engineering Task Force (IETF) as international standard

• Working to make fast, efficient VSI-E implementation on Mark 5B

DRAGON Collaboration

• Collaborative development of API for control of dynamically optically switched networks

• e-VLBI experiments using DRAGON optically-switched networks over part of the path

• 512Mbps intercontinental real-time demonstration at SC05 in Seattle

Bossnet Route

Real-time e-VLBI SC05 DemoNov 2005

Real-time transmission and processing of data from antennas in Westford, MA, Greenbelt, MD, and Onsala, Sweden at 512 Mbps/antenna

All except Kashima equipped with Mark 5 data systems; Kashima uses Japanese K5, included via VSI-E

Correlation results displayed inreal-time at SC05 meeting

Progress towards routine e-VLBI

• April 2005– Start routine e-VLBI transfer from Kashima and Tsukuba (>200Mbps)

• Starting ~June 2005– Automated regular e-VLBI UT1 Intensive data transfers from Wettzell to

ISI-E (disks hand-carried to USNO for correlation)– A few start-up problems, but now operating fairly smoothly

• Spring 2005– Commitment to connect Hobart via optical fiber (schedule unknown)

• September 2005– All CONT05 data from Tsukuba transferred to Haystack via e-VLBI– Also – all Kashima and Syowa data now transferred via e-VLBI from Japan

to Haystack• November 2005

– Project initiated to connect NyAlesund to Haystack through NASA/GSFC at up to 100Mbps

– Data now transmitted routinely from NyAlesund, but slow data rate of ~80Mbps does not allow all data to be transferred; decision by NMA is pending whether to continue, upgrade, etc.

• December 2006– Funds secured to connect Fortaleza at 2.5 Gbps;

will in place by Feb 2007

e-VLBI Production Transfer

0

500

1000

1500

2000

2500

3000

3500

4000

450010/5

/04

12/5

/04

2/5

/05

4/5

/05

6/5

/05

8/5

/05

10/5

/05

12/5

/05

2/5

/06

4/5

/06

6/5

/06

8/5

/06

GB

Ny Alesund

Tsukuba

Kashima

Total data transferred: Tsukuba ~66TB Kashima/Syowa ~8TB NyAlesund ~6TB

EGAE Progress

• ‘Production’ e-VLBI facility has been established at Haystack to support routine e-VLBI transfers

• EGAE is now supporting routine non-real-time e-VLBI data transfers from Tsukuba

• EGAE will soon be used for routine e-VLBI transfers from Wettzell and NyAlesund

File-naming for Mark 5B/e-VLBI

For disk2file operations, Mark 5B has adopted the internationally agreed e-VLBI file-naming convention that uniquely identifies the data and carries all information necessary to satisfy VSI-E requirements, namely:

<exp name>_<stn code>_<scan name>_bm=<bit-stream mask>.m5bExample: ‘exp103_ef_scan001_bm=0x0000ffff.m5b’

A file2disk operation will parse the file name and preserve all relevant information.

Universal adoption of the internationally agreed conventions will result in easier e-VLBI data transfers with fewer errors.

Future directions

• Continued e-VLBI performance improvements, particularly with Mark 5B

• Continuing development and support of routine e-VLBI transfers; distribution of EGAE software package to others (MPI, etc)

• e-VLBI with 10GigE connections; motherboard and NIC performance evaluations

• Development with community of high-speed serial specification forVSI-E (10GigE, etc)

• Investigation of the use and value of Network Attached Storage as temporary buffer for e-VLBI data

• Collaboration with NROA/VLBA in e-VLBI development and (hopefully) deployment

• Continued close association with national and international community for global e-VLBI development and advocacy