e-VLBI progress in the South

Tasso Tzioumis

ATNF, CSIRO

September 2006

““Towards e-VLBI” Towards e-VLBI” “e-VLBI”“e-VLBI”

Outline

• Overview - details in other presentations/posters• Current LBA systems/developments

– Tapes - phasing out– DAS - digital filters– Disks - LBADR– Software Correlators

• Network developments• “Real” e-VLBI tests• Plans & the Future

LBA

x NTD

x New Norcia

Long baseline Array (LBA)• Telescopes in Australia

– ATNF, U.Tasmania, NASA - “core” Oz VLBI network

• + South Africa• + Telescopes in Asia Pacific region

– Japan, China, Hawaii, VLBA,… Asia Pacific Telescope (APT) (VSOP support)

• Open network (EVN model of operations)• http://www.atnf.csiro.au/vlbi

• (+ Tests with NZ - fringes ok - no large antenna)

LBA -S2 tape system

• Canadian S2 VCR-based recorder– 8 VCR recorders 128 Mbps max data rates– S2 maintenance issues (no external support)

• 7-station S2 LBA correlator at ATNF Sydney– XF architecture (ATCA correlator design)– Difficult to upgrade correlator

• The main production-VLBI system for > 12 years

• 2005 workshop: Phase-out S2 within ~3-4 years?• 2006 workshop: Decommission S2 in 2006-7!!

• Move to disk-systems + software correlation– Correlation contracted to Swinburne - 1 October 2006

LBA - DAS• DAS: Based on Digital Filters (> 10 years old!)

– 2 input IFs @ 64 MHz BW (2 pols or 2 frequencies)– Sampling (10 bits) + Digital filters (ideal passbands)– Bands: 64MHz 125 KHz– S2 mode: 2 x 16 MHz ( 128 Mbps to match S2)– Huygens mode: 4 x 16 MHz (256 Mbps)– Maximum: 2 x 64 MHz (512 Mbps) - tested in 2006– Multibit mode: 2 x 16 MHz x 8(10) bits 1 Gbps

• Interference excision?

• ATNF stations: 2 DAS units– 512 Mbps for Huygens (8 x 16 MHz)– 1 Gbps by 4 x 64 MHz channels - developed & tested in 2006

• Remains the main DAS for LBA system(s) – BUT no more units available!!

• Move to new VLBI dBBC units? (Haystack) - timescales?

LBADR - Disk recorders• (See C. Phillips poster for details)

• PCEVN-Metsahovi VSI interface system– VSIC (ECL-LVDS) & VSIB(computer)– Connects to LBA DAS– All LBA antennas equipped

• COTS computers – New Tyan “Thunder K8WE” motherboards (2005)

• Multiple PCI bus systems• Internal disks; IDE removable disks; X-RAID on fibre channel• (Replaced test systems on Dell 1600SC servers)

• Apple Xserve hardware RAID systems (7 TB)

Software correlators• See poster by S. Tingay & A. Deller

• FX software correlator(s)• Swinburne Supercomputer

– 300 Dell computers in cluster– Real time correlation (“normal” modes) - 30-50 nodes

• Smaller clusters at telescopes – (Parkes: 32 nodes; ATCA: 16 nodes)

• Cray XD-1 supercomputer in WA + FPGAs– Under development

• Very versatile & robust (can run on most systems)• Only correlators in Australia for disk systems!

“Real-time” fringe testing• Disk-systems

– Can run parallel to S2 VCRs (S2 C2 output port)– Short segment (~0.1s) data or use full recorded data– Transfer to Swinburne - (slow) ftp or fast links

• Correlate and post on web– Start correlating immediately (not all data needed)– Fringe in a few secs; web post within minutes

• Semi-automatic operation• Routine for all LBA observations

– Setup fringe tests for all observations– http://pentane.ssi.swin.edu.au/~cphillip/

Increased LBA reliability and robustness

Network connectivity -1• See talk by Shaun Amy

• AARNet3 Regional Network• Fibre tails to ATNF antennas in 2005-6

– $2M project, funded by CSIRO corporate– Fibre “tails” owned by CSIRO– “e-vlbi” tests in March 2006 - Official start June 2006!– Legal contract problems!!

• 2 x 1 Gbps connectivity – 1 operating now; 2nd circuit to be connected

• 10 Gbps (x multiple wavelengths) backbone• Good international connectivity

AARNet3 Regional Network

Network connectivity -2• Swinburne: 1 Gbps link(s) constructed

– Operation delayed by legal problems!! (again)– (10 Gbps link from Parkes?)– Enables software correlator connectivity

• Hobart: Fibre links funded and in construction (2006)– 1 Gbps - telescope ---> University– 10 Gbps to AARNet– (BUT Bass-link limited to ~0.5 Gbps?)

• WA Connectivity (Cray @ UWA)– CeNTIE: 3 x 1 Gbps BUT closes in 2006!– AARNet3 links planned– Impact on x-NTD?

• (Tidbinbilla - NASA: 2 Mbps commercial links– Future links? )

First Oz real “e-VLBI”• Date: 24 August 2006• Antenna: Parkes, Mopra, ATCA• Data rate: 128 Mbps (per station)• Bandwidth: 2x16 MHz (per station)• Processor: Dual AMD server (ie only 2 CPUs)• No disk buffering --> “e-VLBI”• Time: 10minutes+

• 1-bas test Parkes-ATCA at 256 Mbps/station

Plans - Near term

• Transition production VLBI to disk systems + software correlation (2006-7)– Investigate remote-disk recording– Acquire ~$0.5M of disks

• Extensive “real” e-VLBI tests– To Swinburne cluster correlator (in 2006)– To JIVE correlator under EXPReS (2006-7)

• Data disk formatting issues (LBA - Mk5a) - VSI-E??

– Tests to Haystack/Japan/Global - VSI-E??

• DAS replacement?

Special “e-VLBI” Instrument?

• Parkes-Mopra- Narrabri– 1 Gbps links installed– 1 Gbps data from 2 x DAS systems– Software correlation (Parkes or ATCA cluster)

Real-time VLBI instrument at 1 Gbps– Cf PTI instrument (Parkes -Tid @ 10 MHz BW)

• Very popular with non-VLBI specialists

– Fast detection observations/surveys– May operate in parallel with disk recording

• E.g. correlate 2nd 512 Mbps stream from ATNF antennas

Plans - Longer term• CABB - Broadband upgrade of ATCA

– 4 Gsamples x (8) bits 32 Gbps streams– 4 IFs per antenna; 8 antennas– Digital Polyphase filters -- > CABB Correlator

• ATCA operation in 2007-8– VLBI capability designed-in (2008?)

• Need for VLBI operation:– n x 10 Gbps connections to ATCA (Narrabri)– Broadbanding of Rx systems (Mopra, Parkes)

• Narrabri polyphase filters?

– Broadband and Fast links from non-ATNF antennas

Multiple 10 Gbps inputs & e-VLBI

2GHz

4k channels

4k channels

Multiple Zoom (>2 possible)Multiple Zoom (>2 possible)

2GHz

4k channels

4k channels

Com pound ZoomCompound Zoom

2GHz

4k channels

Simple ZoomSimple Zoom

2GHz

4k channels

Basic ConfigurationBasic Configuration

FFT

FFT

FIR

FIR

Filterbank

Fringe

Rotators

Correlators

DMUX

A 2GHz bandwidth polyphase digital filterbank with 4096 channels ... ... will fit into four XC2V6000 FPGAs

FPGA hardware may be

completely reprogrammed to produce many different filterbank

configurations, as different

observations may require.

RF, IF and Baseband Signal Path

Filterbank Architecture

A Wideband Upgrade for the Australia Telescope Compact Array

Luneburg Lens Phased Array

Luneburg Lens Phased Array

6 * 22m antenna AT Compact Array

Summary• S2 tape LBA system to be decommissioned 2006-7• LBADR - disk recorder developed• Software correlation - “real time” speeds

– Very versatile, robust and portable

• LBA transition: disk recording + software correlation• Digital DAS - need to augment/replace• 1 Gbps links to ATNF antennas operating

– Links to correlator and other telescopes in construction

• First “real” e-VLBI tests successful • Plans for eVLBI instrument (Pks-Mop-Nar)• Future: 10 Gbps links + CABB correlator