4th e-vlbi workshop, sydney, july 2005, a. szomoru, jive e-vlbi and expres arpad szomoru

4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE

e-VLBI and EXPReS

Arpad Szomoru


Outline

• Current status, recent results

• Network test

• e-VLBI and the EXPReS proposal

• Conclusions, future


Why e-VLBI ?

• Reliability – real-time feedback to the telescopes

• Sensitivity – sustained high data rates possible

• Logistics – No media management

• Rapid science results:• Geodesy

• Precision spacecraft navigation


e-VLBI Proof-of-Concept Project

• DANTE/GÉANT Pan-European Network• SURFnet Dutch NREN• GARR Italian NREN • UKERNA UK NREN• PSNC Polish NREN • DFN German NREN• KTHNOC/NORDUnet Nordic NREN • Manchester University Network application software • JIVE EVN Correlator• Westerbork telescope Netherlands• Onsala Space Observatory Sweden• MRO Finland• MPIfR Germany• Jodrell Bank UK• TCfA Poland • CNR IRA Italy


0SISISISI

GRGRGRGR

CZCZCZCZ

IEIEIEIE

LULULULU

EEEEEEEESKSKSKSK

ILILILILLTLTLTLTCYCYCYCY

ESESESES

HUHUHUHUITITITIT

FRFRFRFR

UKUKUKUK

BEBEBEBE

NLNLNLNLDEDEDEDE SESESESE CHCHCHCH

PTPTPTPT

ATATATAT

GEANTGEANTGEANTGEANT

2.5 G 1.2 G 622M

155 M

BGBGBGBGLVLVLVLVRORORORO

HRHRHRHR

PLPLPLPL

45 M34 M

310 M

GÉANT:Access of NRENs to GÉANT

EVN telescope


• for the EVN and JIVE

• Standards:- Protocols, parameter tuning, procedures at telescope and correlator.

• New Capabilities:- Higher data rates, improved reliability, quicker response.

• for GÉANT and the NRENs• To see significant network usage with multiple Gbit streams

converging on JIVE.• Minimum is three telescopes (not including Westerbork)• Must be seen to enable new science and not just solve an existing

data-transport problem• Real-time operation is seen as the ultimate aim, buffered operation

accepted as a development stage.

POC targets:


Fibre pair from Amsterdam

Optical Splitters

GE lines to correlator

LX Optics converters

Cisco ONS 15252

Correlator Interface

Current status:

•JIVE: 6 lambdas via Netherlight, each capable of 1 Gbps

•1 Gbps connections to Westerbork, Torun, Onsala, Jodrell

•155 Mbps to Arecibo, 128 to Cambridge

•Medicina being connected at 1 Gbps now, Metsahovi 2006

Current status:

•JIVE: 6 lambdas via Netherlight, each capable of 1 Gbps

•1 Gbps connections to Westerbork, Torun, Onsala, Jodrell

•155 Mbps to Arecibo, 128 to Cambridge

•Medicina being connected at 1 Gbps now, Metsahovi 2006


Network testing

• Use existing protocols on currently available hardware

• TCP maximal reliability• Not really required

• Sensitive to congestion• Lot of fine-tuning necessary

• And possible

• UDP connectionless• Unaccountable

• Tailor made protocols?

• Lambda switching

• Internet weather• Hard to quantify

• Hard to pinpoint bottlenecks

0

100

200

300

400

Dw- BoBo- DwDw- Bo*Bo- Dw*

0

200

400

600

800

Dw- Bo

Dw- Tr

Dw- OnM

bps


• February 2005: network transfer test (BWCTL) employing various network monitoring tools involving Jb, Cm, On, Tr, Bologna and JIVE

Onsala-Jive

0

200000000

400000000

600000000

800000000

1000000000

1200000000

1 3 5 7 9 11

13

15

17

19

21

23

25

TCP

UDP

e-VLBI transfer tests


e-VLBI transfer tests


Memory-memory Disk2net-net2disk In2net-net2disk

In2net-net2out

UDP TCP UDP TCP TCP TCP

Bench via patch 930 250 256

Idem, jumbo frames 960 544 512

Bench via Amsterdam 500 360 256

Idem, jumbo frames 341 456

Westerbork-JIVE 867 680 256 256

Idem, jumbo frames 249 378

Bologna-JIVE 670 128 307

Jodrell-JIVE 50 70 64 256

Arecibo-JIVE 88 64

Torun-JIVE 800 260 128

Onsala-JIVE 177 256 256

JIVE-Haystack 612 71

Test results


POC results

• Demonstration of feasibility

• Identification of problems

• Has led to closer ties with networking community and generated political interest

• Has laid the foundation for the next step forward (EXPReS)


The Future… e-VLBI & EXPReS!

• I3 proposal to the EC (Communication & Network Development Call)

• Ranked first out of 43 proposals; nearly fully funded to an amount of 3.9 MEuro.

EXPReS = EXpress

Production

Real-time

e-VLBI

Service


EXPReS aims

• Upgrade EVN to e-EVN• Last mile problem at many telescopes

• Including connections on a global scale

• 16 Gb/s lines into Dwingeloo to JIVE correlator

• Software in field and correlator to become real real-time

• And look beyond 1 Gb/s• More capacity on digital sampling, more bandwidth

• As being implemented for e-MERLIN in UK

• Hardware (PC-based) and protocols for transport

• Correlator with more capacity: distributed correlation


Expanding the e-VLBI Network


New Antennas

40m antenna at Yebes, SpainRadio and mm frequencies

Sardinia Radio Telescope64m; radio to millimeter


Current “e-VLBI” Telescope Status Telescope Status Comments Cost (Meuro)

JIVE 6 x 1 Gbps 16 x 1 Gbps upgrade ……

WSRT (NL) 1 Gbps - -

Onsala (SE) 1 Gbps 10 Gbps upgrade 0.15

Torun (PL) 1 Gbps -

Jodrell Bank &

Cambridge (UK)

2.5 Gbps 10 Gbps upgrade

(7 x 30 Gbps e-MERLIN**)

0.12

Arecibo (USA) 155 Mbps 1 Gbps upgrade 0.4

Medicina (IT) 1 Gbps Summer 2005 -

Metsahovi (FI) 1 Gbps 2006 0.33

** Crucial part of the plan is to combine e-MERLIN and e-VLBI Telescopes transparently.


Telescope Status Comments Cost (Meuro)

Yebes (ES) NYO 1 Gbps planned 2006/7 0.53

VIRAC (LV) NYO 1 Gbps planned 2006/7

VLBI Capable 2006 ?

0.53

Shanghai (CN) NYO 0.1- 1 Gbps planned

Last mile: 25 km

Poor connectivity between Europe & China (ORIENT)

0.35

Miyun (CN) & Kunming (CN)

NYO 0.1-1 Gbps planned

Last mile: few km

0.3

Urumqi (CN) NYO Fibre installed – 2 Mbps

0.1-1 Gbps planned

0.1

Hartebeesthoek (SA) NYO 1 Gbps planned as part of SKA ambitions

Poor connectivity between South Africa & Europe

-

TIGO (Chile) NYO Fibre installed -

Noto (IT) NYO Last mile problem 3.1

Sardinia (IT) NYO Last mile 1.0

Effelsberg (DE) NYO Last mile 5 km but… 2.0


Towards “real”real-time

• Reliability of operation

• Robustness

• Feedback to stations

• Status display at stations and Jive

• Ease of operation

• Continuous monitoring

• Better protocols?

• Software consistency


Protocol, buffer selection


FABRICFuture Array of Broadband Radio-telescopes on Internet Computing • Will need a new correlator for 4 Gb/s

• Current EVN Mk4 based on 16x16x1Gb/s

• Implemented on 1024 special chips

• Next generation will most likely use standard CPUs• Current EVN Mk4 processor equivalent 40 T-ops (2bit)

• LOFAR BlueGene ≈ 27 Tflops

• 32 station x 4 Gb/s 640 T-ops⇒• And requirement to route 32x32 4 Gb/s input stream

• Possible solution: distribute the computing • Use the Internet as the cross-switch

• SETI@home gets 59 Tflops

• Proposed to do pilot on the Grid


Telescopes will do multicast transmission

Compute nodes will be cluster size

Transported data may be time/frequency slices

Resource allocation from central control

Output data stored in central archive, proprietary rights for users

Central astronomical scheduling


Conclusions, future• POC has been a success, but, progress has been both fast

and slow• Bottlenecks hard to identify, packetloss does not equate

congestion, fairness is a problem, relation packetloss—correlation poorly understood

• Geant2, Surfnet6 will be hybrid networks • lambda switching• Bandwidth on demand

• New protocols, software/hardware interfaces• 4—10Gbps transfer—new hardware?• e-VLBI is moving ahead; e-EVN will soon be a reality

4th e-vlbi workshop, sydney, july 2005, a. szomoru, jive e-vlbi and expres arpad szomoru

Documents