1

Development of International e-VBLI Links:Past and Future

Alan R. WhitneyMIT Haystack Observatory

2

Traditional VLBIThe Very-Long Baseline Interferometry (VLBI) Technique(with traditional data recording)

The Global VLBI Array(up to ~20 stations can be used simultaneously)

3

Chautauqua 2001

Quasars, hotspots, polarization

VLBI astronomy example

ASTRONOMY• Highest resolution technique available to

astronomers – tens of microarcseconds• Allows detailed studies of the most distant

objects

GEODESY• Highest precision (few mm) technique

available for global tectonic measurements• Highest spatial and time resolution of

Earth’s motion in space for the study of Earth’s interior

•Earth-rotation measurements important for military/civilian navigation•Fundamental calibration for GPS constellation within Celestial Ref Frame

VLBI Science

Plate-tectonic motions from VLBI measurements

4

Scientific Advantages of e-VLBI

• Bandwidth growth potential for higher sensitivity– VLBI sensitivity (SNR) proportional to square root of Bandwidth

resulting in a large increase in number of observable objects(only alternative is bigger antennas – hugely expensive)

– e-VLBI bandwidth potential growth far exceeds recording capability(practical recording data rate limited to ~10 Gbps)

• Rapid processing turnaround– Astronomy

• Ability to study transient phenomena with feedback to steer observations– Geodesy

• Higher-precision measurements for geophysical investigations• Better Earth-orientation predictions, particularly UT1, important for

military and civilian navigation

5

Practical Advantages of ‘e-VLBI’

• Increased Reliability– remove recording equipment out of field– remote performance monitor & control capability in near real-time

• Lower Cost– Automated Operation Possible

• eliminates manual handling and shipping of storage media– Near Real-time Processing

• forestalls growth of storage-capacity requirements with bandwidth growth – Elimination of recording-media pool (millions of $’s!)

• Avoid unexpected media-shipping interruptions and losses

6

Elements of e-VLBI Development

• Develop eVLBI-compatible data system– Mark 5 system development at MIT Haystack Observatory being supported by BKG, EVN,

KVN, MPI, NASA, NRAO, NASA, – ~40 units now deployed worldwide

• e-VLBI Demonstrations– 1 Gbps e-VLBI using Bossnet (w/ DARPA and NASA support)

• ~700km link between Haystack Observatory and NASA/GSFC• First e-VLBI experiment achieved ~788Mbps transfer rate

• VSI-E: Standards for global e-VLBI– Standards for global interoperability are a must for wide deployment of e-VLBI

• Establish new protocols for e-VLBI– Adaptive network protocol

(newly awarded NSF grant to Haystack Observatory; collaboration with MIT Lab for Computer Science and MIT Lincoln Laboratory);

• New IP-based protocol tailored to operate in shared-network ‘background’ to efficiently using available bandwidth

• Connect with telescopes worldwide (U.S., Europe, Japan)

• Extend e-VLBI connections globally

7

Mark 5 VLBI Disk-Based Data System

• 1 Gbps continuous recording/playback to/from set of 8 inexpensive (ATA) disks•Mostly COTS components• Two removable ‘8-pack’ disk modules in single 5U chassis• With currently available 250GB disks – capacity of 2TB per ‘8-pack’ module; media cost now ~$1/GB and dropping• GigE connection for real-time and quasi-real-time e-VLBI operations• Inexpensive: <$20K

8

Haystack

Westford

NASA/GSFCUSNO

(correlator)

(correlator)

Bossnet 1 Gbps e-VLBI demonstration experiment

(Fall 2002)

•788 Mbps e-VLBI transfer achieved, but took much tuning•Full report at www.haystack.edu/e-vlbi

Initial experiment

Future

9

International e-VLBI experiments

• Westford, MA to Kashima, Japan - experiments in Oct 02, Mar 03, Jun 03– Files exchanged over Abilene/GEMnet/SuperSinet networks

• Best achieved so far ~107 Mbps– Correlation on Mark 4 correlator at Haystack and PC Software correlator at

Kashima; nominal fringes obtained– Further experiments are scheduled; active network testing is in progress

• Kauai, Hawaii to Wettzell, Germany (in progress)– Daily experiments of ~100GB are ideal candidate for early e-VLBI– Data will be transferred to Haystack Observatory for processing

(OC-3 speeds are possible)– Network links are now being developed

11

RTP Capabilities• RTP provides an Internet-standard format for:

– Transmission of sampled analog data– Dissemination of session information– Monitoring of network and end system performance (by participants and third parties)– Adaptation to varying network capability / performance– Message Sequencing / reordering– Multi-cast distribution of statistics, control and data

• RTP allows the reuse of many standard monitoring / analysis tools• RTP seen as internet-friendly by the network community:

– attention to efficiency • protocol designed to have minimum overhead for in-band data

– attention to resource constraints • won't use up all your bandwidth with control information

– attention to scaling issues

RTCP Capabilities• Monitors network’s real-time data delivery performance• Statistics collected from receivers• Information delivered to

–Senders (adapt to prevailing conditions)–Network management (identifies faults, provisioning problems)

• Adaptive, bandwidth-limited design

12

Possible VSI-E Topologies

13

New Application-Layer Protocols for e-VLBI

• Based on observed usage statistics of networks such as Abilene, it is clear there is much unused capacity

• New protocols are being developed which are tailored to e-VLBI characteristics; for example:– Can tolerate some loss of data (perhaps 1% or so) in many cases– Can tolerate delay in transmission of data in many cases

• ‘Experiment-Guided Adaptive Endpoint’ (EGAE) strategy being developed at Haystack Observatory under 3-year NSF grant:– Will ‘scavenge’ and use ‘secondary’ bandwidth– ‘Less than best effort’ service will not interfere with high-priority users– Dr. David Lapsley has joined Haystack staff to lead this effort

14

Typical bit-rate statistics on Abilene network

Conclusion: Average network usage is only a few % of capacity

Usage >20Mbps less than 1% of the time

100 500 Mbps

0.1

1.0

0.01

0.001

15

Typical distribution of heavy traffic on Abilene

Conclusion: Heavy usage of network tends to occur in bursts of <2 minutes

secs200 400 1000

1.0

0.9

0.8

0.7

<10% of ‘bulk’ transfers exceed ~100 secs

16

Possible International Connections

• Surfnet – U.S. to Europe at 10 Gbps• IEEAF – U.S. to Europe at 10 Gbps • SuperSinet – U.S. to Japan at 2.5 Gbps• TransPAC – U.S. to Japan at 655 Mbps (2 links)• GEMNET – U.S. to Japan at 150 Mbps (operated by NTT)• AMPATH – U.S. to S. America (mostly OC-3 – 155 Mbps)• Australia – Hawaii – U.S.: 10 Gbps link under consideration

17

18

19

622 Mbps +10 Gbps

Transoceanic donations to IEEAF (in red)

20

0

UKUKUK

FRFRFR

CHCHCH

SESESE

ITITIT RORORO

HRHRHR

EEEEEE LVLVLV LTLTLT

IEIEIENLNLNL

BEBEBEDEDEDE

PLPLPL

CZCZCZ

HUHUHUATATAT

SISISI

SKSKSK

LULULU

ESESESPTPTPT

GRGRGR

BGBGBG

CYCYCY ILI LI L622622 34343434

4545

622622

622622

155155155155

155155

155155

3434

3434

155155622622

155155 3434 4545

10 Gbps10 Gbps2.5 Gbps2.5 Gbps

SE - PoP for Nordunet

GÉANT:The connectivity at 10 Gbps

21

AMPATH: Research and Education Network and International Exchange Point for the Americas

Launched in March 2000 as a project led by Florida International University (FIU), with industry support from Global Crossing (GX), Cisco Systems, Lucent Technologies, Juniper Networks and Terremark Worldwide

Enables wide-bandwidth digital communications between the Abilene network and 10 National Research and Education Networks (NRNs) in South and Central America, the Caribbean and Mexico

Provides connectivity to US research programs in the region

AMPATH is a project of FIU and the National Science Foundation’s Advanced Networking Infrastructure & Research (ANIR) Division

Note: VLBI telescopes currently in Chile and Brazil

22

Impact of e-VLBI Program

• Opens new doors for astronomical and geophysical research.

• Represents an excellent match between modern Information Technology and a real science need.

• Motivates the development of a new shared-network protocol that will benefit other similar applications.

• Drives an innovative IT research application and fosters a strong international science collaboration.