vlbi_udp an application for transferring vlbi data via udp protocol simon casey e-vlbi meeting,...
TRANSCRIPT
VLBI_UDPVLBI_UDPAn application for An application for
transferring VLBI data via transferring VLBI data via UDP protocolUDP protocol
Simon Casey Simon Casey
e-VLBI meeting, Haystacke-VLBI meeting, Haystack20 Sep 200620 Sep 2006
OverviewOverview
Originally developed by Richard Hughes-JonesOriginally developed by Richard Hughes-Jones Sent multiple, continuous memory-memory UDP Sent multiple, continuous memory-memory UDP
streamsstreams Multi-threaded, contained ring bufferMulti-threaded, contained ring buffer Web interface to manage streams & graph Web interface to manage streams & graph
throughputthroughput
Modified to allow disk-disk transfersModified to allow disk-disk transfers Currently implementing selective packet Currently implementing selective packet
droppingdropping
Ring buffer
Input thread
Ring buffer
Send thread
Disk Memory
Receive thread
Output thread
Disk Memory
UDP Data
TCP Control
Control thread Control thread
Vlbi_control
vlbi_send vlbi_recv
vlbi_udp Flow Diagramvlbi_udp Flow Diagram
Geant2 Launch event, Geant2 Launch event, June 2005June 2005
StabilityStability
Instability in sending Instability in sending rate when not rate when not saturating linksaturating link
~20Mbit/s variation~20Mbit/s variation
Change thread Change thread implementation to use implementation to use sched_yield()sched_yield()
Stable to within 1Mbit/sStable to within 1Mbit/s
vlbi_recv_k269_110d_w20_29Aug06
600
602
604
606
608
610
612
614
616
618
620
0 1000 2000 3000 4000 5000
Time between frames us
Wir
e R
ate
Mb
it/s
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Packet
loss
Packet loss & correlationPacket loss & correlation
Packets are lost - networks Packets are lost - networks (congestion) and/or end-hosts (lack (congestion) and/or end-hosts (lack of processing power)of processing power)
Lost packets will have some effect Lost packets will have some effect on final correlationon final correlation
Want to measure this effectWant to measure this effect
GoalsGoals
Determine optimal way to lose Determine optimal way to lose packets whilst having least effect on packets whilst having least effect on correlationcorrelation
Find reasonable loss rate which can Find reasonable loss rate which can be sustained by station unitsbe sustained by station units
Implement findings into VLBI_UDPImplement findings into VLBI_UDP Can be used to selectively drop packets Can be used to selectively drop packets
if congestion present yet keep if congestion present yet keep correlation runningcorrelation running
Correlation tests at JIVECorrelation tests at JIVE
Run disk2file to create standard linux fileRun disk2file to create standard linux file Use VLBI_UDP in disk-disk modeUse VLBI_UDP in disk-disk mode
Add extra mode to write to local file instead of Add extra mode to write to local file instead of sending to networksending to network
Filter each packet through a dropping Filter each packet through a dropping functionfunction Algorithm can be selected by user and Algorithm can be selected by user and
parameters variedparameters varied Place modified data file(s) back on to disk Place modified data file(s) back on to disk
packpack Play back into correlatorPlay back into correlator
Packet dropping functionPacket dropping function
Pass pointer to valid packet as Pass pointer to valid packet as parameterparameter
Function decides if packet will be Function decides if packet will be droppeddropped Returns either original pointer or Returns either original pointer or
pointer to some dummy datapointer to some dummy data What dropping algorithms are used?What dropping algorithms are used?
Keeping in syncKeeping in sync
Station units look for valid headersStation units look for valid headers No header? Work out where next No header? Work out where next
header should be and keep lookingheader should be and keep looking How many missing headers before sync How many missing headers before sync
is lost?is lost? Lose only VLBI data, keep headersLose only VLBI data, keep headers
SUs will be happy – only care about SUs will be happy – only care about headersheaders
Actual correlation will sufferActual correlation will suffer
Keeping in syncKeeping in sync Assume losing every other header Assume losing every other header
acceptableacceptable At 256Mbit/s, headers occur every 80 000 At 256Mbit/s, headers occur every 80 000
bytesbytes 32 * 2500 byte tracks = 1 'frame'32 * 2500 byte tracks = 1 'frame' 79360 bytes data sandwiched between 384 bytes 79360 bytes data sandwiched between 384 bytes
header and 256 byte footerheader and 256 byte footer ~54 UDP packets/'frame'~54 UDP packets/'frame' Lose Lose ≤≤ 1 packet in 56 (1.79%) – will never lose 2 1 packet in 56 (1.79%) – will never lose 2
consecutive headersconsecutive headers N.B. assumes losses don't occur randomly!N.B. assumes losses don't occur randomly!
Dropping strategiesDropping strategies
Leave first 2 mins of each run intactLeave first 2 mins of each run intact Use loss rates:Use loss rates:
0.5%; 1%; 1.5%; 1.75%; 2%; 2.25%0.5%; 1%; 1.5%; 1.75%; 2%; 2.25% Run each rate twice, dropping in series Run each rate twice, dropping in series
of long bursts or evenly spaced of long bursts or evenly spaced individual packetsindividual packets
Expect rates 2%, 2.25% sync may be Expect rates 2%, 2.25% sync may be lost, below 2% should stay in synclost, below 2% should stay in sync
Total 12 runsTotal 12 runs
Dropping strategiesDropping strategies
Prepare 13Prepare 13thth file to test SUs... file to test SUs... Remove only headers, different loss Remove only headers, different loss
rate every 2 mins (file 14 mins long)rate every 2 mins (file 14 mins long)
Preliminary ResultsPreliminary Results Packet losses produced Packet losses produced
clear effectclear effect Average amplitude Average amplitude
differences for each differences for each loss rateloss rate
Linear relationshipLinear relationship % Amp decrease = 1.8x % Amp decrease = 1.8x
% loss rate% loss rate Header dropping Header dropping
partially successfulpartially successful Dropping every other Dropping every other
header, SU LEDs orange header, SU LEDs orange but no difference in but no difference in correlationcorrelation
Run 3 - 0.005% bunch Diffs
0.000E+00
5.000E-05
1.000E-04
1.500E-04
2.000E-04
2.500E-04
3.000E-04
3.500E-04
4.000E-04
4.500E-04
5.000E-04
30:30
31:00
31:30
32:00
32:30
33:00
33:30
34:00
34:30
35:00
35:30
36:00
36:30
37:00
37:30
38:00
38:30
39:00
39:30
40:00
40:30
41:00
41:30
42:00
42:30
43:00
43:30
44:00
44:30
45:00
Time
Am
plit
ude
Bsln= Ef-Wb SB= sub0 Pol= LL
Bsln= Ef-Jb SB= sub0 Pol=
Bsln= Wb-Jb SB= sub0 Pol= LL
y = 1.82x + 0.0917
R2 = 0.9995
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
0 0.5 1 1.5 2 2.5
Packet loss rate (%)
% d
ecre
ase
in c
orr
elat
ed a
mplitu
de
Future workFuture work
Enable VLBI_UDP to communicate Enable VLBI_UDP to communicate with Mk5A Streamstor cardwith Mk5A Streamstor card
Deploy and test within e-VLBI test Deploy and test within e-VLBI test runrun