march 2013 - nvidia€¦ · precision in microseconds/ntp medium quality link microseconds samples...
TRANSCRIPT
WHITE PAPER
©2013 Mellanox Technologies. All rights reserved.
Highly Accurate Time Synchronization with ConnectX®-3 and TimeKeeper®
Highlights and Benefits:
• Sub-microsecond time synchronization, using off-the-shelf hardware
• Support for both PTP and NTP protocols and master clocks
• TimeKeeper® is easily deployable on any existing ConnectX®-3 network adapter card
Upgrading your trading platforms to reliable and precise time is achievable at a low cost and a rapid deployment model via the combination of Mellanox’s ConnectX-3 network adapter cards and TimeKeeper Client software. TimeKeeper can assure sub-microsecond time precision from both the newer IEEE 1588 Precision Time Protocol (PTP) or the standard Network Time Protocol (NTP) over shared (not dedicated) network links. Flexibility in time sources and automatic adapability to network quality allows for incremental changes to enterprise systems, and immediate high precision timing in critical components while less critical components see incremental performance improvement. For high quality links and time feeds, applications can see time locked to reference well within 500 nanoseconds of variation.
-1
-0.5
0
0.5
1
Precision in microseconds/NTPmedium quality link
Mic
roS
econ
ds
Samples
Figure 1. Precision in Microseconds/NTP Medium Quality Link
High performance comes from the complementary action of two technologies: TimeKeeper’s state-of-the-art time synchronization and management and the Mellanox ConnectX-3 hardware based packet time- stamping. TimeKeeper uses information that the ConnectX-3 provides to factor out packet delay variation (PDV) caused by buffering, network stacks, and OS scheduling and automatically improves precision of locking the local timer to the reference time. Because the ConnectX-3 can time-stamp both PTP and NTP packets, TimeKeeper can greatly improve performance of both protocols.
Figure 2. Precision in Microseconds/PTP Medium Quality Link
March 2013
In collaboration with
FSMLabs
3
-1-0.5
00.5
1
Precision in microseconds/PTP medium quality link
Samples
Mic
roS
econ
ds
page 2WHITE PAPER: Highly Accurate Time Synchronization with ConnectX®-3 and TimeKeeper®
©2013 Mellanox Technologies. All rights reserved.
TimeKeeper’s Client software will automatically detect and take advantage of the time stamp capabilities of the ConnectX-3 – the only configuration needed, is to point the software at the right network interface, IEEE 1588 and with NTP. After which, TimeKeeper can also be used to setup failover, auditable trace logs, automatic spoof and equipment failure detection, and automatic notification of error conditions to management consoles (via SMTP) and IT staff (via email). Customers can also use the web interface to validate network setup and to optimize their time distribution networks.
The graph below, Figure 3, is from the TimeKeeper web management console and shows two time sources being tracked and validated against the other. Discrepancies can trigger a switch of time sources or an alert or both- depending on configuration. TimeKeeper can track any number of independent time sources. This increases system resilience and also produces a trace of system behavior to pinpoint causes of errors. The free software for PTP and NTP can fail silently – allowing application code to keep trading on the basis of incorrect time. The “time map”, Figure 4, capability introduced in TimeKeeper 6 discovers the surrounding time distribution network and helps IT staff find configuration problems and identify sources of network time fragility. This feature is in use in some of the world’s leading financial trading organizations and has uncovered numerous previously undetected system vulnerabilities
Figure 3. Cross Check Second
Figure 4. Time Map
Because TimeKeeper and the ConnectX-3 are protocol agnostic, it is not necessary to make huge, disruptive changes in existing time distribution networks. Various parts of the network can use PTP or NTP as needed, leveraging the strengths of each and the fault tolerance derived from using sources carried over both protocols.
2266689500
Sour
ce ti
ck ra
te (p
er s
econ
d) 2266689000
2266688500
2266688000
2266687500
2266687000
2266686500
2266686000
2266685500
Samples
Source 0 ideal tickrate
Source 1 ideal tickrate
132.163.4.103
ACTS
this_system
64.113.32.5128.252.19.1:GPS
128.252.19.1
128.7.1.66129.250.35.250
204.42.158.152 129.7.1.66:GPS
129.250.35.251
143.166.82.251
166.255.32199.4.29.166
192.5.41.40
128.4.1.1
128.4.1.1. PPS
18.26.4.105.PPS
209.81.9.7:GPS
209.81.9.7
128.9.176.30:GPS
128.9.176.30
128.67.113.92
108.71.253.20:GPS
216.218.254.202:CDMA
216.218.254.202
216.45.57.38
216.171.124.36
43.77.130.254
108.71.253.20
204.123.2.5:GPS204.123.2.5
69.36.224.15
132.163.4.101
172.16.1.36
69.36.224.15:GPS
69.25.96.13
129.6.15.28
128.138.140.44
128.9.57.133
18.26.4.105
192.5.41.40:IRIG
209.51.161.238:CDMA
209.51.161.238
132.163.4.103
ACTS
this_system
64.113.32.5128.252.19.1:GPS
128.252.19.1
128.7.1.66129.250.35.250
204.42.158.152 129.7.1.66:GPS
129.250.35.251
143.166.82.251
166.255.32199.4.29.166
192.5.41.40
128.4.1.1
128.4.1.1. PPS
18.26.4.105.PPS
209.81.9.7:GPS
209.81.9.7
128.9.176.30:GPS
128.9.176.30
128.67.113.92
108.71.253.20:GPS
216.218.254.202:CDMA
216.218.254.202
216.45.57.38
216.171.124.36
43.77.130.254
108.71.253.20
204.123.2.5:GPS204.123.2.5
69.36.224.15
132.163.4.101
172.16.1.36
69.36.224.15:GPS
69.25.96.13
129.6.15.28
128.138.140.44
128.9.57.133
18.26.4.105
192.5.41.40:IRIG
209.51.161.238:CDMA
209.51.161.238
page 3WHITE PAPER: Highly Accurate Time Synchronization with ConnectX®-3 and TimeKeeper®
350 Oakmead Parkway, Suite 100, Sunnyvale, CA 94085Tel: 408-970-3400 • Fax: 408-970-3403www.mellanox.com
© Copyright 2013. Mellanox Technologies. All rights reserved.Mellanox, BridgeX, ConnectX, CORE-Direct, InfiniBridge, InfiniHost, InfiniScale, MLNX-OS, PhyX, SwitchX, UFM, Virtual Protocol Interconnect and Voltaire are registered trademarks of Mellanox Technologies, Ltd. Connect-IB, CoolBox, FabricIT, Mellanox Federal Systems, Mellanox Software Defined Storage, MetroX, Open Ethernet, ScalableHPC and Unbreakable-Link are trademarks of Mellanox Technologies, Ltd. All other trademarks are property of their respective owners. 15-513WP Rev 1.0
Figure 5. Distribution of NTP/PTP in Upgraded TimeKeeper Network
About Mellanox ConnectX-3 Network Adapters:
Mellanox’s cost-effective, high-performance, ConnectX-3 network adapters support a variety of network protocols from 1/10/40GbE and FDR 56Gb/s InfiniBand. ConnectX-3 network adapters are available through OEM server vendors in a variety of form factors, including: LOM, add-on cards, blade mezzanine cards and PCI expansion cards. With ConnectX-3 network adapters, enterprise server engineers can utilize a single, cost-effective adapter for all of their data center needs – trading, time synchronization, database, and virtualization.
For more information on Mellanox ConnectX-3 network adapter card, visit: http://www.mellanox.com/page/products_overview
For more information on TimeKeeper, contact: [email protected]
TimeKeeper and FSMLabs are registered trademarks of Finite State Machine Labs Inc.
TK enabledApplication Server
Application Server
Distribution of NTP/PTP in Upgraded TK Network 2
Switch
TK GrandMasterAppliance
TK enabledApplication Server
TK enabledApplication Server
Application Server
Switch
Application StratumServer
NTP PTP
NTPPTP
Application Server