impact of timing on synchronization in data centers and 5g … · 2020. 10. 14. · webinar...
TRANSCRIPT
-
Webinar – October 14, 2020
Impact of Timing on Synchronization in Data Centers and 5G Networks
Gary Giust, PhD
Sr. Mgr, Technical Mktg
Nazariy Tshchynskyy
Sr. Mgr, Customer Engineering
Jeff Gao
Sr. Dir, Marketing
-
2
Agenda – Impact of Timing on Synchronization in Data Centers and 5G
1. How to Model Oscillator Performance
2. Impact of Temperature Sensitivity
3. Impact of Aging and Wander on Holdover
4. Case Studies
5. Conclusions
-
Modeling Oscillator Performance in 5G Networks and Data Centers
-
4
Oscillators in IEEE 1588PTP with Physical Layer Frequency Support
5G Networks
PTP without Physical Layer Frequency Support
Data Centers
-
5
Modeling Servo Impact to PTP Time Error
-
Loop Filter
Local Time
Local Time
Local Clock
Frequency correctionTimestamp
processing
Network time stamps
Networktime
Local Oscillator
(LO)
Network noise Servo Loop LPF
+Servo Loop HPF for LO
Time Error – combination of Network Performance and
Oscillator Noise
Time_error (s)
time (s)
Filtered network time
LO noiseFiltered LO noise
-
6
Modeling Oscillator-noise Impact on Time Holdover
Time_error (s)
time (s)
Time error (t)
1.5us
-1.5us
f (ppb)
time (s)
OSC frequency in time
∫0
tΔf(t) Δφ(t) /(2πfc)
f (ppb)
T (°C)
f (ppb)
time (s)
f (ppb)
time (s)
Frequency vs Temperature 1 day aging
Temperature ProfileT (°C)
time (s)
T1 T2
T1
T2
Frequency change in time (due to temperature)
Dwell time = 15 minRamp rate = 0.5 °C/min
+
f (ppb)
time (s)
Frequency wander
Time_error (s)
time (s)
Time error (t)
1.5us
-1.5us
f (ppb)
time (s)
TCXO frequency in time
∫0
tΔf(t) Δφ(t) /(2πfc)
PLL responsemodel
f
H(f)
f (ppb)
T (°C)
f (ppb)
time (s)
f (ppb)
time (s)
Frequency vs Temperature 1 day aging
Temperature ProfileT (°C)
time (s)
T1 T2
T1
T2
Frequency change in time (due to temperature)
Dwell time = 15 minRamp rate = 0.5 °C/min
+
f (ppb)
time (s)
Frequency fluctuations at constant temperature
-
Impact of Oscillator Temperature Sensitivity on PTP Time Error
5G Networks and Data Centers
-
8
PTP Time Error for 1 ppb/°C and 10 ppb/°C TCXO (time constant = 1 min)
-
9
PTP Time Error for 1 ppb/°C and 10 ppb/°C TCXO (time constant = 10 min)
-
10
Frequency-over-Temperature Slope, dF/dT
• Frequency-over-temperature slope quantifies frequency change per change in temperature, in units of ppb/°C
Measured MEMS TCXO Frequency Stability
-100
-50
0
50
100
-50 -10 30 70 110
Fre
qu
en
cy S
tab
ility
(p
pb
)
Temperature (C)
ΔT
ΔF
Slope = ΔF/ΔT [ppb/°C]
-
11
Which Part is Better for PTP Applications: 50 or 100 ppb?
-100
-50
0
50
100
-50 -10 30 70 110
Fre
qu
en
cy S
tab
ility
(p
pb
)
Temperature (C)
± 50 ppb
± 100 ppb
What matters is low sensitivity to temperature changes (dF/dT), not lifetime peak-peak stability
-
12
Frequency Stability Video – Impact of dF/dT on Performance
https://www.youtube.com/watch?v=Bttd1f1wo6g&feature=youtu.be
>200 ppb jump
Elite is stable>20 ppb jump with fan on
50C
±50 ppb Quartz TCXO ±100 ppb MEMS TCXO
https://www.youtube.com/watch?v=Bttd1f1wo6g&feature=youtu.be
-
13
MEMS Architecture – Optimized for Low PTP Time Error
• Excellent thermal coupling between 2 MEMS
resonators in same die
• Digital low-noise, high-bandwidth TDC
• Enables compensation of fast temperature changes
• Higher-order compensation enables smallest dF/dT
residue
• Limited thermal coupling between
Quartz and ASIC
• Analog noisy temp sensor
• Limited compensation of slow temperature
changes only
• Higher-order large dF/dT residues
SiTime DualMEMS™ Temperature SensorQuartz TCXO
Quartz Crystal
But temp
sensor is here!
ASIC
Need to measure
temperature here
TempFlat™ Resonator
Temp Sensing Resonator
DualMEMS™
Resonator Die
SiTime ASIC
MEMS
TempFlatTM
ResonatorTemp Sensing
Resonator
-
14
Elite-platform TCXO Architecture – Optimized for PTP Applications
TempFlatResonator
Temp Sensor
DualMEMSThermally Coupled
Mixed-Signal CMOS IC
CLK
I2C
OSC-Temp
OSC-Timing
TempSense(TDC)
Frac-N PLLFrequency Synthesis
TempComp
VCXO ADC I/O
Dividers& Driver
On-ChipRegulators
OEVctrl
TempFlat™ MEMS Resonator
• No aging
• No activity dips
• 30x better vibration immunity
Low Noise CMOS Enabling Frequency Agility
• 1 to 220 MHz, steps in mHz
• 0.2 ps/mv PSNR, 5x better
• In-system programmability
DualMEMS™ Temp Sensing 100% Thermal Coupling
• 30 µK, 10x more accurate
• 350 Hz tracking, 40x faster
• Airflow, temp ramp resistant
-
15
MEMS vs Quartz TCXO dF/dT Performance
1 ppb
- 1 ppb
Better than 1 ppb/°C
-
Impact of Oscillator Aging and Wander on Holdover
5G Networks
-
17
Model of Time Holdover
Time_error (s)
time (s)
Time error (t)
1.5us
-1.5us
f (ppb)
time (s)
OSC frequency in time
∫0
tΔf(t) Δφ(t) /(2πfc)
f (ppb)
T (°C)
f (ppb)
time (s)
f (ppb)
time (s)
Frequency vs Temperature 1 day aging
Temperature ProfileT (°C)
time (s)
T1 T2
T1
T2
Frequency change in time (due to temperature)
Dwell time = 15 minRamp rate = 0.5 °C/min
+
f (ppb)
time (s)
Frequency wander
-
18
Holdover Definition
• “Holdover begins when a clock output no longer reflects the influence of a connected external reference” [1]
• Example: GPS is used to synchronize timing across base stations in telecom networks. Holdover is used to maintain connections when user moves from one cell site to another and GPS is unavailable.
• Two common usages of “holdover”
1. Frequency holdover is the maximum frequency deviation from the average frequency (in ppm) over some time interval (tau), after the system loses lock to its reference.
2. Phase (or time) holdover is the accumulated time error with respect to a reference clock after the system loses lock to its reference.
[1] https://en.wikipedia.org/wiki/Holdover_in_synchronization_applications
https://en.wikipedia.org/wiki/Holdover_in_synchronization_applications
-
19
Phase, Frequency, and Time Error
• Relationship between phase and frequency
𝜙 𝑡 = 0𝑡𝜔 𝑡 𝑑𝑡, where
𝜙 𝑡 is the phase of a signal in radians
𝜔 𝑡 is the frequency of the signal in radians/sec
• Hence time error of a clock signal is the integration of its frequency error
𝑇𝑒𝑟𝑟 𝑡 = 0𝑡∆𝑦 𝑡 𝑑𝑡, where
𝑇𝑒𝑟𝑟 𝑡 is the time error
∆y 𝑡 =∆𝑓 𝑡
𝑓is a fractional frequency error
-
20
Components of Time Holdover
• Useful to separate components impacting oscillator holdover
- For example, to analyze impact of temperature profiles on holdover
• Main oscillator holdover contributors
1. Temperature Variation. Oscillators have frequency sensitivity to temperature changes that can be expressed as ppb/°C (frequency change per 1 °C of temperature change). Therefore frequency of the oscillator is modulated by environment temperature changes.
2. Short term aging quantifies how much frequency of the oscillator changes per day due to aging, and can usually be approximated with a straight line.
3. Time Deviation (TDEV) quantifies wander (random phase fluctuations) in the oscillator.
• If daily aging is a straight line, Time Error (∆𝑇) may be predicted as (per NIST publication 1065):
∆𝑇 = 𝑇0 +∆𝑓
𝑓∙ 𝑡 +
1
2𝐷 ∙ 𝑡2 + 𝜎𝑥 𝑡 , where
• 𝑇0 is the initial synchronization error (seconds)
•∆𝑓
𝑓is the fractional frequency that is the sum of average environment induced frequency error
• 𝐷 is daily aging rate (fractional frequency per second)
• 𝜎𝑥 is the rms noise induced time deviation, or wander (seconds)
-
21
dF/dT = 5 ppt/°C
aging = 0.2 ppb/day
T variation = 25°C to 35°C
Combined Holdover
• Add all 3 components to compute system holdover
• The following plot shows an example time error computation
-
22
Example OCXO Holdover at 45±15 C, 0.5 C/min Temperature Ramp
6.58 Hours @ 1.6 µs
-
23
Aging Compensation Improves Stability
• Daily aging can be predicted with some accuracy
• Aging observed over time can be used to predict/remove frequency drift
1. Fit a line to first 24h of data (solid black line)
2. Extrapolate this line to the next
12h of data (dotted black line)3. Subtract extrapolated
line from the dataset
-
24
Aging Compensation Improves Holdover
Without aging compensation With aging compensation
-
25
Hadamard Deviation (HDEV)
• A measure of clock-frequency stability, similar to Allan deviation (ADEV)
• Hadamard variance removes linear frequency drift from ADEV
𝐻𝜎𝑦2 𝜏 =
1
6(𝑀−2)σ𝑖=1𝑀−2[𝑦𝑖+2 − 2𝑦𝑖+1 − 𝑦𝑖]
2,
where 𝑦𝑖 , 𝑖 = 1. .𝑀 is a set of fractional frequency values measured back to back and each averaged over
the time interval 𝜏
• Why remove linear drift from frequency stability?- Better models networks whose time sources are compensated for daily aging
- Common in time-synchronized networks requiring holdover (5G, telecom, etc.)
-
26
HDEV is Similar to ADEV with Linear Drift Removed
After removing linear drift, ADEV looks like HDEV
Original ADEV
ADEV with linear drift removed
HDEV
Linear drift (aging)
-
27
Holdover Measurement
• Holdover can be measured at component or system level
• Component level measurement requires an instrument that can…
- measure frequency of the clock continuously with high resolution and without gaps for extended periods of time (multiple days), for example Keysight 53230a frequency counter, or
- measure time of edges (timestamp edges) continuously every 1s or more often with high resolution for extended periods of time (multiple days), for example GuideTechGT667 CTIA
• System level measurement…
- monitors the time difference between 1 PPS signal from a time master and 1 PPS signal from a slave under test (e.g. master to slave time error). It can be measured with a frequency counter, like Keysight 53230a, digital oscilloscope, or any other instrument that can accurately measure time between signal edges on 2 independent inputs.
-
28
Component-level Holdover Setup
Rubidium
Reference
Time Interval Analyzer
Guide Tech GT667
or
Frequency Counter
Keysight 53230a
Temperature Chamber
TestEquity 115
Test Board
50 Ohm
coax cable
OCXOCLK
-
29
System-level Holdover Setup
Rubidium
Reference
Frequency Counter
Keysight 53230a
Temperature Chamber
TestEquity 115
Slave
1 pps OCXO
Master (Time Server)
1 pps or
10 MHz 1 pps
1588 link
-
Case Studies
-
31
eCPRI
FronthaulSwitch
RRU
Elite TCXO
IEEE 1588Processing
MEMS Clocks for IEEE 1588/eCPRI
RF
BBU Server
CascadeNetwork
SynchronizereCPRI/SyncE
MEMS Timing Engineered for Robust 5G RRU Outdoor Deployments
Problems that MEMS Timing Solves
• Ensure
-
32
FPGAAccelerator
CascadeNetwork
SynchronizereCPRI/SyncE
Elite TCXO
MEMS Clocks for Accelerator Card
EthernetPCIe
IEEE 1588
IEEE 1588Fronthaul
Switch
BBU Server
RRU
eCPRI
MEMS Timing Enables Zero Downtime in Fronthaul Switches, ORAN HUB
Problems that MEMS Timing Solves
• Enhance robustness: flexible hitless switching and input monitoring for fail-safe
• Ensure
-
33
MEMS Clocks
MEMS Timing in Smart NIC for IEEE 1588 and BBU/DU
NICSOC
Elite TCXO
MEMS Clock for NIC
eCPRI
FronthaulSwitch
vBBUServer
RRU
Problems that MEMS Timing Solves
• Enabling IEEE 1588 for PCIe NIC: 1 mm think profile
• Ensure
-
34
Conclusions
• Time error attributed to 3 noise sources in an oscillator:
1. Temperature sensitivity
2. Aging
3. Wander
• Can’t judge PTP performance by frequency-over-temperature number alone
- Frequency-slope over temperature (dF/dT, ppb/°C) and temperature profile influence max time error
• TCXOs with same frequency-over-temperature stability can have significantly different dF/dT performance, and thus PTP time error
- DualMEMS TCXO architecture is optimized for synchronization applications
- DC-TCXOs enable tuning servo loops with ±5 ppt resolution
• 5G networks additionally require holdover, impacted by aging and wander
• Extend holdover by correcting daily aging, but ultimately limited by wander