frequency and time synchronization in packet based networks · precise time distribution...

1© 2010 Cisco and/or its affiliates. All rights reserved.

Frequency and Time Synchronization In Packet Based NetworksPeter Gaspar, Consulting System Engineer

© 2010 Cisco and/or its affiliates. All rights reserved. 2

• Synchronization Problem Statement

• Overview of the Standardization Works

• Frequency Transfer: techniques and deployment

Synchronous Ethernet

Adaptive Clock Recovery

• Time Synchronization

Two-Way Transfer Time Protocols

• Overview of IEEE Std 1588-2008 for Telecom

• Summary

© 2010 Cisco and/or its affiliates. All rights reserved.Presentation_ID 3

Problem StatementWhat and Why Do We Care About?


• Single domain vs. multiple domains

Internet is a multi-domain network.

Wholesale Ethernet virtual link

• Frequency and time could use different distribution methods.

• Operators may provide synchronization services to their customers.

Aggregation

Subscriber Access

MSE

TDM / ATM

Ethernet

WiMAX

OLT

xPON

xDSL

DSLAM

M-CMTS

DVB-T/H3GPP/2

DOCSIS

Backbone

Hub & Spoke or Ring

P

P

Internet

PEPE

MSA

PE

Peer ISP

MeshP

TDM / ATM

P P

VoD

Content Network

TV SIP

Mobile user

Femto-cell

Mobile TV

Enterprise

ResidentialSoHO


• Frequency

TDM interoperability and Co-existence: Circuit Emulation, TDM, MSAN (MGW)

Access: Wireless Base Stations, PON, DSL

• Time and Phase alignment

Wireless Base Stations

SLA and Performance Measurements

BS : Base Station

PON : Passive Optical Network

DSL : Digital Subscriber Line

SLA : Service Level Agreement


• Inter-CO/LAN (WAN)

• Intra-CO, LAN

• Intra-node, -platform

External Integrated Time and Frequency Server


The Leading Requirements

Application FrequencyPhase Alignment

Time Synchronization

TDM support (e.g. CES, SDH

transformation), Access

PRC-traceability, jitter & wander

limitations

ITU-T G.8261/G.823/G.824/G.825

Mobile

Base

Stations

GSM, WCDMA

and LTE FDD

Frequency assignment (fractional

frequency accuracy) shall be better than

• ± 50ppb (macrocells)

• ± 100ppb (micro- & pico-cells)

• ± 250ppb (femtocells)

N/A (except for MBMS and SFN)

UMTS TDDPhase alignment between base stations

must be < ±2.5µs

TD-SCDMAPhase alignment between base stations

must be < ±3µs

CDMA2KTime alignment error should be less than 3 μs

and shall be less than 10 μs

LTE TDDPhase alignment between base stations

from ±0.5µs to ±50µs (service degradation)

WiMAX Mobile Shall be better than ± 15 ppbPhase alignment between base stations

must be < ±1µs

DVB-S/H/T2 SFN TBDCell synchronization accuracy for SFN support

must be < ± 3µs

MB SFN ServicePhase/time alignment between base stations

requirement can vary but in order of µs

One-way delay and jitter

Performance Measurement

To improve precision << 1 ms

for 10 to 100µs measurement accuracy

need ± 1 µs to ± 10µs ToD accuracy


• Cost

• Limited utilization

Locations

Regulatory & Politics

• Reliability

Geography

Vulnerability

https://www.gsw2008.net/files/Civ%20Vulnerabilities_GSW2008.pdf

746th Test Squadron

Use of GPS (and GNSS alternatives) raises some concerns:

GPS : Global Positioning System

GNSS : Global Navigation Satellite System




• As Replacement or Backup

• Alternative Radio Navigation

LORAN-C ELORAN

• Atomic Clock

Cheap Scale Atomic Clock

Molecular Clock

• Network Clock

Main topic of this session!

LORAN : LOng Range Aid to Navigation


Standardization DevelopmentOrganizationsWho’s doing what?


• Frequency transfer

Parallel (overlay) SDH/SONET network

Radio Navigation (e.g., GPS, LORAN)

PHY-layer mechanisms

Packet-based solutions

• Time transfer (relative and absolute)

Radio Navigation (e.g., GPS, LORAN…)

Packet-based solutions


SDO Techno Status Scope Market

ITU-TSG15 Q13

Synchronous Ethernet

G.8261(2008)

G.8262(2007)+Amend.1

G.8264(2008)

G.781 (2008)

PHY-layer frequency transfer

Service Provider (SP) Metro & Core

Ethernet

Packet-based timing

G.8261 (2006)

Multiple working items: profile, metrics,

modeling…

CES performance

Packet-based frequency, phase and time transfer

Service Provider (SP)

IEEE

1588 PTP

IEEE1588-2002

IEEE1588-2008

No “Telecom” profile

Precise time distribution

Enterprise: Time

SP: Frequency, phase and time ITU-T & IETF

802.1ASBased on PTP

BallotPrecise time distribution

Residential

IETF

NTP NTPNTPv3 Standard

NTPv4 (CY09)Time distribution

Internet

SP domain

TICTOCNTPv5

PTP Profile(s)

New WG

(approved March 08)

Frequency and time transfer

Internet

Specific SP areas


Frequency Transfer

Distribution of Frequency Reference


• Physical layer options

Ex: SONET/SDH, SDSL, GPON, Synchronous Ethernet

Pros: “carrier-class”, well defined, guaranteed results

Cons: node by node link bit timing, requires HW changes

• Packet-based options

Ex: SAToP, CESoPSN, NTP, PTP (protocol of IEEE Std 1588)

Pros: flexible, looks simple, some can do time as well

Cons: the network and the network traffic, not so simple!


• The task of network synchronization is to distribute the reference signal from the PRC to all network elements requiring synchronization.

• The method used for propagating the reference signal in the network is the master-slave method.

• Slave clock must be slaved to clock of higher (or equal) stability. hierarchical model

PRC : Primary Reference Clock

Source: ETSI EG 201 793 “Synchronization network engineering”


• Synchronization equipments

PRC (PRS) and SSU (BITS) do not belong to the Transport network.

• SEC (SDH/SONET Equipment Clock) belong to Transport network.

They are embedded in Network Element : NE.


• Synchronization information is transmitted through the network via synchronization network connections.

• Synchronization network connections are unidirectional and generally point-to-multipoint.

Stratum 1 level

Stratum 2 level

NE(Stratum level 3)

CO


Core Network

Aggregation and

Access Networks

PRC : Primary Reference Clock (≈ PRS)

SSU : Synchronization Supply Unit (≈ BITS)

SEC : SDH Equipment Clock



Receiver for

synchronization

reference signal



Figure 4-2. Recommended BITS Implementation with SONET Timing Distribution

NE’s External

Timing Output

NE’s External

Timing Input

a.k.a.

BITS IN

Source: Telcordia GR-436-CORE . Digital Network Synchronization Plan


• Some of these synchronized trail contain a communication channel, the Synchronization Status Message (SSM) transporting a quality identifier, the QL (quality level) value.

This is a 4-bit field in SDH/SONET frame overhead.

• Purpose: Traceability (and help in prevention of timing loops)

Stratum 1 level

Stratum 2 level

NE level

What clock quality do I

get? Is that the best

source I can use?


SSM Allows Source Traceability

PRC synchronization network connection

SSU synchronization network connection

SEC synchronization network connection

Representation of the PRC network connection

X

Fault Representation of the synchronization network connection in case of failure

Example of restoration of the synchronization


• PHY-layer frequency transfer solution for IEEE802.3 links

• Well-known design rules and metrics

Best fit for operators running SONET/SDH

• Fully specified at ITU-T Working Group 15 Question 13

For both 2.048 and 1.544 kbps hierarchies

• Expected to be fundamental to high quality time transfer

• Drawback : hardware upgrades

All timing chain shall be SyncE capable.


PLL

Synchronous Ethernet capable

Line Card

IEEE802.3 ± 100ppm

ITU-T G.8261SyncE interface jitter & wander

ITU-T G.8262 (EEC):Synchronous Ethernet

Equipment Clock

ITU-T G.781: Clock Selection Process

Synchronous Ethernet capable

Line Card

Frequency distribution

traces

External timing interface inputs

External Equipment BITS/SSU)

External timing interface inputs

PRC-traceable signal from BITS/SSU

ITU-T G.8264ESMC and SSM-QL

External timing interface outputs

Synchronous Ethernet capable Equipment


• Ethernet Synchronization Messaging Channel

Use OSSP from IEEE802.3ay (a revision to IEEE Std 802.3-2005)

• Key purpose: transmit SSM (QL)

Outcome: Simple and efficient

But designed to support extensions

• Protocol model: Event-driven with TLVs

• Two message types

Event message sent when QL value change

Information message sent every second

• TLVs

QL-TLV is currently the unique defined TLV.

Other functions can be developed.

OSSP : Organization Specific Slow Protocol


0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1

|-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|

| Slow Protocols MAC Address |

|-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|

| Slow Protocol MAC Addr (cont) | Source MAC Addr |

|-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|

| Source MAC Address (continued) |

|-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|

|Slow Protocols Ethertype 0x8809| Subtype (10) | ITU-OUI Oct 1 |

|-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|

| ITU-OUI Octets 2/3 (0x0019A7) | ITU Subtype (0x0001)* |

|-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|

| Vers. |C| Reserved |

|-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|

| Type: 0x01 | Length | Resvd | QL |

|-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|

| Future TLV #n (extension TLV) |

|-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|

| |

| Padding or Reserved |

| |

|-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|

| FCS |

|-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|

* Allocated by TSB

IEEE 802.3

OSSP

ITU-T OUI

Header

ESMC Header

QL-TLV

Future TLV

Extension

Payload

OSSP


Assuring The Continuity at PHY Layer

• Extension or replacement of SDH/SONET synchronization chain

• Inherit from previous ITU-T (and Telcordia) recommendations

• Difference: frequency transfer path engineering will define the necessary upgrades.

Only the NE part of the engineered timing chain needs SyncE upgrades.

ITU-T G.8262

(EEC) Node

BITS/SSU

SONET/SDH PHY SyncE

BITS/SSU

PRC/PRS BITS/SSU

PHY SyncE

ITU-T G.8262

(EEC) Node

ITU-T G.8262

(EEC) Node

ITU-T G.8262

(EEC) Node


• Three key steps:

Generation: from signal to packet

Transfer: packet transmission over packet network(s)

Recovery: from packet to signal

Reference Clock

Recovered Clock

PSN


• ITU-T Recommendation G.8261 (2008) Adaptive Clock Recovery Definition

“In this case the timing recovery process is based on the (inter-) arrival time of the packets (e.g., timestamps or CES packets). The information carried by the packets could be used to support this operation. Two-way or one-wayprotocols can be used.”

ACR Protocol / Method One-Way Two-Way Timestamp

CES (SAToP, CESoPSN) X

IETF NTP (X) X X

IEEE Std 1588-2008 PTP X X X

IETF RTP X X X


Independent Timing Stream

TDM TDM

IWF IWF

Recovered TDM timing based on the adaptive clock recovery

ACR Packet StreamReference

Clock

TDM PW bit stream

Clocking method a.k.a. “out-of-band” (here, used for CES clocking)

TDM TDMIWF&

PEC

IWF&

PEC

ACR Packet Stream

TDM PW bit stream

PEC

Reference Clock


Source: Diagram from “Time Domain Representation of Oscillator Performance”,

Marc A. Weiss, Ph.D. NIST


• Frequency Accuracy

≤ ±50ppb at base station radio interface (specified)

Turns into ≤ ± 16ppb at base station traffic interface (not specified*)

• Frequency Stability

For T1, it shall comply to G.824 traffic mask (specification; 3GPP Rel8)

Sometimes* G.824 synchronization mask preferred

* Note: real requirements are variable as they are dependent on base station clock servo.


• Phase measurement

Measure signal under test against a reference signal

• Phase deviation plot

TIE : Time Interval Error

• Analysis

MTIE

TDEV


Step 1 : Phase Measurements

• At a certain signal threshold, time stamp the edges of timing signal.

• Signal edges are the significant instants.

• PHY-layer signals have high frequency (e.g., 1544 kHz)

-0.1 -0.2

+0.1

-0.2

+0.1

Signal

Ref.


Step 2 : Phase Deviation

• Phase deviation or TIE (Time Interval Error)


Step 3: Analysis

• Analysis cover different aspects of the

Clock (oscillator)

e.g. in free-running or holdover mode

Signal

• Primary used measurement analysis are:

Phase (TIE)

Frequency (fractional frequency offset)

Frequency accuracy

MTIE

TDEV


Signal with jitter and wander present


Jitter: Filter out low-frequency components with high-pass filter

FrequencyJitter range10 Hz


Wander: Filter out high-frequency components with low-pass filter

FrequencyWander range 10 Hz


• Both MTIE and TDEV are measures of wander over ranges of values.

From very short-term wander to long-term wander

• MTIE and TDEV analysis shows comparison to standard requirements.

Defined by ATIS/ANSI, Telcordia/Bellcore, ETSI & ITU-T

E.g., ITU-T G.824, ANSI T1.101 or Telcordia GR-253-CORE

• MTIE is a peak detector: simple peak-to-peak analysis.

• TDEV is a highly averaged “rms”-type of calculation.


Frequency Accuracy (Frequency Offset)

ITU-T G.823Traffic Interface (MRTIE mask)

ITU-T G.823Synchronization Interface (MTIE mask)


• Physical layer signals can be characterized.

• Recommendations exist for node clock and interface limits.

• Synchronous Ethernet Equipment Clock (EEC) inherits from SONET NE clock specifications.

• The performance of SyncE-capable NE and SyncE interface are fully specified and metrics exist.


• How to guarantee the packet-based recovered clock quality?

PSN

Reference Clock

Recovered Clock

Slave/ Client

Master/ Server

?

OK

Packet Delay Variation is key impairment factor for timing.

DS1 DS1


• TIE is still a valid measurement for characterizing the packet-based servo (slave).

Oscillators and timing interfaces

• How can the PSN behavior be characterized?

Algorithms use minTDEV value

Need sufficient numbers of minimal latency packets

Packet Delay Variation (PDV) as metric?

• First approach is to reuse known tools to PDV analysis/measurement.

Some can be applied to PDV as to TIE.


10 Switches, 40% Load

10 Switches, 80% Load

minTDEV


• Protocol parameters

• Influenced by : the PSN design, the HW & SW NE configuration, the traffic.

• Master implementation

PSN

Reference Clock

Recovered Clock

Slave/ Client

Master/ Server

PSN Metrics

? ?

Slave implementation

minTDEV used in algorithms, but still not adopted as metric

Even with (still to be agreed) metrics, other parameters will remain critical.

?


1. PHY-layer Synchronization Distribution guarantees the quality.

2. Packet-based Synchronization Distribution provides the flexibility.

3. Mixing the option for getting best of both solutions.

BITS/SSU

PRC/PRS

Thru BITS/SSU

EEC

EEC

EEC

EEC

Consumer

Non-capable PHY Layer Synchronization Network

SEC

PHY-layer methode.g., SDH/SONET, SyncE

Packet-based method (ACR)

PHY-layer Freq Transfer



e.g. SyncE


e.g. SyncE

SyncE consumer

Packet-based

consumer


Time Synchronization

What Specific Challenges Does the Time Distribution Introduce?


• Transmitting time reference can be absolute (from national standards) or relative (bounded timekeeping system).

• Time synchronization is one way achieving phase synchronization.

Phase alignment does not mandate giving a time value.


• This is not phase locking which is often a result of a PLL in a physical timing transfer.

Phase locking implies frequency synchronization and allows phase offset.

• The term phase synchronization (or phase alignment) implies that all associated nodes have access to a reference timing signal whose significant events occur at the same instant (within the relevant phase accuracy requirement).

t

t

timing signal recovered by system A

timing signal recovered by system B

System A

System B

B

Reference timing signal

to system A


to system B

t

t

timing signal recovered by system A

timing signal recovered by system B

System A

System B

B


to system A


to system B

Figure xxx/G.8266 – Phase Synchronization


Target from ±1µs to tens of µs (alignment between BS)

Target from ≤ ±0.5µs to tens of µs (from common reference)

Time Source


• Strictly speaking, the term synchronization applies to alignment of timeand the term syntonization applies to alignment of frequency.

• The master/server and slave/client clocks each have their own time-base and own wall-clock and the intent is to make the slave/client“equal” to the master/server.

• The notion of frequency synchronization (or syntonization) is making thetime-bases “equal”, allowing a fixed (probably unknown) offset in the wall-clocks. The notion of time synchronization is making the wall-clocks“equal”.


NTP vs. PTP Message Exchange

Master

time

Slave

time

t1

t4

t3

t2

Timestamps

known by slave

t2

t1, t

2

t1, t

2, t

3

t1, t

2, t

3, t

4

t-ms

t-sm

Sync

Follow_Up

Delay_Req

Delay_Resp

NTP

PTP

Usual unidirectional ACR protocol

As part of time recovery, there’s always a frequency recovery process.


• Forward and backward delays and delay variations are not identical.


• Each Node and Link can introduce asymmetry.

• There are various sources of asymmetry.


• Link

Link delays and asymmetry

Asymmetric (upstream/downstream) link techniques

Physical layer clock

• Node

Different link speed (forward / reverse)

Node design

LC design

Enabled features

• Network

Traffic path inconsistency

Interface speed change


Summary and Introduction to IEEE Std 1588

• Basis of all packet time transfer protocols (NTP, IEEE1588) is the two way time transfer mechanism.

• TWTT consists of a time transfer mechanism and a time delay “radar”.

• Assumes path symmetry and path consistency.

• IEEE1588 incorporates some in-network correction mechanisms to improve the quality of the transfer.

• IEEE1588 has the concept of asymmetry correction.

But the correction values are not dynamically measured - they need to be statically configured.


IEEE Std 1588-2008 for Telecom

Challenges of IEEE 1588-2008 applied in Service Provider networks


• A set of event messages consisting of:

- Sync

- Delay_Req

- Pdelay_Req

- Pdelay_Resp

• A set of general messages consisting of:

- Follow_Up

- Delay_Resp

- Pdelay_Resp_Follow_Up

- Announce

- Management

- Signaling

Transmission modes: either unicast or multicast (can be mixed)

Encapsulations: L2 Ethernet, IPv4, IPv6 (others possible)

Multiple possible values or range of values, TLVs (possible extensions), …


MASTER SLAVE

Delay_Resp

t1

t3

t4

Timestamps known by slave

t1, t2, t3, t4

SM_Delay

Master time = TM Slave time = TS

t2

t1, t2, t3

t1, t2

SYNC

Delay_Req

MS_Delay


SYNC

MASTER SLAVE

Delay_REQ

Delay_RESP

MAC/PHY MAC/PHYµP µP

Hardware assistance necessary to prevent insertion of errors or inaccuracies.

t1

t2

t3

t4

t4

t1

t2

t3

Need to inject the timestamp into the payload at the time the packet gets out.


t1, t2, t3, t4

t1, t2, t3

t1, t2


SYNC()

MASTER SLAVE

Delay_REQ()

Delay_RESP(t4)

MAC/PHY MAC/PHYµP µP


Follow_Up(t1)

t1

t2

t4

t3

Two-step clock modeVs.One-step (a.k.a. “on-the-fly”) clock mode

t1, t2, t3, t4

t1, t2, t3

t1, t2

t2


• Five basic types of PTP devices (“clocks”)

Ordinary clock (master or slave)

Boundary clock (“master and slave”)

End-to-end Transparent clock

Peer-to-peer Transparent clock

Management node

• All five types implement one or more aspects of the PTP protocol


• BC and TC aims correcting delay variation into intermediate nodes between OCs.

• Can correct link asymmetry if known.

Ref. Clock

Recovered Clock

Ordinary Slave

Ordinary Master

TC BC

Transparent Clock

Boundary Clock


• Can help on scalability when using unicast.

• Equivalent to NTP Stratum (>1) Server UTC

• Node by node: BC slave function is critical

Ref. Clock

Recovered Clock

Ordinary Slave

Ordinary Master

BC

Boundary Clock

BC

Boundary Clock


• TC calculates Residence Time (forward / reverse intra node delays).

• TC are supposed to be transparent but:

One-step clock issue

Ref. Clock

Recovered Clock

Ordinary Slave

Ordinary Master

Transparent Clock

Transparent Clock

TC TC


• If IEEE 1588-2008 is not planned node to node, with every node IEEE 1588 aware and in unique domain…

• Multiple interface types

IEEE 802.3, ITU-T G.709, …

• Multiple interface frequencies

10GE, 100GE, STM64, STM192…

• Multiple encapsulations

Ethernet, IP

MPLS, MPLS-TP, PBB-TE…


Ref.

ClockRecovered

Clock

Ordinary

SlaveOrdinary

Master

TC BC

WholesaleBoundary

Clock

TC BC

• Who owns the master?

• Who owns the slaves?

• Who owns the intermediate nodes?


• How to guarantee the recovered clock quality?

PSN

Ref. Clock

Recovered Clock

Slave/ Client

Master/ Server

?

?

?

TC

? ?

BC

Objective: accuracy and stability from reference


• IEEE Std 1588-2008 is actually a “toolbox” !

What does “support of IEEE 1588” really mean ?

• IEEE Std 1588 itself is not sufficient for telecom operator operations.

Node characterization, modeling, performance, metrics…

• For phase & time support, it is expected any telecom standardization would take time.


Summary


• Timing is a new service many networks shall have to support.

• Different solutions are necessary to cover disparate requirements, network designs and conditions.

Physical layer solutions required to upgrade routers and switches.

Packet-based solutions are more flexible but less deterministic.

• Whatever the timing protocol, it must deal with the same network constraints.

• Each network is different

• Synchronization Experts are welcome to enter the packet based networks and assist with the designs

Thank you.


Registrirajte se za Cisco Live Networkers u Londonu ili Bahreinu!

Više informacija na:

http://www.ciscolive.com/



frequency and time synchronization in packet based networks · precise time distribution...

Documents