high speed link design - cern

CERN 15 february 2011 High speed design seminar IN2P3/CPPM 1

High speed link design

J.-P. Cachemiche, F. Hachon, F. Rethore


Signal integrity (and designer's life) evolution

1990'

2010'

randomjitter

group speedpropagation mode

conversionvia stub

odd modedielectric

losses attenuationskin effect

even modenoise

returncurrent

path

reflectionscoper

roughness

sinusoidaljitter fiber weave

near endcrosstalk

far endcrosstalk

inter symbolinterferences

impedancematching

deterministicjitter

referenceplane

discontinuities

viainductance

via resonance

groundbounce

Crosstalk


Methodology questions

Can we still design at high speed with thumb rules ?

Or do we need to use simulators ?

Many tools and simulators available now

− 2D : Cadence Allegro SI, MentorGraphics Hyperlynx, ...− 3D : Ansoft HFSS, MentorGraphics Hyperlynx Full Wave solver, ...

Many modelisation techniques

− IBIS,− Spice,− IBIS-AMI,− S-Parameter models

All work fine (on paper !)

Do we need to build specific test mock-ups and make measurements ?

Is there a best compromise between speed, low cost and secure design ?


Outline

Quick tour of SI issues

Simulations

Measurements

Example design


Main signal integrity issuesin high speed designs


What degrades your signals

Eye diagramme = qualitative approach to the quality of your signals

Important to understand why an eye closes

Vertical closure : − Distorsions due to reflections, crosstalk, …− Attenuation due to losses, …

Horizontal closure:− Same + noise, thermal effects,


Distorsion by reflections

Main causes

Impedance mismatchNon ideal signal path:

− Stubs− Connectors− Non continuous return path− And at highest frequencies :

via resonance !

No via

Via back drilled

Thru via

Design File: Line_with_vias_near_end_back_drilledHyperLynx LineSim V8.0

U3

TX_diff2

1

TL1

56.7 ohms59.024 ps1.000 cmCoupled StackupTL2

56.7 ohms59.024 ps1.000 cmCoupled Stackup

TL5


TL6


R5

100.0 ohms

TL7

56.7 ohms59.024 ps1.000 cmCoupled StackupTL8


V2TOP

Inne...

V3TOP

Inne...

U4

RX_dif f2

1

Thru hole via

Back drilled via

Current

Current


Distorsions by mode conversion

Mode conversion for differential signaling

Asymmetries in differential lines are converted to common modeas a peak traveling with the signal.

In principle a differential receiver is not sensitive to common signals however ...− Loss of energy in differential mode,− Differential signal terminated by a resistor, but common signal sees an open

and reflects back signal,− EMI generation

V

t


Attenuation

Main causes− Skin effect losses− Dielectric losses

Given by:

dB=4,34RacZ c

Gleakage Z c

Varies with f Varies with f

Source GigaTest Labs


Consequence of attenuation

Inter Symbol Interference (ISI)

Higher frequencies more attenuated than low frequencies/

Rise time increases

When rise time is no more negligiblevs period, high or low level cannot bereached

Signal 480 Mbits/s same pattern 4800 Gbits/s

D = 0cm, Rt=40 ps D = 40cm, Rt=167 ps

Attenuation + jitter


Jitter

Random

jitterRandom

jitterDeterministic

jitterDeterministic

jitter

Sinusoidaljitter

Sinusoidaljitter

Data dependentjitter

Data dependentjitter

Duty cycledistorsion

Duty cycledistorsion

IntersymbolinterferenceIIntersymbolinterferenceI

Boundeduncorelated

jitter

Boundeduncorelated

jitter

Totaljitter

Totaljitter

Gaussian Bounded

Source: flickerThermal noise

Source: Rise andFalltime mismatch

Source: Losses

Source: Crosstalk

Source: Ground bouncePower supply variation

RX TXTXCH

Average value Average value

Crossing 0 Crossing 1

Bit interval

Histogram 0 Histogram 0

Samplingtime

Probability of error


Importance of decoupling

Careful decoupling needed at high speed

Spreading out a set of capacitorsall around your chip is not enough

Inductance of the path between a capacitorand the chip can dominate the capacitive effect


Mitigation of attenuation

Pre-emphasis or equalization compensate high frequencies attenuation

Modern devices embed complex filters

Ex: 3 taps FIR in Stratix IX GX

One tap pre-emphasis principle Equalization principle


Mitigation of attenuation requires tools

Optimization of emission and reception stages

Many parameters. Example: Stratix IV GX from Altera

− VOD :

8 values− Pre-emphasis parameters :

3 taps FIR filter 16 x 32 x 16 possible values− Equalization parameters: 16 values− Matching resistor : 4 values out, 4 values in

131072 combinations for pre-emphasis/equalization !

16 Millions for total

Simulation mandatory !Optimization tolls as well.

Emission side

Reception side


SimulationThe Power of SimulationPractical product design is about balancing three different requirements: 1) acceptable performance, 2) developed on a tight schedule, 3) at the lowest cost.It’s easy meeting any two of these requirements; it is a challenge meeting all three at the same time.

Eric Bogatin


Pros and cons of simulation

Pros

+ Allows understand problems and their relative importance

+ Simulation allows to explore rapidly several design hypothesis at no cost

+ Allows to explore the domain of correct functioning :Valuable help for defining constraints for automatic routing

+ Avoid design errors

+ Allows to design at the right cost

+ Sometimes the only solution to optimize efficiently

Cons

- Not everything can be simulated

- Tools far from perfect : many bugs

- Quite some time need to be invested to thrust simulation results


Comparison between simulation and measurements

Differences difficult to explain

Exemple : 1.6 Gbits/s design

Simulation Measurement

Unexplained distortion


New simulation

Increased length track to simulatepath in the case

Measurement

Hypothesis : path to the die notpresent in the encrypted model


Simulation trends (1)

Components models

IBIS models were widely used because they made impossible reverse-engineering.But not adapted to high speed links termination:

− no possibility to integrate equalization or pre-emphasis features

Encrypted Spice models have apeared but simulations takes days

IBIS-AMI (IBIS Algorithmic Modeling Interface) is a modeling standard for SerDes PHYs that enables fast, accurate, statistically significant simulation of multi-gigabit serial links

− Simulator agnostic− Modeling of Equalization− “Executable” blackbox (DLL)− 10 million bit simulations run in 10 minutes or less


Simulation trends (2)

Modeling paths

As frequency increases, problems get close to those of RF designers.RF designers tools appear more and more in simulators

In particular S-Parameters

Scatered-parameters are a frequencyrepresentation of time domain signals

As they have been progressively integratedin all simulation tools, they become a handyway to export or import designs betweenheterogeneous tools.

Their analysis gives precious information on:− which kind of perturbation you have, − the importance of it,− the frequency at which they occur


S-parameters in a nutshell

A single ended lossy line can be represented as a two ports network:

Output waves are a linear function of input waves at each port:

b1 = a

1 s

11 + a

2 s

12

b2 = a

1 s

21 + a

2 s

22

Where:

− s11

is the port-1 reflection coefficient: s11

= b1/a

1 when a

2 = 0

− s22

is the port-2 reflection coefficient: s22

= b2/a

2 when a

1 = 0

− s21

is the forward transmission coefficient: s21

= b2/a

1 when a

2 = 0

− s12

is the reverse transmission coefficient: s12

= b1/a

2 when a

1 = 0

wherea

1 is the wave into port 1

b1

is the wave out of port 1a

2 is the wave into port 2

b2

is the wave out of port 2

Port 1 Port 2


S-Parameters often used

Return loss (S11

)fraction of sine wave that reflects back from an interconnect when the sourcedriving it and the far end termination are each 50 ohms

Must be small ! (negative dB)

Insertion loss (S21

)fraction of sine wave that is transmittedthrough an interconnect, also when it isdriven by and terminated in 50 ohms

Must be close to 1 (0 dB)

S21

S11

S-parameters example


Differential S-parameters

Principe is the same for differential lines: 4 ports and 16 parameters

With the following numbering convention

Parameters are:

SingleEnded

SingleEnded

1

3

2

4

Differential

1

Differential

2

Source Agilent


Most used parameters

SDD21

and SDD11

: insertion and return losses for differential signals

SCD21

is a measure of the transmitted mode conversion. Indicates how much of a differential signal, launched into the channel, is converted into a common signal by the end of the channel.

Gives information on any asymmetries between the two lines.

SCD11

is a measure of the common signal that reflects back to the source when a differential signal is incident to the channel.

Converted back to time domain, gives information about the location of the asymmetry.


Datamining S-parameters

Analyzing parameters is a powerful means of understanding relative importance of perturbations at the transmission frequency

No via

Via back drilled

Thru via

Comparison of S21 parameter for via resonance

S11 parameter shows than more energy is reflectedby a via for high frequencies


Limits to simulation

Things are difficult to simulate accurately : Ex. Surface roughness

Irregularities of copper added to skin effect increase the actual pathAt 2.5 Gbits/s, skin depth below 2 µm !

T

Source Nanya CCL Ltd

0,00E+00 5,00E+02 1,00E+03 1,50E+03 2,00E+03 2,50E+03 3,00E+031,00

10,00

100,00

1000,00

10000,00

Skin effect

Epaisseur peau en µm

Frequence en MhzE

pais

seur

pea

u en

µm

18 µm

5 µm

Not easy to get data from the manufacturersSimulators use imperfect estimations

5 µmcurre

nt

copper


Measurements


Additional reasons for measuring

Models issues

Models are not always well adapted to your needs, in particular connector models:− Wrong size or model

Or they simply do not exist.

Real implementation of your board differs from expected

Track width (irregular etching).

Copper and dielectric thickness.

Copper roughness.

Malfunctioning

There is a problem, you don't know where it is.

It is essential to compare simulations with measurements

Trust the models or trust the measurements.


Modelling from measurement

S-parameters can be obtained from 2 types of devices

Time Domain Reflectometer (TDR)

Operate in time domain.Allow to see the line profile and have a raw idea of importance of defaults as well as their location.

Vector Network Analyzer (VNA)

Operate in frequency domainBetter dynamic range than TDR (~110 dB)Not significant in signal integrity issueswhere range is limited to 40-50 dB.40 dB ~ less than 1%

Both device give similar resultsfor S-parameters

Source Lecroy

Source Agilent


Method 1

Build a separate set-up

Implement different vias and track topologies,same connectors, same dielectric andstack-up as your future design

Needs SMA connectors for signal launchand on receiving end

Advantages

Can be done in parallel with actual design: allows to explore

SMAs allow easy connection with TDR or VNA

Drawbacks

Perturbation due to SMA connectors and tracks leading to the part to be measured.

Comparison with simulations on such setup requires a high speed pattern generator


Method 2

Build an additional PCB

Mount only connectors on a second PCBwith no components.

Advantages

Exact realistic paths.

All the paths are there: allows to investigate in case of problem.

Allows a fine tuned pre-emphasis and equalization parameters optimization.

Drawbacks

Available late.

Connection quite difficult.


De-embedding

S-parameters matrix can be converted into Transmission-parameters and vice versa

T-parameters are cascadable

If T1 is the transfer function for the launching probe, and T3 the transfer function of the receiving probe, they can be mathematically removed. Only measurement of the DUT remains. This is called de-embedding

b1

b2=S 11 S12

S 21 S 22∗a1

a2 a1

b1=T 11 T 12

T 21 T 22∗a2

b2S-parameters T-parameters

Fixture DUT Fixture

T1 T2 T3


De-embedding in practice

What you need to de-embed

− The cables− The probes− The onboard track connection to the portion

to be measured

Methods

Cables : calibration procedure (SOLT: Short Open Load Thru)

Probes if any : it is possible to calibrate or you can measure it !

Track connection:− Model the track, get the S-parameters then de-embed− Gating: you remove the signal up to the DUT part

All these methods are available with TDRs and VNAs

Ex. : Software PLTS from AgilentGating


Simulation with S-parameters

Few issues

Very often simulation does not converge because of insufficient quality of S-parameters.Most often due to bad elimination of test fixtures

Not easy to detect because models are quite different from physical device measured.2 main issues:

− Passivity : the system must not provide energy− Causality

Some tools propose to detect and correct the modelsNot recommended because means model is wrongBetter to find the cause


Example designGoing from 8 to 10 Gbits/s


System architecture

SNAP12

SNAP12

SFP+

Stratix IVFPGA

Stratix IVFPGA

Crossbar144 x 144

max6.5 Gbits/s Cyclone III

FPGA

1 Gigabit Ethernet transceiver

Acquisition board (AMC)

NAT MCH (Tongues 1 and 2)

Supervision

NIOS

Standard backplane(dual star layout)

Stratix IVFPGA

Stratix IVFPGA

Quad transceiver

Output board (AMC)

NIOS

10 boards

6

6

2 to MCH1

1mezzanine board

mezzanine board

Switch board (Tongues 3 and 4)

GigabitEthernetSwitch

CPU

Ethernet link

SMAClockdistribution

Tongue 1

Tongue 2

Tongues 3 & 4

External clock

2 to MCH2

2 to MCH12 to MCH2

2

2

2 from MCH12 from MCH2

2 from MCH12 from MCH2

PHY

6

6

Tracks to simulate


AMC board and mezzanine

AMC board

Mezzanine

Mezzanine mounted


Theorical approach

Mezzanine link

Checking the simulation− Eye diagram measurements at 8 Gbits/s to get a reference− Check of coherence between measures and simulations

Feasibility− Simulation at 10 Gbits/s− Optimization of pre-emphasis and equalization filters

Improvement of models− TDR measurements to obtain model of full data path− Extraction of S-parameters− Simulation at 10 Gbits/s− New optimization of filters

Backplane link

No model of backplane nor connectors !− TDR measurements to obtain model of full data path− Extraction of S-parameters− Simulation at 6.5 Gbits/s− Check of coherence between measures and simulations− Optimization of filters


Comparison between simulations and measurements at 8 Gbits/s

One way simulation between FPGA and optical device

Measured opening at 10-16 : 0.78 UISimulated opening at 10-16 : 0.67 UI

Simulation more pessimistic but approximate connector model

Design File: GT_TX_RXtstHyperLynx LineSim V8.1

U1

stratix4_gtstratix4_gt_tx_N

2

1

R2

100.0 ohms

J15

TX_SERIAL_TO_...

Port1 Port2

Port3 Port4

J10

sivgx_pkg_tx.s4p

Port1

Port2

Port3

Port4

J16

ELDO_seam035_s...SEAx10mm_conn

SEA... SEA...

SEA... SEA...

J17

TD6_c.s4p

Port1 Port2

Port3 Port4

Stratix IV GX track trackconnector

SimulationMeasurement


Simulation and preemphasisat 10 Gbits/s

Without preemplasisSimulated opening at 10-16 : 0.31 UI

With preemphasisSimulated opening at 10-16 : 0.63 UI(was 0.67 UI at 8 Gbits/s without error)

Preemphasis cancels the eye closure

No pre-emphasis

3 taps pre-emphasis


S-parameters extraction

Positioners


Connection issues

Card sawed to get accessto bottom side of mezzanine

Bad contactdifficult to see


Final setup


Issues met

Whole measurement chain has been tested

But quality of model insufficientSlight shift between measuring channels

Apparently a problem of calibration


Backplane Model

Nodel required to optimize pre-emphasisand equalization

Need another method to makemeasurements on the backplane

Build small size AMC cards with SMAConnectors

Backplane link at 4.8 Gbits/s


Conclusion

No universal recipe for high speed design, but rather a mix of several tools and methods

Thumb rules can be used … but not sufficient

Simulation allows to explore several hypothesis and specially to weight the relative importance of each source of perturbation but …

− You can simulate everything but not at a reasonable cost nor in a reasonable time− It is not accurate until you have crosschecked it with real measurements

Measurements can give powerful models you don't find from the manufacturer and understand the source of dysfunction.

Not an easy road

Simulations and measurements are difficult

All this has a high cost− Steep learning curve− Expensive measurement devices− Time

The main challenge is to find the best compromise between safe design, tight schedule and final cost: very difficult exercise !.

high speed link design - cern

Documents