DesignCon 2004

Design of 3.125 Gb/s Interconnect for High-bandwidth FPGAs Sherri Azgomi, Altera Corporation sazgomi@altera.com Lawrence Williams, Ph.D., Ansoft Corporation williams@ansoft.com CF-031505-1.0

Abstract The push toward FPGA platform solutions with high bandwidth DSP and Gigahertz-speed I/O functionality has led to devices that place greater demands on printed circuit board (PCB) design. This paper details how to apply commercial electromagnetic (EM) finite element software combined with transient circuit simulation to characterize transmission lines, vias and connectors for systems that incorporate high-bandwidth FPGAs. The test structure includes an FPGA development board, a backplane, daughter cards, and associated connectors and coax cables. Two-dimensional EM simulation was used to extract quasistatic circuit models for the PCB transmission lines and coax cables. Three-dimensional EM simulation was used to extract models for vias. SPICE was used for end-to-end simulations. Measured TDR and eye diagrams show that simulations can be used to accurately predict interconnect performance at 3.125 Gb/s. Author(s) Biography Sherri Azgomi is a BSEE graduate from San Jose State University with emphasis on Digital Design. For the last seven years she was a Senior Applications Engineer at Altera Corporation. She has specialized on signal integrity and has written numerous articles, application notes, and other literature on design features of FPGAs. Lawrence Williams is Director of Business Development at Ansoft Corporation. He has over 17 years experience in the fields of electromagnetics and communications engineering and has published numerous technical papers on the subject. He received his Masters, Engineers, and Ph.D. degrees from UCLA in 1989, 1993 and 1995, respectively.

Introduction FPGA platform solutions with high bandwidth DSP and I/O functionality have led to devices that place greater demands on printed circuit board (PCB) design. Today's FPGA's provide serial I/O speeds up to 3.125 Gb/s with near-future devices providing 10 Gb/s. Careful signal integrity design is required for proper operation of these systems. This paper details how to apply commercial electromagnetic (EM) finite element software and laboratory measurements combined with transient circuit simulation to characterize transmission lines, vias and connectors for systems that incorporate high-bandwidth FPGAs. The test structure includes an FPGA development board, a backplane, daughter cards, and associated connectors and coax cables. Two-dimensional EM simulation was used to extract quasistatic circuit models for the PCB transmission lines and coax cables. Three-dimensional EM simulation was used to extract models for vias; time-domain reflectometry (TDR) measurements were used to extract SMA connectors; and SPICE was used for end-to-end simulations. The authors have endeavored to cover enough fundamentals so that a novice modeling engineer can get started modeling their system using the various models provided. It is assumed that the user has familiarity with HSPICE—this document only covers modeling system parasitics. Transmission line and via models have been validated using laboratory measurements and are considered reliable for interconnects up to 12 inches in length. The rest of the paper is organized as follows. The second section provides a system overview of the communications channel and the various elements in the transmission path. Transmission line modeling follows, with extraction of models for coupled stripline and coaxial cables using two-dimensional EM simulations. The fourth section covers circuit via modeling using three-dimensional EM simulation to extract HSPICE netlists. SMA connectors are modeled using TDR measurements and associated simple equivalent circuits are developed in the fifth section. Section six briefly illustrates how to apply the models in an HSPICE netlist. Transient simulation results are compared with lab measurements for varying trace lengths in section seven. Conclusions are provided at the end. System Overview It is critical to model the transmission path when designing a high-performance, high-speed serial interconnect system. The transmission path may include long transmission lines, connectors, vias and crosstalk from adjacent interconnect. Figure 1 depicts such a system with a driver, transmission path, and receiver. At the driver the data output as shown in the eye diagram has very clearly-defined amplitude levels with minimal jitter around zero crossings. There can be significant degradation to the data after it has passed through the transmission path. This degradation includes loss of signal amplitude, reduction of signal rise time, and a spreading at the zero crossings.

Figure 1. System model.

Altera and other FPGA suppliers provide characterization reports on I/O drivers and receivers including complete data on the transmitter characteristics such as rise time and eye opening. Also included are data on the receiver sensitivity and tolerance. Designers should consult the characterization reports for required device specifications. Figure 2 shows the set up that was used for simulation and lab measurement. An Altera Stratix GX driver and transmitter package model were connected to the board transmission line, via and the board SMA. It was then connected to a Tyco backplane, consisting of a line card, HMZD connector, a 16 or 32 inch backplane interconnect, another HMZD connector, another line card and finally the coax cable to the oscilloscope. The complexity of this transmission medium containing numerous discontinuities is typical of modern high-speed backplane designs. Extraction of models for the transmission lines, vias, and connectors are discussed in the next sections.

PkgModel T-line Linecard Con. Bkpln LinecardCon. cable

outaSMAVia

outb

Figure 2. Test and simulation setup for Stratix GX driver to oscilloscope.

Transmission Line Models An electromagnetic field solver was used to extract the transmission line parameters. Field solvers use Maxwell’s equations to determine the actual field behavior present in a device. There are two types of field solvers: quasi-static and full wave. This paper covers applications using quasi-static field solvers.

A transmission line can be described at the circuit level using series inductance and resistance combined with shunt capacitance and conductance. An infinitesimal unit length of the transmission line looks like the circuit in Figure 3.

Figure 3. RLCG parameters for a segment of a transmission line.

In Figure 3,

R = Series Resistance per unit length L = Series Inductance per unit length G = Shunt Conductance per unit length C = Shunt Capacitance per unit length.

Ansoft’s SI2D two-dimensional field solver was used to extract the RLGC parameters for transmission lines. These parameters were used to create an HSPICE-compatible W-element for all transmission structures. After drawing transmission line geometries and entering material parameters, the Ansoft field solver automatically generates a mesh and calculates the electric and magnetic fields using a quasistatic approach. Values for R, L, C, and G are extracted automatically. It is a good idea to perform self-comparison tests to substantiate the results. This is done by observing values produced by the field solver and also by observing some of the plots that can be generated. For a differential pair, a good thing to observe is the skin effect resistance plot for each of the two circuit traces. Figure 4 shows the skin effect resistance plot for a differential pair. This extraction looks satisfactory because the curves for the two traces overlap each other showing the expected symmetry.

Figure 4. Skin resistance plot for two traces of a differential pair.

Four transmission lines structures were modeled in the SI2D field solver to extract RLCG parameters. The models are for three differential stripline traces as found on the Stratix GX board, the Tyco backplane and the Tyco linecard. Additionally, simulations for a coaxial cable were computed. The coax cable is the standard M17/128-RG400 MIL-c-17 27478 from Harbour Industries. Figure 5 shows the trace geometry for the Stratix GX development board; Figure 6 shows the trace geometry for the Tyco backplane and linecard boards. The skin effect resistance plot for the transmission lines were verified for accuracy of the transmission line models. Figure 7, shows the skin effect resistance plot for the coax cable. Table 1 shows the characteristic impedance for GX transmission line, coax cable, Tyco backplane and linecard.

10.2 mils

4.5 mils

1.4 mils

1.4 mils

0.7 mils15 mils5 mils

Dielectric FR4Er=4.25

Loss Tangent =0.015

RefPlane

RefPlane

Figure 5. Stratix GX development board trace geometry.

Figure 6. HMZD XAUI test backplane and linecard board trace geometries.

Figure 7. Skin effect resistance of coaxial cable.

Table 1. Characteristic impedance for transmission lines and the coaxial cable.

Model Characteristic Impedance (Ω)

Stratix GX T-line Single ended Differential

49 96

Tyco Linecard Single ended Differential

49 95

Tyco Backplane Single ended Differential

52 94

Coax Cable 49

The extracted W-element RLGC parameters for the transmission lines were as follows. RLGC parameters for Stratix GX board: .model gx_tline W modeltype=RLGC N=2 + Lo= 3.372422173812121e-007 5.464530764862712e-009 3.37217715985489e-007 + Co= 1.40255495312275e-010 -2.272806348031037e-012 1.402656859222826e-010 + Ro= 91.16920995613322 2.630659436008461 128.1523605301604 + Go= 0.06640885710842964 -0.001087493365952726 0.06646140849911818

RLGC parameters for backplane board: .model tyco_backplane W modeltype=RLGC N=2 + Lo= 3.628107780173747e-007 3.623569811203107e-008 3.632197082161472e-007 + Co= 1.339425591351141e-010 -1.344737615837475e-011 1.339533359535629e-010 + Ro= 1.98840709666684 0.07953628386667692 1.988407096666994 + Rs= 0.000343881684307448 1.471739178249039e-005 0.0002855352823410895 + Gd= 1.686031415906164e-011 -1.691861448335916e-012 1.684572301092438e-011

RLGC parameters for line card board: .model tyco_linecard W modeltype=RLGC N=2 + Lo= 3.362729949759179e-007 1.067979775181489e-008 3.366673414376468e-007 + Co= 1.403120902626182e-010 -4.482967123423322e-012 1.403008513153242e-010 + Ro= 6.521975277068002 0.1590725677333606 6.521975277067504 + Rs= 0.001204901372207414 2.715935906588883e-005 0.0008959393454384264 + Gd= 1.807490866627052e-011 -6.030522233159035e-013 1.816428860057189e-011 RLGC parameters for coaxial cable: .model coax_cable W modeltype=RLGC N=1 + Lo= 2.210676739058297e-007 + Co= 9.075249807462414e-011 + Ro= 5.47407179762798 + Go= 0.003400579647438758

Via Modeling A three-dimensional field solver was used to extract the parasitics associated with via structures. Modeling a transmission medium component such as a via in the time-domain is often performed using a quasistatic assumption. Quasistatic analysis is valid whenever the physical discontinuity is smaller than λ/10, where λ is the wavelength of the highest frequency component present in the signal. Consider a signal that has a rise time of Tr. It is often accepted that the highest frequency component present in the signal can be determined using the rule-of-thumb:

BW = .5 / Tr where BW is the bandwidth or highest frequency component. We know that,

Velocity = frequency * wavelength. Velocity in the dielectric is the inverse of delay. In an FR4 medium, the velocity is 5 G in/s, therefore the wavelength in FR4 is 5G /BW in. A differential via on one of the high speed I/Os on the Stratix GX board was modeled. For this task a quasistatic 3D field solver Ansoft Q3D was used. The following calculations were performed to determine if the via could be modeled in the time domain. The rise time of the driver is 100 ps. The highest frequency component in the signal was calculated: BW = .5/100ps = 5 GHz. The wavelength was calculated,

λ = (5 G in/s) / 5 GHz = 1 in.

therefore,

λ/10 = 0.1 in.

From this we conclude that if the length of the component is less than 100 mils, then the component can be modeled as a lumped element. The via in this case could be modeled as lumped element using a quasistatic field solver since it is only 93 mils long (the board is 93 mils thick).

Figure 8. Differential via structure as modeled in Ansoft Q3D.

Figure 8 shows a pictorial representation of the through-hole differential vias surrounded by reference vias as modeled in Ansoft Q3D. Two ports were assigned for each signal via: input port and output port. The reference vias and ground planes were connected together, then the input and output ports were referenced to the ground. The result was a five-port model: two differential inputs (inp & inn), two differential outputs (outp & outn), and a reference ground. The following is the extracted HSPICE model for the differential vias along with the reference vias for the Stratix GX board.

.SUBCKT diff_via_model inp inn outp outn GND C001 6 5 5.34769E-013 C002 7 5 5.34679E-013 C001_002 6 7 1.42431E-014 V001 1 8 DC 0 V002 2 9 DC 0 L001 8 10 5.49277E-010 L002 9 11 5.48365E-010 K001_002 L001 L002 0.203342 R001 10 6 0.00859401 F001R002 6 10 V002 0.0373394 R002 11 7 0.00854854 F002R001 7 11 V001 0.037538 V003 6 12 DC 0 V004 7 13 DC 0 L003 12 14 5.49277E-010 L004 13 15 5.48365E-010 K003_004 L003 L004 0.203342 R003 14 3 0.00859401 F003R004 3 14 V004 0.0373394 R004 15 4 0.00854854 F004R003 4 15 V003 0.037538 .ENDS diff_via_model

SMA Model The SMA was modeled using TDR techniques [ref]. In its most basic form, a TDR consists of a step generator and an oscilloscope. The generator launches a fast edged step signal into the transmission structure under investigation while the oscilloscope monitors the resulting waveform. Incident and reflected voltage waves superimpose on the oscilloscope display, revealing the characteristic impedance of the line and the position and nature (resistive, inductive, or capacitive) of each discontinuity along the line. Discontinuities on a printed circuit board can be characterized using this technique. In that which follows, the measured response of a transmission system that contains an SMA connector is used to determine the connector’s impedance. An equivalent circuit is then created using this impedance. Figure 9 shows the measured TDR plot for the Tyco line card and HMZD connector section. Every discontinuity on the transmission path can be seen on the TDR since the rise time of the step being sent for TDR analysis is very small, roughly 20 ps. As can be seen in the figure, both inductive and capacitive discontinuities exist, all of which cause

reflections in the transmission path. This plot shows the discontinuities associated with the line card, SMA connector, via and HMZD connector. As the plot shows, there is an inductive discontinuity due to the SMA connector on the line card. The via used on this particular board was a blind via. By observing the TDR response it was concluded that the capacitance due to this via is very small and hence negligible. Capacitances introduced by HMZD connector can be seen in the figure.

Figure 9. Measured TDR plot for Tyco linecard and the HMZD connectors.

A capacitive discontinuity on the transmission path provides an upward peak on the TDR voltage plot. This is because the impedance of an ideal, lossless transmission line is defined by

CLZo = . (2) As seen in (2), the impedance is reduced as the capacitance increases. Conversely, if a discontinuity is more inductive in nature then the curve will get pulled up. The capacitance and inductance can be calculated using the curves on a TDR plot. For a signal traveling through a capacitive dip region, the equivalent circuit looks like Figure 10.

Figure 10. Equivalent circuit for a capacitive discontinuity on the transmission line.

The two transmission lines behave as if they are in parallel with each other. The RC time constant for the circuit of Figure 10 is given by

RC = ZoC/2. (3) The rise time (Tr) and the voltage (ΔV) can be observed from the TDR plot. These values are then plugged into the equation (for Zo = 50 ohms):

(ΔV/250mV) = 1- (Tr / 2RC). (4) This equation provides the RC time constant. The RC value can also be approximated from the curve keeping in mind that 63% of the rise is RC. Values for RC can be substituted into (3) and the capacitive discontinuity as seen by the signal can be obtained. If the discontinuity is inductive in nature (the curve goes up) then the signal sees the circuit in Figure 11.

Figure 11. Equivalent circuit for an inductive discontinuity on the transmission line.

In this case,

L/R = L/2Zo. (5)

To get the inductance value we can use the equation (for Zo = 50 ohms):

(ΔV/250mV) = 1- (Tr * Zo / L). (6)

In this case, values for the rise time and ΔV were

ΔV =16 mV Tr=31 ps

(16 mV/250 mV) = 1- (31 ps * 50 / L)

L= 1.65 nH

The SMA lumped models were extracted using the TDR technique. The SMA provides an inductive discontinuity; the inductance value calculated was 1.65 nH. An inductor with value 1.65 nH was added in series with the transmission path for all SMA instances. Using the Models in HSPICE This section briefly describes the method used to construct a SPICE netlist that contains instances of the extracted models. Via: A 3D field solver was used to extract a SPICE model of the Stratix GX development board via in the “Via Modeling” section above. To use this model in a SPICE netlist, simply create a text document named, e.g., “via_model.txt” containing the netlist code given above. Then in the HSPICE deck call it using the .include command:

.include 'via_model.txt'. To create an instance of the via, use the following syntax

Xvia txoutp_20 txoutn_20 X2 Y2 GND gxvia_model

Transmission line: Include the RLGC parameters in the HSPICE deck, for example:

.model gx_board_tline W modeltype=RLGC N=2 + Lo= + 3.333860058414301e-007 + 5.469192441806368e-009 + 3.378657524075581e-007 + Co= + 1.40255495312275e-010 + -2.272806348031037e-012 + 1.402656859222826e-010 + Ro= + 7.730926791840954 + 0.09544354064001091 + 7.730926791840527 + Rs= + 0.001092578203251477 + 6.368753520505269e-006 + 0.001232929412655127 + Gd= + 1.328177142168593e-011 + -2.174986731905452e-013 + 1.329228169982364e-011

Then call it for one instance of transmission line (2000 mils long in this example),

W1 X2 Y2 GND X3 Y3 GND RLGCMODEL=gx_tline N=2 l=2000mil SMA Connector: The SMA—or any other simple discontinuity—if it is inductive it goes in series, if it is capacitive, it is tied to ground. In our case,

L1 X4 X5 1.65n L2 Y4 Y5 1.65n

Coaxial Cable: To connect the coaxial cable of length 36 inches:

W2 X5 0 outA 0 N=1 L=36000mil RLGCMODEL=m17_coaxial_cable W3 Y5 0 outB 0 N=1 L=36000mil RLGCMODEL=m17_coaxial_cable

Simulation Versus Measurement HSPICE simulations and laboratory measurements were performed on the backplane without the GX driver to confirm the accuracy of the models. An input pattern, 10101010 was sent to the linecard at one end and the output was monitored on the other

line card, at the end of a 16 inch trace on the backplane. A pattern generator (HP 7004A) was used to generate the pattern. The transmission path consisted of the SMA connector, linecard transmission line, HMZD connector, backplane transmission line, and coax cable. Figure 12 shows the lab measurement results. For simulation, the RLGC models for the backplane, line card, and the HMZD connector models provided by Tyco were included in the SPICE deck. Also, the SMA inductance, obtained from TDR, was included in the simulation. Figure 13 shows HSPICE simulation results. The differential signals were terminated with 100 ohms. Following were the specifications for the 21-inch trace:

coax cable = 35 inch Line card (each) = 2 inch Connector (each)= 0.5 inch Back plane trace= 16 inch

Figure 12. Lab measurement for the Tyco backplane, linecard, and coaxial cable with 21 inch trace.

Figure 13. HSPICE simulation for Tyco backplane, linecard and coaxial cable with 21 inch trace.

Table 2 shows the comparison between the simulation and lab measurement result in terms of amplitude and rise time.

Table 2. Simulation vs. lab measurement for backplane board with 21 inch trace.

Amplitude Rise time Simulation 420 mV 340 ps Lab measurement 400 mV 322 ps

Next, the Stratix GX driver was used instead of the pattern generator. PRBS-7 data pattern was applied to driver and the output was monitored. In all cases, the driver settings were as follows:

VOD = 1000, Pre-emphasis = 0 and Termination = 100 ohms. Figure 14 shows the set up for simulation and lab measurement. In this setup, the GX driver and its distributed package model were connected to the board transmission line, SMA and coax cable to the scope. Two different channels were used on the board with various trace length. For channel 3, the trace length of transmission line was about 1.5 inches, and for channel 1, the trace length of transmission line was about 11 inches.

Figure 14. Setup for Stratix GX driver to oscilloscope.

Figure 15 shows the differential eye diagram for the PRBS pattern out of the Stratix GX device. This channel has about 1.5 inch of trace length on the board. Figure 16 shows the differential HSPICE eye diagram with the Stratix GX driver model. The Stratix GX transmitter package model was included in the simulation. The transmission line length for this simulation was roughly 1.5 inches.

Figure 15. Lab measurement of differential eye opening using the Stratix GX driver with 1.5 inch

transmission line.

Figure 16. Simulation result of differential eye opening using the Stratix GX driver with 1.5 inch

transmission line.

Table 3 shows the comparison between the simulation and lab measurement result in terms of eye opening and rise time, when only the Stratix GX driver was used with 11-inches of transmission line.

Table 3. Simulation vs. lab measurement using Stratix GX driver with 1.5 inch transmission line.

Eye Opening Rise Time

Simulation 838 mV 170 ps Lab Measurement 854 mV 170 ps

Figure 17 shows the differential eye diagram for the PRBS pattern out of the Stratix GX device. This channel has about 11-inch of trace on the board. Figure 18 shows the differential HSPICE eye diagram using the Stratix GX driver model. The Stratix GX transmitter package model was included in the simulation. The transmission line length for this simulation was about 11 inches. Table 4 shows the comparison between the simulation and lab measurement result in terms of eye opening and rise time, when only the Stratix GX driver was used with 11-inch of transmission line.

Figure 17. Lab measurement of differential eye opening using the Stratix GX driver with 11 inch

transmission line.

Figure 18. Simulation result of differential eye opening using the Stratix GX driver with 11 inch transmission line.

Table 4. Simulation vs. lab measurement using Stratix GX driver with 11 inch transmission line.

Eye Opening Rise Time

Simulation 645mV 223 ps Lab Measurement 648 mV 231ps

Next, the Stratix GX driver was used with the backplane model. To validate the transmission line models, the Tyco backplane was added to Stratix GX board and the result were monitored after 23 and 39 inches of trace. Figure 2 shows the set up for full simulation and lab measurement. In this setup, the Stratix GX driver and transmitter package model were connected to the board transmission line, via and the board SMA. It was also connected to the Tyco backplane which consisted of line card, HMZD connector, 16 or 32 inch backplane, another HMZD connector, another line card and finally the coax cable to the scope. Figure 19 shows the differential eye diagram from lab measurement when the driver of the Stratix GX device was driving out on the HSSI side (channel 3) and it was connected to the Tyco back plane. The transmission line length on Stratix GX board was about 1.5-inches, the trace length on Tyco line card was 2 inches, on HMZD connecter was about 0.5 inches, and Tyco backplane was 16 inches, so the total trace length was about 23 inches (1.5+2.5+16+2.5).

Figure 19. Differential eye measured in the lab with 23 inch trace.

Figure 20 shows the simulation result when 20-inch of trace was added to the Stratix GX driver. The input data pattern applied was PRBS-7. For this simulation, the RLGC models for line card and back plane and the HSPICE model for HMZD connector were included in the simulation. To represent the SMA discontinuity, the inductance value obtained from TDR was included in the simulation.

Figure 20. Simulated differential eye with 23 inch trace.

Table 5 shows the comparison between the simulation result and lab measurement in terms of eye opening and rise time, when 23-inch of trace was used.

Table 5. Simulation vs. lab measurement for Stratix GX driver with 23 inch trace.

Eye Opening Rise Time Simulation 370mV 265 ps Lab Measurement 330 mV 250 ps

Figure 21 shows the differential eye diagram from the lab measurement when the driver of the Stratix GX was driving out on the HSSI side (channel 3) and was connected to the Tyco back plane. The transmission line length on the Stratix GX board was about 1.5 inches, the trace length on Tyco line card was 2 inches, on HMZD connecter was about 0.5 inches, and Tyco backplane board was 32 inches, so the total trace length was about 39 inches (1.5+2.5+32+2.5).

Figure 21. Differential eye measured in the lab for 39 inch trace.

Figure 22 shows the differential eye diagram from simulation result when 39 inches of trace was added to the Stratix GX driver. The input data pattern applied was PRBS-7. For this simulation, the RLGC models for linecard and back plane along with HMZD connector model were included in the simulation. To represent the SMA discontinuity, the inductance value obtained from TDR was included in simulation.

Figure 22. Simulated differential eye for 39 inch trace. Table 6 shows the comparison between the simulation result and lab measurement in terms of eye opening and the rise time, when 39 inches of trace was added.

Table 6. Simulation vs. lab measurement for Stratix GX driver and 39 inch trace.

Eye Opening Rise Time Simulation 252 mV 306ps Lab Measurement 200 mV 300ps

Conclusion Modern platform FPGA devices provide wide bandwidth processing and high-speed I/O. Serial I/O with speeds in the Gigabit realm creates new challenges for printed circuit board designers. This paper showed how to combine EM simulations, lab measurements, and HSPICE simulations to predict interconnect performance for systems that contain PCB traces, vias, connectors, and cables. Results for the test structure including an FPGA development board, a backplane, daughter cards, and associated connectors and coax cables show that accurate modeling can be achieved using this method. Simulation and measurement showed that models provided in this paper are reliable for system interconnects up to 12 inches in length; beyond 12 inches simulation accuracy is limited. Discrepancies between measured and simulated results became progressively worse as the backplane trace length was increased; the simulations under-predicted the backplane trace loss. Further investigation of the transmission line material and model parameters may reveal the cause of these minor discrepancies.