FdSPICE: A Parallel Simulation Technique for Lossy and Dispersive Interconnect Networks

Jian Wang, Min Tang, Lizhuang Ma, and Junfa Mao Key Laboratory of Ministry of Education of China for Research of Design and Electromagnetic

Compatibility of High Speed Electronic Systems, Shanghai Jiao Tong University, China, 200240 xiaohei520321@sjtu.edu.cn

tm222@sjtu.edu.cn

Abstract—This paper presents a parallel algorithm for the simulation of large complex interconnect networks with frequency-dependent parameters. The equivalent transmission line model, which is suitable for time-domain simulation, is introduced for the modeling of dispersive interconnects. By means of the equivalent multi-port model, the interconnects are separated from other circuit elements, and thus makes the parallel simulation feasible. Based on the proposed technique, a program named FdSPICE is developed for parallel simulation of large interconnect networks. The accuracy and efficiency of FdSPICE are demonstrated by numerical examples.

I. INTRODUCTION With the rapid increase in operating frequencies and

continuous decrease in feature sizes of modern very large-scale integration (VLSI) technology, the electrical property of interconnects has become a key factor in determining the overall electrical performance of high-speed circuits and systems [1-3]. Nevertheless, accurate and efficient simulation of interconnects is still a challenge task even in the most advanced circuit solver. One of the difficulties lies in the fact that the interconnect characteristics are best described in the frequency domain whereas network nonlinearities and/or time-dependent components require a time domain analysis. The second challenge is that due to skin, edge, and proximity effects and lossy substrates, the frequency dependence of interconnect parameters becomes more and more popular in practical cases and must be taken into account for accurate circuit simulations. The third problem comes from the high computational cost and low efficiency in simulating large interconnect networks.

Numerous approaches have been proposed for the modeling and transient analysis of lossy interconnects with nonlinear termination networks [4-8]. However, most conventional algorithms are not efficient in the simulation of large complex interconnect networks. One of the main reasons is that they are not suitable for the parallel processing environment. With the rapid development of distributed and parallel computing techniques, new algorithms should be found which are suited for the solution of large problems using parallel processors. In recent years, a parallel algorithm based on transverse partitioning and waveform relaxation techniques has been proposed for efficient simulation of massively coupled interconnects [8]. The computational cost grows almost linearly with the number of lines. Nevertheless, this algorithm

doesn’t hold advantage in simulating small coupled interconnects.

In this paper, a new parallel technique is presented to address the simulation of large interconnect networks with frequency-dependent parameters. By means of the equivalent multi-port model, the proposed method is highly suitable for parallel implementation since the interconnects are separated from other circuit elements and the simulation of individual interconnects can be performed simultaneously. The finite-difference time-domain (FDTD) method is utilized for the computation of interconnects while other circuit elements are simulated in the SPICE environment. Their simulation results are exchanged in every time step. Numerical examples are used to demonstrate the accuracy and efficiency of the proposed method.

II. MODELING OF INTERCONNECTS For M -coupled interconnects, it is assumed that the

frequency-dependent parameters of the lines are obtained at a set of discrete frequencies that span the frequency bandwidth of interest. The vector-fitting algorithm [9] followed by passive check and compensation technique [10] is used to generate the positive-real closed form of the p.u.l. series impedance matrix and parallel admittance matrix of the lines:

1

0 01

1( )Q

qq q

s ss P=

≅ + +−∑Z R L R (1)

2

0 01

1( )Q

qq q

s ss B=

≅ + +−∑Y G C G (2)

where 1Q and 2Q represent the number of poles used in the

rational approximation. For the convenience of time-domain simulation, a set of

differential equations have been derived based on the equivalent circuit models of frequency-dependent interconnects as below [11].

1

0 01

0Q

q qqx t =

∂ ∂+ + − =∂ ∂ ∑v iR i L R i (3a)

0qq q q qt

∂+ − =

∂i

R i L R i (3b)

2

0 01

0Q

q qqx t =

∂ ∂+ + − =∂ ∂ ∑i vG v C G v (4a)

0qq q q qt

∂+ − =

∂v

G v C G v (4b)

Note that the parameters in these equations are all frequency-independent. Their detailed expressions are described in [11].

The FDTD algorithm with second-order accuracy is utilized to solve these differential equations. Assume that the spatial interval is xΔ and the time step is tΔ . Each voltage and adjacent current solution point is separated by 2xΔ as shown

in Fig. 1. The central-difference algorithm is employed to discretize

these differential equations. The update equations of current and voltage have been described in [11]. Due to limitations of space, here we only present the equivalent multi-port model, which is the key factor in the proposed algorithm. The compact expression of the model is given by

1 1 2 11

m m mS SH eq

+ + += +i i G v (5) 1 1 2 1

1m m mL LH eq N

+ + ++= +i i G v (6)

where 1mS

+i and 1mL

+i are the terminal currents, 11m+v and 1

1mN

++v

are the terminal voltages, 1 2mSH

+i and 1 2mLH

+i are the equivalent current sources, and eqG is the equivalent conductance. The

detailed circuit model is depicted in Fig. 2, where the conductance values are given by

( )1

,M

j eqm

G G j m=

=∑

and

( ), ,j k eqG G j k= − ( ), 1, ,j k M= (7)

1v 2v 1Nv +Nv

1i Ni2i

Fig. 1 Sampling point along the interconnect

( )11

+mv j

( )1+mSi j

jG

,j kG

( )1 2+mSHi j

( )11

+mv k

( )1+mSi k

kG

jG

kG

( )11

++

mNv j

( )11

++

mNv k( )1 2+m

SHi k

( )1 2+mLHi j

( )1 2+mLHi k

( )1+mLi j

( )1+mLi k

,j kG

Fig. 2 Detailed equivalent multi-port model for SPICE simulation

The working procedure can be described as follows. At each time step, the FDTD algorithm is called to update the equivalent current sources SHi and LHi . Then, the equivalent

multi-port model together with the terminal networks is solved in SPICE. After that, terminal currents and voltages are returned to the FDTD algorithm for the update of equivalent current sources at the next time step.

The advantage of the equivalent multi-port model is twofold. Firstly, it is compatible with SPICE environment, which contributes to the convenience of dealing with arbitrary nonlinear termination networks. Secondly, it separates the interconnects from other circuit elements, which makes the proposed algorithm highly suitable for parallel implementation.

III. PARALLEL SIMULATION TECHNIQUE In the following, a parallel algorithm is introduced for

interconnect simulation using the C++ programming language based on a cluster of personal computers (PCs). The socket technique is used for client-server communications between the master and slave nodes as shown in Fig. 3.

An important issue in the parallel algorithm is to partition the interconnect simulation task, which is executed by the master node. It is assumed that the number of interconnects is N in a large network and the number of PCs is P ( N is usually much larger than P ). Here, we use the technique of uniform partitioning. That is, the thi slave node is assigned to simulate the interconnects from 1 1iP− + to iP . We have

iP iN P= ⎢ ⎥⎣ ⎦ (8)

where ⎢ ⎥⎣ ⎦ rounds the elements to the nearest integers.

The SPICE simulation is executed in the master node while the interconnect calculation is implemented in the slave nodes. It is worth noting that in order to save computing resources the master node also behaves as a slave one during the calculation of interconnects. At every time step, the master and slave nodes will communicate twice. During the first time, the master node transfers the data of terminal voltages and currents of interconnects to the slave nodes. Then, when the slave nodes complete their computing task of interconnects, they will return the information of equivalent current sources to the master node as the second communication.

Master node

Slave node 1

Socket communication

Slave node 2 Slave node P-1

Fig. 3 Distributed network model

IV. NUMERICAL RESULTS

In order to validate the accuracy of the proposed method, we first investigate a simple interconnect networks as shown in Fig. 4. The distributed frequency-dependent parameters of interconnects are obtained from [12]. The input voltage is a saturated ramp with rise time 100 ps. In FDTD calculation, the time step is chosen to be 1/20 of the rise time of input signal and the spatial interval is specified to satisfy the Courant limit in order to ensure the stability of the algorithm. The transient voltages at several sampling nodes of the interconnection system are depicted in Figs. 5 and 6. Due to the linearity of the termination loads, the reference results are achieved by the inverse fast Fourier transform method [13] with high accuracy. It is shown that the results from both methods are in very good agreement. Note that not only the delays are exactly modeled using FdSPICE, but also the crosstalk voltages at the coupled lines are well simulated.

The second example is a large interconnect network which consists of 30 subcircuits, see Fig. 7. In every subcircuit, the distributed parameters of both single and two-coupled interconnects are referenced from [12], while those of the

1pF

1pF

0.5pF

10Ω

( )e t

1pF1 0.5l cm=

2 1l cm=

0.5pF

3 0.5l cm=A

30Ω

B

1pF

C

D

Fig. 4 Coupled interconnect system

Fig. 5 Transient voltages at points A and D

Fig. 6 Transient voltages at points B and C four-coupled interconnects are obtained from [14]. The nonlinear load is characterized by the relation

( )10 exp 1 nATi v V= −⎡ ⎤⎣ ⎦

where i is the current flowing through the load, v is the corresponding voltage drop, and 25mVTV = . The input

voltage source is the same to the first example. The transient voltages at the ends of first several subcircuits are depicted in Fig. 8. It is observed that due to the interconnect effects, the deterioration of attenuation and transmission delay become more and more serious. At the end of the fourth subcircuit, the maximum magnitude of the signal is less than 0.01V. On a PC with Intel Duo CPU 2.93GHz and 4GB RAM, the total simulation time is 50.6s, in which the simulation of interconnects takes 32.9s. Considering that the part of interconnects simulation is highly suitable for parallel implementation, we utilize several PCs to improve its efficiency. The number of PCs and the corresponding simulation times are listed in Table I. It is shown that the simulation time of interconnects can be reduced significantly by using the parallel computing technique. Note that since the SPICE simulation is not executed in parallel, the speed-up in the table only refer to the simulation time of interconnects, but not the total CPU time.

TABLE I SIMULATION TIME AND PARALLEL SPEEDUP

Number of PC

Total CPU time

Interconnect simulation Speed-up*

1 50.6s 32.9s -- 3 30.0s 12.0s 2.7 4 27.6s 9.9s 3.3 5 26.3s 8.6s 3.8

* Refer to the simulation time of interconnects.

2Ω5nH 2Ω3nH

5Ω

2pF

1pF

1pF

1pF1pF

4 0.5cml =3 0.5cml =

1pF

( )e t

1pF 1pF

1pF 1pF 1pF1pF

2 10cml =

5 10cml =6 5cml =

( )i f v=

1 1cml =

( )i f v=

2Ω5nH 2Ω3nH

5Ω

2pF

1pF

1pF

1pF1pF

4 0.5cml =3 0.5cml =

1pF

1pF 1pF

1pF 1pF 1pF1pF

2 10cml =

5 10cml =6 5cml =

( )i f v=

1 0.5cml =

( )i f v=

2Ω5nH 2Ω3nH

5Ω

2pF

1pF

1pF

1pF1pF

4 0.5cml =3 0.5cml =

1pF

1pF 1pF

1pF 1pF 1pF1pF

2 10cml =

5 10cml =6 5cml =

( )i f v=

1 0.5cml =

( )i f v=

2Ω5nH 2Ω3nH

5Ω

2pF

1pF

1pF

1pF1pF

4 0.5cml =3 0.5cml =

1pF

1pF 1pF

1pF 1pF 1pF1pF

2 10cml =

5 10cml =6 5cml =

( )i f v=

1 0.5cml =

( )i f v=

Fig. 7 Complex interconnect network

Fig. 8 Transient voltages at the ends of different subcircuits

V. CONCLUSIONS

A new method based on the combination of the FDTD algorithm and SPICE solver is presented for the simulation of large interconnect networks with frequency-dependent parameters. The equivalent transmission line model is introduced for the modeling of dispersive interconnects. The equivalent multi-port model of interconnects not only contributes to the convenience of dealing with nonlinear termination networks, but also separates the interconnects

from other circuit elements, which makes the proposed algorithm highly suitable for parallel implementation. The FDTD method is employed for the solution of interconnects in a parallel way while other circuit elements are simulated in the SPICE environment. Numerical experiments show that the proposed algorithm is accurate and efficient in simulating large interconnect networks.

ACKNOWLEDGMENT This work was supported by the National Science Fund for

Creative Research Groups of China (60821062) and the National Basic Research Program of China (2009CB320202).

REFERENCES [1] H. B. Bakoglu, Circuits, Interconnections and Packaging for VLSI.

Reading, MA: Addison-Wesley, 1990. [2] R. Achar and M. Nakhla, “Simulation of high-speed interconnects,”

Proc. IEEE, vol. 89, no. 5, pp. 693–728, May 2001. [3] A. Deutsch et al., “Frequency-dependent losses on high-performance

interconnections,” IEEE Trans. Electromagn. Compat., vol. 43, no. 4, pp. 446–465, Nov. 2001.

[4] D. B. Kuznetsov and J. E. Schutt-Ainé, “Optimal transient simulation of transmission lines,” IEEE Trans. Circuits Syst. I, vol. 43, no. 2, pp. 110–121, Feb. 1996.

[5] A. Dounavis, R. Achar, and M. S. Nakhla, “Efficient passive circuit models for distributed networks with frequency-dependent parameters,” IEEE Trans. Advanced Packag., vol. 23, no. 3, pp. 382–392, Aug. 2000.

[6] A. Orlandi and C. R. Paul, “FDTD analysis of lossy, multiconductor transmission lines terminated in arbitrary loads,” IEEE Trans. Electromagn. Compat., vol. 38, no. 3, pp. 388–399, Aug. 1996.

[7] G. Antonini, “A new methodology for the transient analysis of lossy and dispersive multiconductor transmission lines,” IEEE Trans. Microw. Theory Tech., vol. 52, no. 9, pp. 2227–2239, Sep. 2004.

[8] Natalie Nakhla, Albert Ruehli, Michel Nakhla, Ram Achar, and Changzhong Chen, “Waveform relaxation techniques for simulation of coupled interconnects with frequency-dependent parameters,” IEEE Trans. Advanced Packag., vol. 30, no. 2, pp. 257–269. May 2007.

[9] B. Gustavsen and A. Semlyen, “Rational approximation of frequency domain responses by vector fitting,” IEEE Trans. Power Delivery, vol. 14, no. 3, pp. 1052–1061, 1999.

[10] D. Saraswat, R. Achar, and M. Nakhla, “A fast algorithm and practical considerations for passive macromodeling of measured/simulated data,” IEEE Trans. Adv. Packag., vol. 27, no. 1, pp. 57–70, Feb. 2004.

[11] Min Tang and Junfa Mao, “Finite-difference analysis of interconnects with frequency-dependent parameters based on equivalent circuit models”, IEEE Trans. Advanced Packag., vol. 33, no. 2, pp: 457-467, May 2010.

[12] A. C. Cangellaris, K. M. Coperich, H. M. Huang and A. Ruehli, “Investigation of the numerical accuracy of transmission line-based transient analysis of high-speed interconnects and cables,” Proc. IEEE Int. Symp. Electromagn. Compat., vol. 2, 2001, pp. 821–826.

[13] A. R. Djordjevic and T. K. Sarkar, “Analysis of time response of lossy multiconductor transmission line network,” IEEE Trans. Microwave Theory Tech., vol. 35, no. 10, pp. 898–907, Oct. 1987.

[14] D. Mardare and J. LoVetri, “The finite-difference time domain solution of lossy MTL networks with nolinear junctions,” IEEE Trans. Electromagn. Compat., vol. 37, no. 2, pp. 252–259, May 1995.