co-optimization of signal, power, and thermal distribution … · · 2009-02-28concerns in the 3d...

Co-Optimization of Signal, Power, and Thermal DistributionNetworks for 3D ICs

Young-Joon Lee and Sung Kyu LimElectrical And Computer Engineering, Georgia Institute of Technology

777 Atlantic Drive NW, Atlanta GA 30332, U.S.A.{yjlee, limsk}@ece.gatech.edu

Abstract— Heat removal and power delivery are two major reliabilityconcerns in the 3D stacked IC technology. Liquid cooling based on micro-fluidic channels is proposed as a viable solution to dramatically reducethe operating temperature of 3D ICs. In addition, designers use a highlycomplex hierarchical power distribution network in conjunction withdecoupling capacitors to deliver currents. However, these thermal andpower/ground interconnects together with those used for signal deliverycompete with each other for routing resources including various typesof Through-Silicon-Vias (TSVs). This paper presents the first work onrouting with these multi-functional interconnects in 3D: signal, thermal,and power distribution networks. We demonstrate how to considervarious physical, electrical, and thermo-mechnical requirements of thesemulti-functional interconnects to successfully complete routing whileaddressing various reliability concerns.

I. INTRODUCTION

It is widely accepted that three-dimensional (3D) system integra-tion is a key enabling technology and has recently gained significantmomentum in the semiconductor industry. However, there are anumber of interconnect challenges that need to be addressed to enablestacking of high-performance dice, especially in the area of cooling.When two 100W/cm2 microprocessors are stacked on top of eachother, for example, the net power density becomes 200W/cm2, whichis beyond the heat removal limits of currently available air cooled heatsinks. Thus, cooling is the key limiter to stacking of high-performancechips today. This issue has recently been addressed with a novel 3Dintegration technology that features the use of a microchannel heatsink in each strata of the 3D system and the use of wafer-level batchfabricated electrical and fluidic chip input/output (I/O) interconnects[1].

Another major challenge is power delivery. As the fabricationtechnology advances, power consumption of the chip increases.As multiples chips are stacked together into a smaller footprint,delivering current to all parts of the 3D stack while meeting thenoise constraints becomes highly challenging. This is mainly becausethe number of through-silicon-vias (TSVs) available for signal netsand P/G nets is limited, causing routing congestion if many 3Dconnections are desired. This issue is further exacerbated if micro-fluidic channels (and TSVs) are used for liquid cooling. Figure 1shows a possible configuration of microfluidic channels and signaland P/G TSVs, all competing for layout resource.

We present the first work on routing with multi-functionalinterconnects—signal, power, thermal—in 3D ICs. We demonstratehow to consider various physical, electrical, and thermo-mechnicalrequirements of these multi-functional interconnects to successfullycomplete routing while addressing thermal and noise concerns. Wedemonstrate the effectiveness of our approach using large-scale gate-level benchmarks that contain up to 1.2 million gates. We reportrouting congestion, thermal distribution, and power supply noisebased on full-layouts of 4-die 3D IC.

Fig. 1. A 3D die structure with microfluidic channels, power/ground TSVs,and signal TSVs. Transistors and signal wires are not shown for simplicity.

II. THERMO-FLUIDIC INTERCONNECT FOR 3D ICS

A. Fabrication of Microfluidic Channels and TSVs

Unlike air-cooled heat sinks, liquid cooling using microchannelsoffers a larger heat transfer coefficient (and thus lower convectivethermal resistance) and chip-scale cooling solution. In [1], bothelectrical and fluidic TSVs and I/Os were demonstrated. The electricalinterconnects are used for power delivery and signaling betweenstrata, and the fluidic interconnects are used to deliver a coolantto each microchannel heat sink in the 3D stack and thus enablethe rejection of heat from each stratum in the 3D stack. The chipsare designed such that when they are stacked, each chip makeselectrical and fluidic interconnection to the dice above and below.Consequently, power delivery and signaling can be supported by theelectrical interconnects (solder bumps and copper TSVs), and heatremoval for each stratum can be supported by the fluidic I/Os andmicrochannel heat sinks. The thermal resistance of the microchannelheat sink for single chip was previously measured [2]. When de-ionized water was used as coolant, the junction-to-ambient thermalresistance of the heat sink was 0.24◦C/W at a flow-rate of about65mL/min without TSVs (impact of copper TSVs on thermalconductivity of the silicon microchannel wall is negligible), which issignificantly better than current state-of-the-art air cooled heat sinks[2]. The smallest resistance possible for air-cooled heat sink is around0.5◦C/W .

B. Routing Requirements for Thermo-fluidic Network

The on-chip thermo-fluidic network is composed of fluidic TSVsand microfluidic channels. In this work, we assume that all the fluidicTSVs are located outside the main region where all the gates andmetal wirings are distributed. Thus, only the microfluidic channelsare considered for routing requirement analysis. Since microfluidicchannels are fabricated on the back side of a silicon die, they do

978-1-4244-2634-8/08/$25.00 ©2008 IEEE 2008 Electrical Design of Advanced Packaging and Systems Symposium163

Authorized licensed use limited to: Georgia Institute of Technology. Downloaded on February 28, 2009 at 18:55 from IEEE Xplore. Restrictions apply.

10 um 400 um

thick P/G wirethin P/G wire

Purple: M7 - horizontalLight Blue : M8 - vertical

P G P

G

P

G

signal wire

1um

0.28 um0.56 um

power TSVground TSV

40 u

m40

0 u

m

4.64 um

via

P/G tile

Fig. 2. A top view of dual-mesh structured power and ground net. Only apart of the chip is shown.

not affect routing capability on metal layers. More important issue isthat these channels obstruct TSV connections. Considering significantsize of the microfluidic channels, they decrease the routing capacityof TSVs quite considerably. The scarce resource is usually the signalTSV capacity. Due to the microfluidic channels, many routing tileshave no capacity left for signal TSVs.

III. POWER DELIVERY NETWORK FOR 3D ICS

Overall power and ground structure is shown in Figure 2. In ourwork, power and ground (P/G) TSVs are placed regularly in a meshstructure with a predefined pitch (pV pg = 400µm). The width of aP/G tile is a half the power TSV pitch (wpg tile = pV pg/2 = 200µmand contains one quarter of power TSV and one quarter of groundTSV. The total number of P/G TSVs in a die is:

NV pg =

⌊Schip

w2pg tile

⌋× (1/4 + 1/4)

where Schip is the area of the chip.Power and ground nets are routed on metal layer 7 and 8. The

following three-levels of wiring hierarchy is used:• Thick wires have 10µm width and runs between P/G TSVs.• Between the thick wires, thin wires of 1µm width and 4.64µm

pitch are placed.• Between two thin wires, up to six signal wires can be routed.In our 3D technology, two TSVs from two different dies are

connected using the metal layers and vias in these dies. This meanstwo stacked signal TSVs are not directly connected. On the otherhand,P/G TSVs pierce through all 4 dies for efficient power delivery (seeFigure 3). Thus, no cell can be placed at the P/G TSV locations.Considering the size of power and ground TSVs, this area is notnegligible. Thus, the total area required for P/G TSVs can becalculated by multiplying the number of P/G TSVs by the P/G TSVarea: Stot V pg = NV pg×SV pg . Based on our structural assumptions,P/G TSVs occupy around 2% of the chip area. In the routing tile withsome part of a P/G TSV, routing capacity is decreased by a largeamount as discussed in Section IV-B.

IV. SIGNAL INTERCONNECT FOR 3D ICS

We perform global routing onto nr × nr grid. In our routing, thethermal, power, and signal nets are routed sequentially in this order.We first route the microfluidic channels, followed by the P/G TSVs on

40front

back

bulk Si

metal layer

active layer

10

ground TSV signal TSVMF channelpower TSV

100 100 100

100

50

40

Fig. 3. Side view of a die layer in a stacked chip. Die is flipped and theactive layer is facing down. Shapes are drawn to scale. Unit is µm. Note thatin our technology assumption, P/G TSVs span the entire 4-die stack, whilesignal TSVs only span single die, requiring metal layers and vias for signalTSV-to-TSV connection.

routing tiles with no microfluidic channels. Lastly, we add P/G wires(thick and thin). The remaining area is used for signal net routing.Since the thermal and P/G interconnects are routing obstacles, we usethe thermal-aware 3D maze router for obstacle avoidance under thegiven thermal profile [3]. After the routing is finished, we run powernoise and thermal analysis to see if given constraints are satisfied.We may repeat the entire or some parts of the physical design stepswith more routing area if needed.

A. Geometries of Wires and Vias

As for the signal wires, we use the metal interconnect dimensionsfrom Intel’s 45nm technology [4]. The TSV formation approach wasassumed to be via-first. In via-first approach, TSVs are formed beforemetal wirings are constructed. From the signal routing point of view,this is beneficial because TSVs do not touch the metal interconnectionlayer. By contrast, in via-last approach, a TSV passes through theentire structure and decreases the metal wire interconnection spaceby a large amount. TSV aspect ratio (AR, the ratio of the die thicknessto the minimum TSV diameter) was assumed to be 15:1.

Figure 3 shows the side view of a die. The diameter of signal TSVsis set to the minimum size to accommodate as many connections aspossible. In contrast, the diameter of P/G TSVs is 40µm, whichis comparable to our previous work [5]. Table II provides moredetails on related geometries. We fix the width of our routing tilewr tile = 50µm. Since routing tile is square, Sr tile = w2

r tile. Thetotal number of grids is calculated as: Schip/Sr tile. Figure 4 showsthe routing tile objects.

B. Routing Capacity Calculation

For each routing tile, there are x-, y-, and z-direction routingcapacity values. x- and y-direction capacity represents availablerouting space on metal layers, while z-direction capacity is for signalTSVs. Basically, x- and y-direction capacity values of a metal layerare calculated from dividing the routing tile size by the pitch of themetal layer. Since the benchmark circuits used in this study are gate-level designs, metal layer 1 and 2 are used for local routing. Thus, weassume that only 20% of the routing capacity is available in metallayer 1 and 2. Metal 3-6 are dedicated to signal routing. In metal7 and 8, we decrease number of routing capacity values due to theP/G nets. The capacity values based on multiple metal layer stackare added together for each tile. If the tile is pre-occupied with P/GTSVs, we decrease the capacity accordingly.

2008 Electrical Design of Advanced Packaging and Systems Symposium164


100

50

20050

100

Routing Tile Fluidic TSV

100

Microfluidic Channel

10

0

P/G TSV Signal TSV

M7/M8

40 40 10

50

0.2

Fig. 4. Top view of routing tile. Objects are drawn to scale.

TABLE IISPD 2006 BENCHMARK CIRCUITS. FOOTPRINT AREA IS IN mm2 AND

WIDTH IN µm. WE ALSO REPORT THE DIMENSIONS OF THE ROUTING,P/G, AND THERMAL GRIDS BASED ON THE CHIP AREA.

adaptec1 newblue1 newblue3 adaptec5 newblue5# cells 211,447 330,474 494,011 843,128 1,233,058# nets 221,142 338,901 552,199 867,798 1,284,251area 36 56.25 64 225 256

width 6,000 7,500 8,000 15,000 16,000r-grid 120x120x4 150x150x4 160x160x4 300x300x4 320x320x4p-grid 30x30x4 37x37x4 40x40x4 75x75x4 80x80x4t-grid 30x80x4 37x80x4 40x80x4 75x80x4 80x80x4

For z-direction capacity, we calculate the remaining surface areaof each routing tile. Starting from the routing tile area, we extract theplaced cell area and the P/G TSV area. Since we place P/G TSVsat the center of four routing tiles, only one quarter of P/G TSVis included in a routing tile. Then, we divide the resulting area bythe minimum pitch signal TSV area. We set our routing tile size to50×50µm, which results in the following three types of routing tilesfor signal net routing:• type 1 (contains no obstacle): 618 wires for x/y-direction wires

and 6 signal TSVs.• type 2 (contains P/G interconnect obstacles): 371 wires for x/y-

direction wires and 5 signal TSVs.• type 3 (contains microfluidic channel obstacles): 618 wires for

x/y-direction wires and 0 signal TSVs.In case of x-direction interconnects, the power and thermal TSVs

occupy 2.5% and 0% of the available routing area, respectively. Thus,97.5% of x-direction routing resource is available for signal netrouting. The same is true for y-direction. In case of z-direction, thepower TSVs and the thermal interconnects (= microfluidic channels)occupy 2% and 50% of the available routing area, respectively. Thus,48% of x-direction routing resource is available for signal TSVs.

V. EXPERIMENTAL RESULTS

We implemented our routing package in C++/STL and Matlab, andran the package on a 64 bit Linux server with two AMD DualcoreOpteron [email protected] CPUs and 16GB main memory. The circuitsare from the ISPD 2006 Placement Contest benchmark that rangefrom 200K to 1.2M gates as shown in Table I. We also show the chipsize. The dimension of routing, P/G, and thermal grids is calculatedfrom the chip size. The technology and setting parameters used inour experiments are shown in Table II.

TABLE IITECHNOLOGY PARAMETERS USED IN OUR EXPERIMENTS.

Item ValueNumber of dies 4Bonding type face-to-backDie thickness (µm) 150Bonding layer thickness (µm) 10TSV aspect ratio 15:1Routing grid size (µm) 50Signal TSV diameter (µm) 10Signal TSV minimum pitch (µm) 20P/G TSV diameter (µm) 40P/G TSV pitch (µm) 400P/G grid size (µm) 200Microfluidic channel width (µm) 100Microfluidic channel pitch (µm) 200Microfluidic channel depth (µm) 100

x [mm], (Channel width)z [

mm

], (

Flu

id f

low

)

Wall temp. [oC] − Die 1

0 4 8 12 160

4

8

12

16

20

30

40

50

60

x [mm], (Channel width)

z [

mm

], (

Flu

id f

low

)


0 4 8 12 160

4

8

12

16

20

30

40

50

60


z [

mm

], (

Flu

id f

low

)


0 4 8 12 160

4

8

12

16

20

30

40

50

60


z [

mm

], (

Flu

id f

low

)


0 4 8 12 160

4

8

12

16

20

30

40

50

60

Fig. 6. Silicon wall temperature of newblue5.

Figure 5 shows x, y, and z-direction routing tile utilization ondie 1 for newblue5. We note that the congestion of z-direction ismore severe than that of x/y-directions. In this case, a large routingcongestion “hotspot” is located in the bottom of the chip.

Figure 6 shows that the die temperature is maintained well below85◦C using microfluidic cooling. Pressure drop between inlet andoutlet was constrained to 70kPa for all microfluidic channels. Inletworking fluid temperature was assumed to be 20◦C. The maximumtemperature difference between the die and the working fluid isonly around 40◦C, which means that the convective heat transfercoefficients was higher than estimated and/or thermal diffusion dueto conduction played a significant role.

Table III shows the summary of thermal analysis results. For allthe cases, the coolant temperature never exceeded 40◦C. Maximumsilicon wall temperature was kept under 65◦C for all die layers. Thismeans that microfluidic scheme had enough cooling capability forthermal management of the simulated 3D circuits. Standard deviationof overall wall temperature was around 5◦C.

Figure 7 and 8 show the power noise level and the voltagewaveform at the grid with maximum peak noise on the top die ofnewblue5. Only top die result is shown because it is the farthestone from power supply and thus tends to show the highest powernoise level. Some parts have greater power noise due to higher powerdemand. The fluctuation of voltage is mainly due to the parasiticinductance of package and the capacitance of the wires.



x [grid index]

y [

grid

in

de

x]

50 100 150 200 250 300

50

100

150

200

250

300

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

x [grid index]

y [

grid

in

de

x]

50 100 150 200 250 300

50

100

150

200

250

300

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

x [grid index]

y [

grid

in

de

x]

50 100 150 200 250 300

50

100

150

200

250

300

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

(a) (b) (c)

Fig. 5. Routing usage of newblue5: (a) x-direction, (b) y-direction, (c) z-direction (= between die 1 and die 2). The horizontal white lines denote microfluidicchannels.

TABLE IIISUMMARY OF THERMAL ANALYSIS RESULTS. TEMPERATURES ARE

REPORTED IN ◦C , AND POWER AND POWER DENSITY VALUES ARE IN

WATTS AND W/cm2 , RESPECTIVELY.

adaptec1 newblue1 newblue3 adaptec5 newblue5# hotspots 42 66 98 168 245ave die power 13.86 21.49 33.31 56.17 80.78ave pwr density 38.50 38.20 52.05 24.96 31.55ave fluid temp. 21.21 21.85 22.85 24.94 26.73max fluid temp. 23.73 25.63 28.60 33.53 37.40ave wall temp. 30.92 31.50 35.94 31.22 34.65max wall temp. 53.44 50.58 60.14 55.25 62.36σ wall temp. 4.24 4.30 5.83 4.66 6.34pump power 0.380 0.376 0.388 0.397 0.411

x [grid index]

y [grid index]

5 10 15 20 25

5

10

15

20

25 0.105

0.11

0.115

0.12

0.125

Fig. 7. Peak power noise on the top layer of newblue5. Unit is V .

Table IV shows the summary of power noise simulation results.Peak power noise values are quite high for all circuits. It was foundthat the maximum peak power noise location has high correlationwith the maximum power consumption location. This is because weassumed the worst-case scenario in which the current profile for theentire stack is simultaneously switching on.

VI. CONCLUSIONS

In this paper, we presented the first work on routing with sig-nal, thermal, and P/G interconnects. Our studies revealed that theliquid cooling based on micro-fluidic channels is highly effective inremoving the hotspots in 3D designs. We also learned that the P/Gdistribution network for 3D IC requires a high demand on TSVsand interfere with signal net routing. The major signal net routingbottleneck was related to TSVs, since these signal TSVs compete with

no

ise

(m

V)

-120

-100

-80

-60

-40

-20

0

time (ns)0 10 20 30 40

Fig. 8. Waveform of the grid with maximum peak noise in newblue5.

TABLE IVSUMMARY OF POWER NOISE SIMULATION RESULTS. WE REPORT AVERAGE

AND MAXIMUM PEAK POWER NOISE IN mV .

adaptec1 newblue1 newblue3 adaptec5 newblue5ave 174.2 163.2 164.5 91.6 113.1max 210.5 181.4 190.9 115.3 127.2

P/G TSVs and micro-fluidic channels for vertical routing resources.

REFERENCES

[1] M. S. Bakir, C. King, D. Sekar, H. Thacker, B. Dang, G. Huang,A. Naeemi, and J. D. Meindl, “3D Heterogeneous Integrated Systems:Liquid Cooling, Power Delivery, and Implementation,” in Proc. IEEECustom Integrated Circuits Conf., 2008.

[2] M. Bakir, B. Dang, and J. Meindl, “Revolutionary NanoSilicon AncillaryTechnologies for Ultimate-Performance Gigascale Systems,” in Proc.IEEE Custom Integrated Circuits Conf., 2007, pp. 421–428.

[3] M. Pathak and S. K. Lim, “Thermal-aware Steiner Routing for 3D StackedICs,” in Proc. IEEE Int. Conf. on Computer-Aided Design, 2007, pp. 205–211.

[4] K. M. et.al., “A 45nm Logic Technology with High-k+Metal GateTransistors, Strained Silicon, 9 Cu Interconnect Layers, 193nm DryPatterning, and 100% Pb-free Packaging,” in IEEE International ElectronDevices Meeting, 2007, pp. 247–250.

[5] G. Huang, M. Bakir, A. Naeemi, H. Chen, and J. Meindl, “Power Deliveryfor 3D Chip Stacks: Physical Modeling and Design Implication,” in Proc.IEEE Electrical Performance of Electronic Packaging, 2007, pp. 205–208.



co-optimization of signal, power, and thermal distribution … · · 2009-02-28concerns in the 3d...

Documents