intelligent network interfaced device and system for accelerated
TRANSCRIPT
(12) United States Patent Boucher et al.
US006427173B1
(10) Patent N0.: (45) Date of Patent:
US 6,427,173 B1 *Jul. 30, 2002
(54)
(75)
(73) ( * )
(21) (22)
(63)
(60)
(51) (52) (58)
(56)
4,366,538 A
INTELLIGENT NETWORK INTERFACED DEVICE AND SYSTEM FOR ACCELERATED COMMUNICATION
Inventors: Laurence B. Boucher, Saratoga; Stephen E. J. Blightman, San Jose; Peter K. Craft, San Francisco; David A. Higgen, Saratoga; Clive M. Philbrick, San Jose; Daryl D. Starr, Milpitas, all of CA (US)
Assignee: Alacritech, Inc., San Jose, CA (US)
Notice: Subject to any disclaimer, the term of this patent is extended or adjusted under 35 U.S.C. 154(b) by 0 days.
This patent is subject to a terminal dis claimer.
Appl. No.: 09/464,283 Filed: Dec. 15, 1999
Related US. Application Data
Continuation-in-part of application No. 09/439,603, ?led on Nov. 12, 1999, and a continuation-in-part of application No. 09/384,792, ?led on Aug. 27, 1999, and a continuation of application No. 09/067,544, ?led on Apr. 27, 1998, now Pat. No. 6,226,680 Provisional application No. 60/061,809, ?led on Oct. 14, 1997, and provisional application No. 60/098,296, ?led on Aug. 27, 1998.
Int. Cl.7 ...................... .. G06F 15/172; G06F 15/16
US. Cl. ...................... .. 709/238; 709/230; 709/250
Field of Search ............................... .. 709/230, 238,
709/236, 228, 232, 250, 225 References Cited
U.S. PATENT DOCUMENTS
12/1982 Johnson et al. ........... .. 264/200
(List continued on next page.)
FOREIGN PATENT DOCUMENTS
WO98/19412 5/1998
(List continued on next page.)
NETWDRK
@2103
SEQUENCERS ' 2m CONFM;
RXSEQ mid? 2105 2104
OTHER PUBLICATIONS
Internet pages entitled: DART Fast Application—Level Net Working Via Data—Copy Avoidance, by Robert J. Walsh, printed Jun. 3, 1999.
(List continued on next page.)
Primary Examiner—Zarni Maung (74) Attorney, Agent, or Firm—T. Lester Wallace; Mark Lauer
(57) ABSTRACT
An intelligent network interface card (INIC) or communi cation processing device (CPD) Works With a host computer for data communication. The device provides a fast-path that avoids protocol processing for most messages, greatly accel erating data transfer and offloading time-intensive process ing tasks from the host CPU. The host retains a fallback processing capability for messages that do not ?t fast-path criteria, With the device providing assistance such as vali dation even for sloW-path messages, and messages being selected for either fast-path or sloW-path processing. A context for a connection is de?ned that alloWs the device to move data, free of headers, directly to or from a destination or source in the host. The context can be passed back to the host for message processing by the host. The device contains specialized hardWare circuits that are much faster at their speci?c tasks than a general purpose CPU. A preferred embodiment includes a trio of pipelined processors devoted to transmit, receive and utility processing, providing full duplex communication for four Fast Ethernet nodes. The device provides a fast-path that avoids protocol processing for most messages, greatly accelerating data transfer and offloading time-intensive processing tasks from the host CPU. The host retains a fallback processing capability for messages that do not ?t fast-path criteria, With the device providing assistance such as validation even for sloW-path messages, and messages being selected for either fast-path or sloW-path processing. A context for a connection is de?ned that alloWs the device to move data, free of headers, directly to or from a destination or source in the host. The context can be passed back to the host for message process ing by the host.
20 Claims, 21 Drawing Sheets
PROCESSOR 470
HOSTZU \ ***** reefer“
DRAM
2223 STATUS
1 Asa
BUFFER 1 2114
US 6,427,173 B1 Page 2
US. PATENT DOCUMENTS WO PCT/US98/14729 1/1999 WO WO99/04343 1/1999
5,163,131 A 11/1992 ROW et al. ................ .. 395/200 W0 WO 99/65219 6/1999 5,212,778 A 5/1993 Dally et al. . 395/400 W0 W0 O1/O477O A2 7/2000 5,280,477 A 1/1994 Trapp 370/85.1 W0 WO 01/05107 A1 7/2000 5,289,580 A 2/1994 Latif et al. ...... .. 395/275 W0 W0 O1/O5116 A2 7/2000 5,303,344 A 4/1994 Yokoyama et al. ....... .. 395/200 W0 WO /01/05123 A1 7/2000 5,412,782 A 5/1995 Hausman et al. ......... .. 395/250 W0 WO 01/40960 A1 6/2001 5,448,566 A 9/1995 Richter et al. .... .. 370/94.1
5,485,579 A 1/1996 HitZ et al. .. .. 395/200.12 5,506,966 A 4/1996 Ban ......................... .. 395/250 OTHER PUBLICATIONS
5,511,169 A 4/1996 Suda ........................ .. 395/280
5,548,730 A 8/1996 Young et al. .. 395/280 Internet pages of InterProphet entitled: Frequently Asked 5,566,170 A 10/1996 Bakke et al. .. ....... .. 370/60 Questions, by Lynne Jolitz, printed Jun. 14, 1999. 5,588,121 A 12/1996 Reddin et a1. . .. 395/200.15 _ _ _ _
5,590,328 A 12/1996 Seno et a1_ _____ __ 395/675 Internet pages entitled: Technical White Paper—Xpolnt’s 5,592,622 A 1/1997 lsfeld et a1. __ 395/200,02 Disk—to—LAN Acceleration Solution for Windows NT 5,629,933 A 5/1997 Delp et al. ................ .. 370/411 Server, printed Jun. 5, 1997. 5,634,099 A 5/1997 Andrews et al. ..... .. 395/200.07 _ _ 5,634,127 A 5/1997 Cloud et a1_ _____ __ 395/680 Jato Technologies Internet pages entitled: Network Accel 5,642,482 A 6/1997 Pardillos _________ __ 395/2002 erator Chip Architecture, twelve—slide presentation, printed 5,664,114 A 9/1997 Krech, Jr. et al. .... .. 395/200.64 Aug. 19, 1998. 5,671,355 A 9/1997 Collins .................. .. 395/200.2 _ _ _ _
5,678,060 A 10/1997 yokoyama et a1_ _____ __ 709/212 EETIMES article entitled: Enterprise System Uses Flexible 5,692,130 A 11/1997 Shobu et a1. ........ .. 395/200.12 Spec, by Christopher Harrer and Pauline Shulman, dated 5,699,317 A 12/1997 Sartore et al. ....... .. 395/230.06 Aug. 10, 1998, Issue 1020, printed Nov. 25, 1998. 5,701,434 A 12/1997 Nakagawa ..... .. 395/484 _ _ _
5,701,516 A 12/1997 Cheng et a1_ __ 395/842 Internet pages entitled: lReady About Us and lReady Prod 5,749,095 A 5/1998 Hagersten .. 711/141 ucts, printed Nov' 25, 1998. 5751715 A 5 1998 Ch t l. . 370 455 . 5:752j078 A 41998 D551; :1_ ________________ __ 3952327 Internet ‘pages entitled: Srnart Ethernet Network Interface 5,758,084 A 5/1998 Silverstein et al. 395/200.58 Car, WhICh Berend OZCGH 1S developlng, Pnnted NOV- 25, 5,758,089 A 5/1998 Gentry et al. ........ .. 395/200.64 1998 5 758186 A 5 1998 H ~lt t l. ..... .. 395 831 . . 5’758’194 A 41998 Kirznrilaoflij _____________ __ 3952386 Internet pages entitled: Hardware Asslsted Protocol Process 5,771,349 A 6/1998 PicaZo, Jr. et a1. .... .. 395/188.01 ing, Which Eugene Feinberg is Working On, Printed NOV- 25, 5,790,804 A 8/1998 Osborne ......... .. .. 395/200.75 1998
5 794 061 A 8 1998 H t l .. 395 800.01 . . . . 5:802j580 A 41998 M25652; 7/11/149 Internet pages of XaQtl Corporation entitled: Glga POWER 5,809,328 A 9/1998 Nogales et a1_ _____ __ 395/825 Protocol Processor Product Preview, printed Nov. 25, 1998.
5,812,775 A 9/1998 Van Seters et al. 395/200.43 Internet a es of X Oint Technolo ies X Oint Com 5,815,464 A 9/1998 Purcell et al. ............ .. 395/163 b . psg p. d g ”' ”' "' p ' 5,878,228 A 3/1999 Bilansky et a1. ..... .. 395/200.57 We S1te( Pages) Pnnte Dec- 19,1997
5913928 A 6/1999 Wang et a1‘ """"" " 395/20033 Internet pages relating to iReady Corporation and the iReady 5,930,830 A 7/1999 Mendelson et al. ....... .. 711/171 I t tT M d 1 . t d N 2 1998 5,931,918 A 8/1999 Row et a1. ................ .. 709/300 n eme uner O u 6’ PH“ 6 0V‘ > '
2 gurayamatetlal ~~~~~~ Internet pages entitled: Asante and 100BASE—T Fast Eth , , onnery e a . ....... .. . -
5,941,969 A 8/1999 Rarn et a1. ................ .. 710/128 ernet’ pnnted May 27’ 1997'
5,941,972 A 8/1999 H9659 et a1~ ~~~~~~~~~~~~~~ ~~ 710/129 Internet pages entitled: A Guide to the Paragon XP/S—A7 5,950,203 A 9/1999 Stakuls et al' "" " 707/10 Supercornputer at Indiana University, printed Dec. 21, 1998. 5,991,599 A 11/1999 Radogna et al. .. 370/392 6,005,849 A 12/ 1999 Roach et al- - 370/276 Andrew S. Tanenbaurn, Computer Networks, Third Edition, 6,009,478 A 12/1999 Panner et al. .. ..... .. 710/5 ISBN ()_13_349945_6 (1996) 6,016,513 A 1/2000 Lowe ...... .. . 709/250
6,026,452 A 2/2000 Pitts .......................... .. 710/56 Richard Stevens, “TCP/IP Illustrated, vol. 1, The Protocols”, 6,034,963 A 3/2000 Minami et a1. ........... .. 370/401 pp. 325—326 (1994). 6,044,438 A 3/2000 Olnowich ....... .. 711/130 _ _
6,047,356 A 4/2000 Anderson et a1 711 /129 VT8501 Apollo MVP4 Docurnentatlon, VIA Technologies, 6,057,863 A 5/2000 Olarig ..... .. . 345/520 Inc., pp. i—iv, 1—11, cover and copyright page, revision 1.3 6,061,368 A 5/2000 HitZelberger 370/537 (Feb. 1, 2000). 6,065,096 A 5/2000 Day et a1. .... .. 711/114 _ _ _
6,141,705 A 10/2000 Anand et a1_ __ ____ __ 710/15 Internet pages entitled: Northrldge/Southrldge vs. Intel Hub
6,226,680 B1 * 5/2001 Boucher et al. 709/230 Architecture, 4 pages, printed Feb. 10, 2001. 6,246,683 B1 6/2001 Connery et a1. 370/392 _ _ _ _ _ _
6,247,060 B1 6/2001 Boucher et a1. .......... .. 709/238 Glgablt Ethernet Technlcal Bnef, Achlevlng End—I0—EI1d 6,334,153 B2 * 12/2001 Boucher et a1. .......... .. 709/230 Performance~ Alteon Networks, Inc, First Edition, 5911
FOREIGN PATENT DOCUMENTS
WO 00/13091 WO98/50852
11/1998 11/1998
1996.
Internet pages directed to; Technical Brief on Alteon Eth ernet Gigabit NIC technology, www.alteon.corn, 14 pages, printed Mar. 15, 1997.
US 6,427,173 B1 Page 3
Internet web pages from “Adaptec.corn” website directed to the Adaptec, AEA—7110C iSCSI Host Bus Adapter and about Adaptec’s IP storage activities, 11 pages, downloaded and printed Oct. 1, 2001. Internet web pages from “iSCSIhba.corn” website directed to JNI iSCSI HBAs including the “FCE—3210/6410”, 10 pages, downloaded and printed Oct. 1, 2001. Internet web pages from the “ISCSI Storage.corn” website that mention an ErnuleX HBA, 2 pages, downloaded and printed Oct. 1, 2001. Internet web pages from the “iSCSIhba.corn” website that mention QLogic HBAs including the “SANblade 2300 Series”, 8 pages, downloaded and printed Oct. 1, 2001. “Two Way TCP Traf?c over Rate Controlled Channels: Effects and Analysis”, IEEE Transactions on Networking, vol. 6, No. 6, pp. 729—743 (Dec. 1998). Internet pages from IReady News Archives entitled “iReady Rounding Out management team with two key executives,” 2 pages, (printed Nov. 28, 1998). Toshiba, “Toshiba Delivers First Chips to Make Consurner Devices Internet—Ready Based on iReady’s Design”, 3 pages, Press release Oct. 14, 1998, downloaded Nov. 28, 1998. Internet pages entitled “iReady Products” from website http://www.ireadyco.corn/products.htrnl, 2 pages, printed Nov. 25, 1998. Iready News Archives. Toshiba, iReady shipping Internet Chip, 1 page printed Nov. 28, 1998. Internet site www.interprophet.corn, 17 pages, printed Mar. 1, 2000. The I—1000 Internet Tuner Features, iReady Corporation, 2 pages, date unknown. Internet pages from website http://www.ireadyco.corn/ about.htrnl, 3 pages, downloaded Nov. 25, 1998. IReady News Archives, “Revolutionary Approach to Con surner Electronics Internet Connectivity Funded”, San Jose, CA. 2 pages, Nov. 20, 1997, downloaded and printed Nov. 2, 1998.
IReady News Archives, “Seiko Instruments Inc. (SII) Intro duces World’s First Internet—Ready Intelligent LCD Mod ules Based on IReady Technology”, 2 pages Oct. 26, 1998, downloaded and printed Nov. 2, 1998. NEWSwatch—IReady Internet Tuner to Web Enable Devices, Nov. 5, 1996. 2 pages, printed Nov. 2, 1998. David Larnrners, EETirnes, Jun. 13, 1997, “Tuner for Toshiba, Toshiba taps iReady for Internet tuner,” 2 pages, printed Nov. 2, 1998. Internet pages entitled: Comparison of Novell Netware and TCP/IP Protocol Architectures, by Janique S. Carbone, 19 pages, Jul. 16, 1995, downloaded and printed Apr. 10, 1998. 60/053,240 (US. Provisional Application), by JolitZ et al. (listed ?ling date Jul. 18, 1997). Zilog Product Brief entitled “Z85C30 CMOS SCC Serial Cornrnunication Controller”, Zilog Inc., 3 pages (1997). US. application No. 08/964,304, by Napolitano, et al., entitled “File Array Storage Architecture”, ?led Nov. 4, 1997.
Article by D. HitZ, et al., “File System Design For An NFS File Server Appliance”, 13 pages, 1996. Adaptec Press Release, “Adaptec Announces EtherStorage Technology”, 2 pages, May 4, 2000, printed Jun. 14, 2000. Adaptec article, “EtherStorage Frequently Asked Ques tions,” 5 pages, printed Jul. 19, 2000. Adaptec article, “EtherStorage White Paper,” 7 pages, printed Jul. 19, 2000. CIBC World Markets article by J. Berlino et al., “Cornput ers; Storage”, 9 pages, dated Aug. 7, 2000. Merrill Lynch article by S. Milunovich, “Storage Futures”, 22 pages, dated May 10, 2000. Internet—draft of J. Satran, et al., “SCSI/TCP (SCSI over TCP)”, 38 pages, dated Feb. 2000. Article by S. Taylor, “Montreal Start—Up Battles Data Stor age Botttleneck,” 2 pages, dated Mar. 5, 2000.
* cited by eXarniner
U.S. Patent Jul. 30, 2002 Sheet 1 0f 21 US 6,427,173 B1
CPU
STORAGE
REMOTE HOST
FIG. 1
5 3
@K 2) MA 0 w
8 \
6 4
4 4
K REM Kvm RYc KN mAAmom WWMMSWA UmEmNTT K rm// w 4 8 %
M 3
A
K 50
CONTEXT
INIC/ CPD
52
FIG. 2
U.S. Patent Jul. 30, 2002 Sheet 2 0f 21 US 6,427,173 B1
RECEIVE PACKET FROM NETWORK \_/\47
BY CPD
V
VALIDATE PACKET, SUMMARIZE \_/\57 HEADERS
61
FAST PATH SEND PACKET TO CANDIDATE” STACK FOR SLOW
' PATH PROCESSING
65 w
SEND PACKET TO MATCH WITH STACK FOR SLOW
CCB? PATH PROCESSING
YES 69 K v
SEND TO DESTINATION CREATE CCB FOR IN HOST VIA MESSAGE FAST-PATH
1 51
FIG. 3
U.S. Patent Jul. 30, 2002 Sheet 4 0f 21 US 6,427,173 B1
REMOTE HOST
1 52" \\l _ - - - _ _ _ _ _ _ _ _ _ _ _ 1
FA T-P ' S ATH l > SOURCE/DEST I 159 168 JV " i
' APPLICATION ' K 166 "T. I
\ | |
150 \L170\ 1% 164% TRANSPORT : |_ W _ _ _ _ _ _ _ _ _ _ _ _ _ ' ' _| I |
' I 162 I l | \K : PROCESSOR I l NETWORK I I S 1 160 l 1 { HARDWARE LOGIC } SLOW-PATH I DATA LINK : ' | ' \ I
U.S. Patent Sheet 5 0f 21 Jul. 30, 2002
MEDIA ACCESS
US 6,427,173 B1
172 CONTROLLER N
ASSEMBLY N 174 178 176 REGISTER S S
PACKET —~—> FLY BY
SEQUENCER ' CONTROL v { SEQUENCER
MULTIPLEXOR N 180 +
3182 S183 SRAM
SRAM ' CONTROL ‘
A
v
DRAM CONTROL ¢
& 186 #
DRAM 188 QUEUE N 184'“ MANAGER
FIG. 7
U.S. Patent
178
Jul. 30, 2002 Sheet 6 0f 21
PACKET
SEQUENCER
MAC N191
SEQUENCER
NETWORK N 192
SEQUENCER
TRANSPORT N 194
SEQUENCER
SESSION N 195
SEQUENCER
180
S
US 6,427,173 B1
174
8 CONTROL ———>
ASSEMBLY REGISTER
MULTIPLEXOR
FIG. 8
U.S. Patent Jul. 30, 2002 Sheet 8 0f 21 US 6,427,173 B1
202»\_2l I l 1 300 318 i I \y /\/ UPPER LAYER <—{—-—— ' | ' | l
UPPER LAYER INTERFACE I 330% I ' |
325% TRANSPORT TRANSPORT my 316 ' | l
322 N‘, NETWORK NETWORK wig 314 ' l |
320/\:/ DATA LINK DATA LINK ’\L/ 312
306 ’\/ INIC MINIPORT DRIVER
I 240 ’\/
4T INIC 4 210 I
INIC MEMORY
FIG. 10
U.S. Patent Jul. 30, 2002 Sheet 10 0f 21 US 6,427,173 B1
_
_ 6
_ D. 1U &D.
C TV A C
U“ M mR 4 _ 2 _ 8 4 n 6 m 2%
n w 2
_ J j m“ m w _ C 4 &C4 _ C T . UAYA‘LVMW “ M X R
_ m
\J M m _ 4
m _J 2 J _ B M &
B
_ O A V! T _ Au.‘ A MW _ M XR
210 400\\ 240 ____________L__
M AC-A
1/410 XMT& 418 RCV-A
EXTERNAL MEMORY CTRL
|||||J
/480
I 47 REG FILE WCS ROM
PCI BUS INTERFACE UNIT ||||||||||||||||||||||||||||||||||||||||||||||||||||||||L FIG. 13
U.S. Patent Jul. 30, 2002 Sheet 11 0f 21 US 6,427,173 B1
CLOCK /
I f f 1 l l l
: 49L CONTROLS FOR FIRST REGISTER SET
/ /| 500 ’\ 490
1
: V \ V V V V
1%» FIRST REGISTER SET I RAM FILE L REGISTER / >- — —————————————————————————————— — — — — — — — —
l 498 I x v v v v v } INSTRUCTION DECODER } AND
,/ OPERAND MULTIPLEXER 560\ 496
l V X V V V V V |
H—:> SECOND REGISTER SET I l \ \ L_ ____________________________________________ __
// E V V V V , STACK : EXCHANGE ARITHMETIC LOGIC UNIT QUEUE
' K J J 608 60% 602 503
\l v v v v v 0
TI» THIRD REGISTER SET |
'\ 501 / 470 L‘; ____ “<1 ____ “Kn-“bx ____ “Y ____ ""
U.S. Patent
U2 .GE
US 6,427,173 B1 n. M m
32m
O3 wow 0% 26 N8
m % \ \ \ \ a \ \ x
m. :55
, div @EEQVE v52
0 / - IV V
m 9:5 M25 % 55 a?
m 236
|||||||||||||||||||||| Ell lilll I} {I ll -llll ,
/
/
U.S. Patent Jul. 30, 2002 Sheet 15 0f 21 US 6,427,173 B1
FIG. 16
745
J 768 7% /
772 7 /
764
750
755
K790 / 792
/ 796 7
/ 788
760
FIG. 17
U.S. Patent Jul. 30, 2002 Sheet 16 0f 21 US 6,427,173 B1
Proc DZQ QZD XMT RCV 806 802 R Seq Seq Seq Seq 844 Wrlte
eq Req Req Req Req Data
808 8 10 804 /815 846
ARBITER \ MUX
v K REGISTER A
I v v Addr DIn Addr 825 Out In
QRAM DOut
820 V V
REGISTER B
828 v '
QALU
822\ ' 821
r___|____________._ _~__________J_1 1 v v v v | l v v 1! Ir I v
| Q Q Q Q l I Sram Sram Bofiy Body I srafn Empty Fun In Out I | Req Addr Wnte Read Wnte
: RD Y I | Req Req | Data
FIG. 18
U.S. Patent Jul. 30, 2002 Sheet 17 0f 21 US 6,427,173 B1
MRU
6
MRU
R13 R14 R15 13 4
R13 R14 R15
R8 R9 12 10
R8 R9 12
R7 2
FIG. 19A
R7
R2 7
R2
' FIG. 19B
R1 1
R1 0
LRU
R0 9
LRU
R 0 0\
O 0
9
MRU
R13 R14 R15 6
LRU
1O 9 R8 R7 R2 R1 0
FIG. 19C
MRU
R13 R14 R15 10 12
R7 R2 R1
LRU 0 0
FIG. 19D