![Page 1: Scalable Kernel Performance for Internet Servers under Realistic Loads. Gaurav Banga, etc... Western Research Lab : Research Report 1998/06 (Proceedings](https://reader030.vdocuments.site/reader030/viewer/2022032415/56649efe5503460f94c135b3/html5/thumbnails/1.jpg)
Scalable Kernel Performance for Internet Servers under Realistic Loads.
Gaurav Banga, etc...
Western Research Lab : Research Report 1998/06(Proceedings of the 1998 USENIX Annual Technical Conference)
Computer Architecture Lab. CS Dept. KAIST
2000/11/
Kim, Sung-Wan
![Page 2: Scalable Kernel Performance for Internet Servers under Realistic Loads. Gaurav Banga, etc... Western Research Lab : Research Report 1998/06 (Proceedings](https://reader030.vdocuments.site/reader030/viewer/2022032415/56649efe5503460f94c135b3/html5/thumbnails/2.jpg)
2/16
Contents
• Introduction
• Problems of select() & ufalloc() in event-driven servers
• Scalable select() & ufalloc()
• Experimental evaluation
• Performance of a live system
• Conclusions
![Page 3: Scalable Kernel Performance for Internet Servers under Realistic Loads. Gaurav Banga, etc... Western Research Lab : Research Report 1998/06 (Proceedings](https://reader030.vdocuments.site/reader030/viewer/2022032415/56649efe5503460f94c135b3/html5/thumbnails/3.jpg)
3/16
Introduction
• Event-driven servers– A single thread
• manage all connections– Lower context-switching & synchronization overhead
• faster than a thread-per-connection or pre-forked system– But, perform poorly under real conditions
• select() & ufalloc()
• select()– Asynchronous I/O
• ufalloc()– Allocation of a new file descriptor for a process
![Page 4: Scalable Kernel Performance for Internet Servers under Realistic Loads. Gaurav Banga, etc... Western Research Lab : Research Report 1998/06 (Proceedings](https://reader030.vdocuments.site/reader030/viewer/2022032415/56649efe5503460f94c135b3/html5/thumbnails/4.jpg)
4/16
Problems in select() & ufalloc()
• WAN environments– Larger round-trip time and packet losses than LAN environments
– Many open connections
• select()– select() -> do_scan() -> selscan() -> soo_select()
– select_wakeup() -> do_scan() -> selscan() -> soo_select()
– soo_select()
• check to see if the condition is true
• Linear search for all opened socket
• ufalloc()– Single bitmap (first lower descriptor number)
– Too cost
![Page 5: Scalable Kernel Performance for Internet Servers under Realistic Loads. Gaurav Banga, etc... Western Research Lab : Research Report 1998/06 (Proceedings](https://reader030.vdocuments.site/reader030/viewer/2022032415/56649efe5503460f94c135b3/html5/thumbnails/5.jpg)
5/16
Environment
• Server– AlphaStation 500(400Mhz), 192 MB of main memory
– Digital UNIX 4.0B
– Squid 1.1.11, NetCache 3.1.2c-OSF
• Client– AlphaStation 500(333Mhz)
– Digital UNIX 3.2C
– S-Client
• Network– 100Mbps FDDI
• Profiling– DCPI
![Page 6: Scalable Kernel Performance for Internet Servers under Realistic Loads. Gaurav Banga, etc... Western Research Lab : Research Report 1998/06 (Proceedings](https://reader030.vdocuments.site/reader030/viewer/2022032415/56649efe5503460f94c135b3/html5/thumbnails/6.jpg)
6/16
• CPU times in unmodified kernel
![Page 7: Scalable Kernel Performance for Internet Servers under Realistic Loads. Gaurav Banga, etc... Western Research Lab : Research Report 1998/06 (Proceedings](https://reader030.vdocuments.site/reader030/viewer/2022032415/56649efe5503460f94c135b3/html5/thumbnails/7.jpg)
7/16
Scalable select() & ufalloc()
• select()– READY, INTERESTED, HINTS set
– sowakeup()
• Records a hint in the HINTS sets of each of the threads in the referencing processes for which this socket is present in the INTERESTED set of the thread.
• ufalloc()– 2-level bitmap 1 0
1 1 1 1 0 0 1 1
Level 0 map
Level 1 map
INTERESTEDnew = SELECTING U INTERESTEDold
READYnew = C (INTERESTEDnew ^ (!INTERESTEDold U READYold U HINTS))READYto_user = SELECTING ^ READYnew
![Page 8: Scalable Kernel Performance for Internet Servers under Realistic Loads. Gaurav Banga, etc... Western Research Lab : Research Report 1998/06 (Proceedings](https://reader030.vdocuments.site/reader030/viewer/2022032415/56649efe5503460f94c135b3/html5/thumbnails/8.jpg)
8/16
Experimental Evaluation- Scalability with respect to connection rate
* 750 infinitely slow connections
![Page 9: Scalable Kernel Performance for Internet Servers under Realistic Loads. Gaurav Banga, etc... Western Research Lab : Research Report 1998/06 (Proceedings](https://reader030.vdocuments.site/reader030/viewer/2022032415/56649efe5503460f94c135b3/html5/thumbnails/9.jpg)
9/16
Experimental Evaluation- Scalability with respect to connection rate
![Page 10: Scalable Kernel Performance for Internet Servers under Realistic Loads. Gaurav Banga, etc... Western Research Lab : Research Report 1998/06 (Proceedings](https://reader030.vdocuments.site/reader030/viewer/2022032415/56649efe5503460f94c135b3/html5/thumbnails/10.jpg)
10/16
Experimental Evaluation- Scalability with respect to connection count
![Page 11: Scalable Kernel Performance for Internet Servers under Realistic Loads. Gaurav Banga, etc... Western Research Lab : Research Report 1998/06 (Proceedings](https://reader030.vdocuments.site/reader030/viewer/2022032415/56649efe5503460f94c135b3/html5/thumbnails/11.jpg)
11/16
Performance of a live system
• Server– A Web proxy system at DEC
– AlphaStation 500 (500 MHz), 512 MB of RAM
– Running the system for an entire day
– Proxy
• Squid
• NetCache
![Page 12: Scalable Kernel Performance for Internet Servers under Realistic Loads. Gaurav Banga, etc... Western Research Lab : Research Report 1998/06 (Proceedings](https://reader030.vdocuments.site/reader030/viewer/2022032415/56649efe5503460f94c135b3/html5/thumbnails/12.jpg)
12/16
Performance of a live system- NetCache with caching disabled
![Page 13: Scalable Kernel Performance for Internet Servers under Realistic Loads. Gaurav Banga, etc... Western Research Lab : Research Report 1998/06 (Proceedings](https://reader030.vdocuments.site/reader030/viewer/2022032415/56649efe5503460f94c135b3/html5/thumbnails/13.jpg)
13/16
Performance of a live system- NetCache with caching disabled
![Page 14: Scalable Kernel Performance for Internet Servers under Realistic Loads. Gaurav Banga, etc... Western Research Lab : Research Report 1998/06 (Proceedings](https://reader030.vdocuments.site/reader030/viewer/2022032415/56649efe5503460f94c135b3/html5/thumbnails/14.jpg)
14/16
Performance of a live system- NetCache with caching enabled
![Page 15: Scalable Kernel Performance for Internet Servers under Realistic Loads. Gaurav Banga, etc... Western Research Lab : Research Report 1998/06 (Proceedings](https://reader030.vdocuments.site/reader030/viewer/2022032415/56649efe5503460f94c135b3/html5/thumbnails/15.jpg)
15/16
Performance of a live system- NetCache with caching enabled
![Page 16: Scalable Kernel Performance for Internet Servers under Realistic Loads. Gaurav Banga, etc... Western Research Lab : Research Report 1998/06 (Proceedings](https://reader030.vdocuments.site/reader030/viewer/2022032415/56649efe5503460f94c135b3/html5/thumbnails/16.jpg)
16/16
Performance of a live system- Squid with caching disabled
![Page 17: Scalable Kernel Performance for Internet Servers under Realistic Loads. Gaurav Banga, etc... Western Research Lab : Research Report 1998/06 (Proceedings](https://reader030.vdocuments.site/reader030/viewer/2022032415/56649efe5503460f94c135b3/html5/thumbnails/17.jpg)
17/16
Performance of a live system- Squid with caching disabled
![Page 18: Scalable Kernel Performance for Internet Servers under Realistic Loads. Gaurav Banga, etc... Western Research Lab : Research Report 1998/06 (Proceedings](https://reader030.vdocuments.site/reader030/viewer/2022032415/56649efe5503460f94c135b3/html5/thumbnails/18.jpg)
18/16
Performance of a live system- Squid with caching disabled
![Page 19: Scalable Kernel Performance for Internet Servers under Realistic Loads. Gaurav Banga, etc... Western Research Lab : Research Report 1998/06 (Proceedings](https://reader030.vdocuments.site/reader030/viewer/2022032415/56649efe5503460f94c135b3/html5/thumbnails/19.jpg)
19/16
Conclusions
• WAN delays
• Linear scaling in the select() & ufalloc()– lead to excessive kernel CPU computation
• Scalable versions– improve the performance of Web servers and proxies
![Page 20: Scalable Kernel Performance for Internet Servers under Realistic Loads. Gaurav Banga, etc... Western Research Lab : Research Report 1998/06 (Proceedings](https://reader030.vdocuments.site/reader030/viewer/2022032415/56649efe5503460f94c135b3/html5/thumbnails/20.jpg)
20/16
select(maxfd, &readfds, &writefds, …, …);
1008 for (i = 0; i < maxfd; i++) { 1009 /* Check each open socket for a handler. */ 1010 if (fd_table[i].read_handler) { 1011 if (fd_table[i].stall_until <= squid_curtime) { 1012 nfds++; 1013 FD_SET( i, &readfds); 1014 } 1015 } 1016 if (fd_table[i].write_handler) { 1017 nfds++; 1018 FD_SET(i, &writefds); 1019 } 1020 }