Intelligent Proxy Serverin TANET/TWAREN

National Chi Nan University

Ya‐Ling Wang,

Ying‐Chieh Chang, 

Quincy Wu

Chinese paper published in ITIA 2010.

Introduction

• The choice of web proxy servers• The best proxy server?• Priori speed tests• The fastest proxy server• Experimental result : Better

2010/07/10 NCNU YaLing, Wang 2

Default Proxy Server

2010/07/10 3NCNU YaLing, Wang

TANETProxy Server

NCNUProxy Server

TWAREN 線路

TANET 線路

TWARENProxy Server

USER

.edu .org .gov .jp .cn .hk .vn

.th .kr .in .my .ph .sg .ca .nz

TANETProxy Server

TWARENProxy Server

Foreign sitesTANET

TWARENTWAREN

TANET

NCNUProxy Server

Intelligent Proxy Server

2010/07/10 NCNU YaLing, Wang 4

TANETProxy Server

TWARENProxy Server

NCNUProxy Server

USER

System Components 

• CentOS 5.4 – Linux operating system• Squid version 3.1 – http proxy server• Report generator

– Webalizer– Sarg

• Upstream proxy server– TWAREN : proxy1.twaren.net– TANET : proxy.moe.edu.tw

2010/07/10 5NCNU YaLing, Wang

Data Collection Process 

2010/07/10 NCNU YaLing, Wang 6

Start

For each URL in the access log

Extract the hostname part

Perform the speed test

Depending on which proxy server provides the fastest service, save the hostname in 

1.   list.twaren2.   list.tanet 3.   list.ncnu

The squid access.log contains recently used URLs

Each site will be stored in the fastest proxy server list. There are three proxy servers, so at the end we shall have three lists.

root crontab：Perform the test at 4AM in every morning

Download a file from the host

Data Collection Process 

2010/07/10 NCNU YaLing, Wang 7

Remove the hostname from other lists if it is in  those lists

Delete this host from other lists

YES

Merge squid.conf file

End

You can do reload later.

Get a site from the list of a proxy server, the new rules will be the fastest, to merge with the original profile.

NO

Data Collection Process 

2010/07/10 NCNU YaLing, Wang 8

Today

NCNU proxy server list

www.google.com

TWAREN proxy server list

TANET proxy server list

Yesterday

www.google.comwww.google.com

www.google.com

Data Collection Process 

2010/07/10 NCNU YaLing, Wang 9

A

B

Data Collection Process 

2010/07/10 NCNU YaLing, Wang 10

A

TANET proxy server list

TWAREN proxy server list

C

Squid ConfigBNCNU proxy server list

Experimental Results 

2010/07/10 11NCNU YaLing, Wang

NCNU school inflows rankings

Experimental Results 

2010/07/10 12NCNU YaLing, Wang

Date：2010/05/073191

Experimental Results 

2010/07/10 NCNU YaLing, Wang 13

Date：2010/05/11157

Conclusion

• More User• More Webpages• More Accurate• More Upstream Proxy Server→ Automatic, Learning, Intelligent Proxy Server

2010/07/10 14NCNU YaLing, Wang

Future Work

• Re‐use the perfSONAR system on Internet2 for bandwidth measurement. 

• Re‐use network routing table to choose the “shortest” path.

2010/07/10 15NCNU YaLing, Wang

Any Question?

ENDThanks for your listening

2010/07/10 16NCNU YaLing, Wang

Measurement Tools 

• http_ping –proxy ServerName:Port URL

2010/07/10 17NCNU YaLing, Wang

Measurement Tools

• Useful options– http_ping –proxy ServerName:Port– http_ping –count Number of retries

• Commands in our experiments

2010/07/10 18NCNU YaLing, Wang

Intelligent Proxy Server�in TANET/TWARENIntroduction Default Proxy ServerIntelligent Proxy ServerSystem Components Data Collection Process Data Collection Process Data Collection Process Data Collection Process Data Collection Process Experimental Results Experimental Results Experimental Results ConclusionFuture WorkSlide Number 16Measurement Tools Measurement Tools