Almost No-CostWeb Traffic Management

Mark Bizzell & Matthew MengelUniversity of Southern Queensland

1996 - The Problems

• Enormous growth in Internet traffic charges– growing academic requirements for access

• but have you checked your proxy logs lately?

– increasing bandwidth capacity• local, national, international

– more and more students “discovered” the net

– ease of use of tools (browsers, etc)

1996 - The Problems

0

10

20

30

40

Gig

abyt

es

1996 - The Problems

• Limited Access Points– computer laboratories

• lab seats taken up by Internet users• required for booked sessions• required for other academic tasks

– assignment preparation– programming

– modems• limited number, always jam packed

The Task

• Find a way to manage Internet (i.e. Web) traffic before it became a financial problem

• Also manage the limited laboratory and modem resources

• By the way, your budget is…

$ 0(or thereabouts)

The Concept - Traffic

• Student Quota System– daily “allowance”

• accumulation capped at 10 days worth• allocated once daily

– once over quota, no more access• quota excess capped at 7 days over

– simple and effective• no money involved• can be extended to staff

– fair?

The Concept - Access

• Lab Booking System– 3 modes of access

• bookable open Internet access• quota Internet access• no Internet access

– addresses issues of balancing Internet and other activities

– modems• too hard (for now)

Lab Booking System

Log Processing

Authentication

Proxy Servers

The Solution

• MS Access database

• Predefined list of Labs

• Web access

• Nightly Update of booking information

- Today + 6 days

Lab Booking System

Proxy System

• Squid Proxy Servers

• Reconfigure daily with lab bookings ACL’s

• Allow user to authenticate

• Cache Authenticated access for 5 minutes

• Rotate Log files every 10 minutes

Authentication

• Only required from the labs and modems

• LDAP Server

- Authenticates Staff and Students

• NT Domain

- Alternate Staff authentication

• “No proxy” group membership

Log Processing

• Logs files collected every 10 minutes

• Processed by the quota system

– single perl script

– < 400 lines

• Update “No Proxy” LDAP group

– LDAP ldif file

Information Out

• Students can query their remaining quota

• Students can see their traffic download volumes for the month

• Some staff query tools

• All perl-based cgi scripts

Tools

• Squid proxy server

• Netscape LDAP server

• ssh – Secure shell

• perl

Issues

• No perfect system

- No online quota updates

- Two hour modem limit for students

- no “interactive” modem access restriction

- Updates only done once a day

So did it work?

• June 1996 40 Gb• June 2000 180 Gb

• Costs are acceptably under control– as defined by management– more use– more bandwidth– increase in quota from 1 Mb/day to 3

Mb/day

So did it work?

• Lab use acceptably moderated

• Modem congestion still a problem

• So, broadly speaking,

Yes.

Future direction

• Rewrite the lab booking system

• Integration with Active Directory

• Quota system “database”

– replace current flat file structure

Acknowledgments

• Peter Dobson– ex-ITS, USQ; now DEC, USQ– original concept and lab booking system

• Scott Sorley– ex-ITS, USQ; now NextEd– original ACLs when using Netscape proxy

• Dennis Rochford– ex-ITS, USQ; now NextEd– original lab booking system

Acknowledgments

• Mark Bizzell– ITS, USQ– Squid proxy implementation, use of ssh

• Matthew Mengel– ITS, USQ– original quota methodology, wrote quota system

Questions?

bizzell@usq.edu.auMatthew.Mengel@usq.edu.au

Quota system perl is available on request.– Use at your own risk, and let us know if modifications

you make are useful.