AntNet: Distributed Stigmetric Control for Communications Networks

Gianni Di Caro & Marco Dorigo

Journal of Artificial Intelligence Research 1998

Presentation by

Tavaris Thomas

Presentation Contents

Introduction/Background Model Description AntNet: An Adaptive Agent-based Routing

Algorithm Other Routing Algorithms Experimental Networks Used Results Conclusions and Future Work

Introduction/Background

Increase in the supply and demand of network communication services

Network Control – online and off-line monitoring and management of the network resources

Routing – process or method of determining and prescribing incoming packets to an outgoing path (forwarding messages)

Swarm Intelligence (SI)

New research field Collective behavior of social insects and

other organisms ants, honey bees – states/actions

Stimergy – Complex and intelligent behavior performed through the interaction of thousands of autonomous swarm members

Ant Colony Optimization(ACO)

Foraging behavior of ants and is used successfully to solve combinatorial optimization problems. traveling salesman genome matching routing in telecommunications networks load balancing

Model Description

WAN Irregular topology connection-less network Network communication is mapped on a

directed weighted graph with N processing/forwarding nodes

Links characterized by bandwidth (bit/sec) and transmission delay (sec)

2 types of packets (routing and data) routing have greater priority

C++ based discrete event driven simulator

AntNet

Adaptive, distributed, and mobile agent-based routing algorithm

Reinforcement learning problems with hidden state (Bertsekas & Tsitsiklis, 1996; Kaelbling, Littman, & Moore, 1996; McCallum, 1995).

AntNet Algorithm Overview

Mobile agents are asynchronously launched towards randomly selected destination nodes.

Each agent searches for a minimum cost path joining its source and destination nodes.

Each agent moves step-by-step towards its destination node. At each intermediate node a greedy stochastic policy is applied to choose the next node to move to. The policy makes use of (i) local agent-generated and maintained information, (ii) local problem-dependent heuristic information, and (iii) agent-private information.

While moving, the agents collect information about the time length, the congestion status and the node identifiers of the followed path.

AntNet Algorithm Overview

Once they have arrived at the destination, the agents go back to their source nodes by moving along the same path as before but in the opposite direction.

During this backward travel, local models of the network status and the local routing table of each visited node are modified by the agents as a function of the path they followed and of its goodness.

Once they have returned to their source node, the agents die.

Routing Table Contents

NdPNkn

nd ,1,1

)(kneighborsN k

Goodness (desirability)ndP

kT Routing table

dddk WM ,, 2

Array of ds defining parametric statistical model for the traffic distribution over the network as seen by local node k

Mean, variance, and best

AntNet Algorithm

The heuristic correction ln is a [0,1] normalized value proportional to the length qn (in bits waiting to be sent) of the queue of the link connecting the node k with its neighbor n:

The value of alpha weights the importance of the heuristic correction with respect to the probability values stored in the routing table. Agent's decisions are taken on the basis of a combination of a long-term learning process and an instantaneous heuristic prediction.

Ideal alpha between 0.2 and 0.5

AntNet Algorithm

The backward ant updates the routing table and arrays stored at each node as it propagates through network.

''' 1 fdfdfd PrPP

fnNnrPPP kndndnd ,,'''

kMTrr ,

1,0r

Positive reinforcement

Negative reinforcement

Reinforcement to be a function of the goodness where

Other Routing Algorithms Compared

OSPF (static, link state)Open Shortest Path First

SPF (adaptive, link-state) Shortest Path First BF (adaptive, distance-vector) Bellman Ford Q-R (adaptive, distance-vector): Q-Routing PQ-R (adaptive, distance-vector): is the

Predictive Q-Routing algorithm Daemon (adaptive, optimal routing): is an

approximation of an ideal algorithm

Networks Used

SimpleNet (1.9, 0.7, 8)

10Mbit/s and propagation delay of 1msec

mean shortest path distance,in terms of hops, between all pairs of nodes, the variance Of this average, and the total number of nodes

Networks Used

NFSNET(2.2,0.8,14)

1.5Mbps propagation delays4-20 msec

Networks Used

NTTnet(6.5,3.8,57) 6Mbps propagation

delay 1 to 5

msec

Metrics for Performance Evaluation

Throughput Delay Distribution- the authors used whole

empirical distribution or to use the 90th percentile statistic, which allows one to compare the algorithms on the basis of the upper value of delay they were able to keep the 90% of the correctly delivered packets

Network Capacity Usage (as expressed by the as the sum of the link capacities divided total available link capacity)

SimpleNet Throughput Results

SimpleNet: Comparison of algorithms for F-CBR traffic directed from node 1 to node 6)

The delay distribution showed similar results

*note AntNet outperformed

NFSNET Delay Results

Comparison of algorithms for increasing load for UP traffic. The load is increased reducing the MSIA (mean inter arrival time) value from 2.4 to 2 seconds

** note that throughput results were similar amongst all algorithms but SPF and BF were the best

NTTnet Delay Results

NTTnet: Comparison of algorithms for increasing load for UP-HS traffic. The load is increased reducing the MSIA value from 4.1 to 3.7 seconds.

** note that throughput results were similar amongst all algorithms but SPF and BF were the best

Routing Overhead

Routing Overhead: ratio between the bandwidth occupied by the routing packets and the total available network bandwidth. All data are scaled by a factor of 10^-3

Conclusions and Future Work

AntNet showed superior performance and robustness to internal parameter settings for almost all the experiments.

AntNet's most innovative aspect is the use of stigmetric communication to coordinate the actions of a set of agents that cooperate to build adaptive routing tables.

Future Work

To add flow and error control to the algorithm Change the priority of ants as the propagate

through the system Greater study of the negative reinforcement of

connection Greater survivability in the presence of faults

(disaster situations)