transport fleet sizing by using make and buy decision - doiserbia
TRANSCRIPT
The University of Birmingham
Nature Inspired Techniques in Traffic
Management & Route Optimisation By Thomas Barker, Andrew Matthews & Nathan Collins
07/03/2011
Abstract: Traditional research into the problems associated with traffic management
have focussed on complex scientific models, such as fluid dynamics, to model and find
optimal traffic solutions. This report instead draws inspiration from nature to highlight
systems that can be computationally cheaper and manage traffic more efficiently than
conventional techniques. All three natural techniques presented; cellular automata,
evolutionary algorithms and ant algorithms, put forward an alternative to prevalent
centralised traffic control systems that are slow to react to or are blind to the current
traffic conditions.
Table of Contents
Nature Inspired Techniques in Traffic Management & Route Optimisation .........................................1
1 Introduction ....................................................................................................................................3
2 Cellular automata in traffic modelling and control.........................................................................4
2.1 Cellular automaton introduction ............................................................................................4
2.2 Traffic models using cellular automaton ................................................................................4
2.3 Traffic control using cellular automata...................................................................................6
2.3.1 Current traffic control methods......................................................................................6
2.3.2 Self organising traffic lights (SOTL) .................................................................................7
2.4 Cellular Automata in traffic summary.....................................................................................9
3 Introduction to Evolutionary Algorithms......................................................................................11
3.1 Salt Routing Optimisation .....................................................................................................11
3.2 Adaptive Traffic Control........................................................................................................12
3.3 Evolutionary Computation Applied to Urban Traffic Optimisation ......................................13
3.4 Decentralised Car Traffic Control Using Message Propagation............................................14
4 Ant Colony Optimisation...............................................................................................................17
4.1 Vehicle Routing Problems.....................................................................................................18
4.1.1 Representation and Evaluation.....................................................................................18
4.2 Time-dependant Travel Times ..............................................................................................19
4.3 Dynamic Traffic Routing........................................................................................................21
5 Conclusion.....................................................................................................................................24
6 References ....................................................................................................................................25
1 Introduction
Traffic is a nuisance that wastes time and costs money. It causes additional stress for the people
involved and often harms social relations. On top of this emissions are equivalent to some industrial
sectors, contributing to greenhouse gases, and the location of emissions are predominantly in
densely populated areas.
It is difficult to calculate estimates of the economical costs of traffic, but as they can take hours off
an employee’s potential working hours per week, burn vast quantities of fuel unnecessarily, and
cause delays to business deliveries, the costs are assumed to be large. In Copenhagen, it has been
estimated that the economic loss due to vehicle delays is 750 million Euros per year, and for the
whole of Germany the damage has been estimated as 100 billion dollars per year [Szklarski 2010].
But with an exploding global population, of which an unprecedented 50 % now live in urban areas,
the challenges to improve traffic flow are great. Particularly when western middle classes have
grown, bringing with them an increase in average number of cars per household and therefore total
vehicles on road networks designed for significantly smaller capacities. These trends are likely to be
seen in the fast growing Eastern economies of China and India, which have amongst them some of
the largest cities in the World, in Mumbai and Shanghai. Efficiently processing vehicles through a
road network, particularly in urban areas has therefore become an important economic, social and
technical challenge of the modern industrial World.
In nature we see many examples of systems that can cope with dynamic changing environments
very successfully. These examples range from small scale interactions on a cellular level to processes
that take a quarter of a million years. This report will focus on the nature inspired techniques of
cellular automata, evolutionary algorithms and ant algorithms.
2 Cellular automata in traffic modelling and control
2.1 Cellular automaton introduction
Understanding the flow of traffic has traditionally been considered a time consuming, complex,
multivariable system that requires detailed models to be developed to simulate the behaviour of
individual vehicles. However more recently a lot of work has been focussed on modelling traffic
behaviour using cellular automata models, which have the main advantage of being more efficient
and have a faster performance when simulated on a computer.
Cellular automata have largely been considered a curiosity or side project since early work by Von
Neumann was made famous through Conway’s “Game of life”. In this game, a discrete element
known as a cellular automaton interacts with its neighbours over discrete increments of time, based
on predefined simple, local rules of interaction. These rules often lead to more complex and
interesting global behaviour, and are furthermore influenced by the initial states of the cellular
automata located on a discrete lattice of cells. The global behaviour that emerges is difficult to
predict and many different rule sets have been devised.
Four main attributes need to be defined when building a cellular automata model; the physical
environment, the cell’s state, the cell’s neighbourhood and, of course, a local transition rule
[Quartieri et al, 2010]. The physical environment describes the structure of the lattice that consists
of these cellular automata i.e. is it a finite square lattice, or hexagonal, etc. Typically each cellular
automata will be of equal size, and the dimensionality of the lattice also needs to be defined. A 1D
string of cells would be considered an elementary cellular automaton (ECA). The cell’s state is
normally assigned a binary integer that corresponds to one of two states, like in the “Game of life”
where each cell can either be alive (1) or dead (0). In a 2D lattice neighbourhoods fall in two main
categories, the Moore neighbourhood which considers all of the surrounding eight cells and the Von
Neumann neighbourhood which considers the four cells directly north, south, east and west of a
particular cell. However in an ECA the neighbourhood could be considered simply the two cells
either side of a particular cell or multiple cells in either direction. Finally the local transition rule is
defined that determines whether each cell will change state or not based on its own state and the
states of the cells in its neighbourhood. The same rule must be applied to all the elements else the
system is called a hybrid cellular automaton. If there are no probabilistic interactions incorporated in
the automaton then it can further be called a deterministic model [Quartieri et al, 2010].
2.2 Traffic models using cellular automaton
In the classic model developed by Kai Nagel and Michael Schreckenburg (1992), only single lane
traffic is considered in an ECA lattice, based upon rules that coincide with Wolfram’s rule 184
[Boccara & Fuks, 1998]. Here each of the cells can either be empty or occupied by exactly one
vehicle and the length of each cell is normally defined as 7.5 m; the average length of a car plus
reasonable safety distance. Given a time increment of 1 second, this corresponds to a change in
velocity of 28 km/h or 17.4 miles/h for the Nagel and Schreckenburg model [Quartieri et al, 2010]
which is built for highway (motorway) driving, where these changes in speed are more likely. Each
vehicle in the model has a velocity between zero and Vmax (where Vmax may or may not correspond to
the speed limit), and during a time increment, four consecutive steps are executed to update each
individual velocity [Nagel & Schreckenburg, 1992]:
1. Acceleration – if the velocity v of a car is less than Vmax, and if the distance to the next car is
larger than v + 1, the speed is advanced by one i.e. v = v + 1.
2. Slowing down (due to other cars) – if a vehicle at cell i sees a vehicle ahead at cell i + j
(where j<v), it reduces its speed to j-1 i.e. v = j-1.
3. Randomisation – with probability p, the velocity of each vehicle (if > 0) is decreased by one
i.e. v = v-1.
4. Car motion – each vehicle is advanced v sites.
This stochastic cellular automaton model, due to the inclusion of step three, exhibits non-trivial and
realistic behaviour, despite its relatively simple rules. The randomness incorporates observed human
behaviour, where a car may slow down for no apparent reason.
Figure 2.1 Simulated traffic at a density of 0.1 cars per cell. Each new line indicates a time increment. The
dots represent empty cells and the integers represent cars that are travelling at that velocity. Here Vmax = 5
[Nagel & Schreckenberg 1992].
Note from Figure 2., that the traffic at this density is not free flowing and that a congestion cluster
can be viewed travelling in the opposite direction to the flow of the vehicles. Again, this is a
phenomenon viewed in human road networks.
There is a clear similarity between Figure 2.2 (a) and (b) in the arrowhead shape of the two curves.
Both also peak in between 0.1 and 0.2 cars per cell, which represents the highest flow or throughput
of cars on this single road of traffic. This model has been extended with success to include two lanes
and lane changing behaviour [Nagel et al, 1998].
2.3 Traffic control using cellular automata
Currently traffic is largely controlled by pre-defined traffic light sequences that in built up areas aim
to push traffic along the main “arterial” routes of the road network. Adding time conditions for the
traffic patterns at particular times in the day and for certain days in the week can help to ease the
flow of traffic, but cannot respond to the traffic and are prescribed solutions. More intelligent traffic
control systems can be implemented via a centralised system that can input multiple information
streams and calculate an optimum response, but this relies on each information node, i.e. traffic
light, being connected to the system and for the solution to be arrived at quickly (not always
practical as the problem can become complex).
Cellular automata offer an alternative traffic control system that is, due to the very nature of cellular
automata, decentralised. This allows each traffic light set to work independently and respond in real
time to traffic fluctuations. The traffic problem therefore has moved away from a complex,
centralised optimisation approach and has become focused on adaptation [Szklarski, 2010], which
will be shown throughout this section.
2.3.1 Current traffic control methods
Traffic lights currently are usually set up to run in a cyclic or “marching” system, where in its simplest
form will have a constant cycle period, independent of time or traffic flow. Another approach is to
implement the so called “green wave” or “optim” control, where a car or set of cars stopped by a
given red traffic light, will find the subsequent lights to be green once they are given the green light
to move. This is achieved by setting the traffic lights in the direction of the stopped car to the same
cycling period plus a lag due to distance between lights. The green wave approach is very popular
with avenues that have priority over crossing streets [Brockfeld et al, 2001] and is also advantageous
Figure 2.2(a) and (b) – Both show the flow of cars per time step compared to the density of vehicles on
the road. (a) is based on a simulation using the traffic model described above and (b) is based on real
traffic information [Nagel & Schreckenberg 1992].
during rush hour, when large numbers of cars tend to travel in a single direction. However it still
relies on communication between lights and a centralised system to tell the lights at what times to
exhibit this behaviour, which is based on previous experience or a preconception of traffic flow at
that instance and does not take into account the current conditions.
2.3.2 Self organising traffic lights (SOTL)
Three alternative traffic light control systems are proposed by [Gershenson et al, 2005], the SOTL –
request, SOTL – phase and the SOTL – platoon control systems, which will be referred to without
their prefixes from now on. All three use a similar principle; that all lights keep a counter that is set
to zero when the light turns red. This counter is incremented when an approaching car is stopped, or
is going to be stopped by the given traffic light during its red phase. When the counter reaches a
specified threshold, the traffic lights change colour and cars that have been or were going to be
stopped are allowed to pass through the intersection. This underlying system exhibits self
organisation as, if one or only a few cars are waiting at an intersection, they are likely to be joined by
more cars and as each group grows the less time that they will have to spend at red lights as a large
group is likely to be given precedence at an approaching intersection. This phenomenon is called
creating “green corridors” or “platoons” and is more efficient at improving traffic flow over
homogeneously distributed cars. Platoons simplify the traffic problem and reduce oscillations in the
system by reducing the interactions between groups, as there are less groups to interact
[Gershenson, 2005].
1. Request control – The system is based upon the conditions given above. Traffic lights change
only when necessary i.e. if there are no cars approaching a given light, the partner light may
stay green.
2. Phase control – This system is the same as request control apart from the additional
constraint that the traffic lights will not be induced to change by oncoming traffic until a
minimum phase time has passed. This prevents the lights switching too frequently and
hampering the overall flow of traffic.
3. Platoon control – This system is pro-platoons and incorporates two rules for this. The first
stops a red light from turning green upon approach of car if on the crossing road there is at
least one car coming along the intersecting avenue. This is done so that a platoon is more
likely to stay together. However for a high density of vehicles this would cause jams to build
up along the intersections of passing platoons so to prevent this the first additional rule is
overlooked if there are a certain number of cars approaching an intersection.
Figure 2.3 (i) Average speed, (ii) Percentage of cars stopped & (iii) Average waiting times of various different
methods of traffic control. NB, in graph (iii) waiting times that exceed the limit of the y-axis indicate that the
traffic network has become gridlocked [Gershenson 2005].
Figure 2. clearly shows that at lower densities, the SOTL methods, often far outperform the
conventional marching and optim systems. However at higher densities the trend is reversed. Here
the request system performs worst due to its aforementioned tendencies to constantly change the
lights as at this density of traffic, multiple vehicles are coming from different directions and inducing
very frequent switching. This hampers the overall traffic flow as the switching takes time in the
model.
Platoon is the best performer over a wide range of low to medium density situations, reaching very
high average speeds due to the platoons. Here the cars stigmergically pass information between
traffic light sets. One set of traffic lights affects a cars speed and journey and may build up a platoon
and this platoon will then influence the behaviour of the next set of traffic lights by “demanding” a
light switch. It is important here to note that this platoon behaviour is not forced upon the cars by
the system, but induced, and the cars reciprocate by simulating a synchronisation between traffic
lights that produces very efficient traffic flow. However at higher densities the platoon system fails
because platoons combine too much and cause multiple jams along their length. Future work could
be done here to develop this method so that it could deal with situations that involve high density
traffic, as the Gershenson was reluctant to expand the model, preferring simple rules.
The model was expanded however to mimic further real traffic characteristics. A directional
preference was introduced to simulate rush hour traffic, traffic flow in all four directions was
considered and cars given a probability of turning at intersections. Overall however none of these
characteristic significantly changed the overall behaviour of the different traffic systems. Further
work can still be done to better our understanding of these traffic systems. The work could be
improved to include multiple lanes, different driving behaviours, lane changing behaviour, multiple
street intersections and non-homogenous streets. Another interesting condition to consider would
be how the systems cope with random delays, incorporated to simulate traffic accidents. Which
system will gridlock first on average?
The reader can get a better appreciation of how the different methods compare using an online java
application [NetLogo, 2009].
2.4 Cellular Automata in traffic summary
Cellular automata have provided efficient and realistic ways of simulating traffic flow that is
exhibited in current road networks. Nagel and Schrenkenberg’s [1992] original work has been
extended to incorporate lane changing behaviour to good success [Nagel et al 1998] but further
work could be done to characterise better different driving patterns, and different vehicles e.g.
cyclists and lorry drivers. The SOTL systems provide much better traffic throughput at low to
medium traffic densities than conventional rigid traffic control, and have the huge advantage of
being easy to implement and cheaper as there is no need for central control and traffic controllers to
modulate the frequency of traffic phases, and they adapt to the current traffic situation. However
further work needs to be done to increase their performance at high density traffic, even if this
means simply switching back to a marching phase system once a critical density is reached (this
would however have to be implemented centrally). A higher density situation is less likely to exist in
the first place as traffic is more efficiently processed in the first place so perhaps this is more of a
tweak than a major hurdle. Overall traffic flows could be increased by 10 – 30% via self-organising
traffic light systems [ScienceDaily, 2010] and if traffic lights could determine between types of
vehicles, then precedence could be given to public transport and cycling, which in turn would reduce
traffic.
3 Introduction to Evolutionary Algorithms
Evolutionary Algorithms, like all of natural computation draw on inspiration from nature to help
solve problems that traditional methods of computer science cannot solve, or solve inefficiently. All
evolutionary algorithms use the same common ideas; some form of genetic structure, a system for
determining success for each generation and mutation on the genetic structures. The genetic
structure somehow maps to what the data that needs to be “evolved”, this is mixed with other areas
of evolutionary theory such as natural selection (in the form of fitness functions) and genetic cross
over. Evolutionary Algorithms are good for problems where there is a base to build up on, as the
base can be used as the first generation and anything evolved after that will be an improvement on
that base.
3.1 Salt Routing Optimisation
The data structure for this problem was in the form of a graph with routes between nodes. The
route that a salting truck takes is stored in the form of a list of numbers which relate to the nodes
that the salting truck visits. This is the “chromosome” that the evolutionary algorithm runs on. The
fitness function is a way of testing the solution against the constraints in the system.
Figure 3.1 A tour example for a chromosome 2 6 s1 5 4 7 1 s2 8 3
The fitness function used for the SRO problem is:
F = (Cti + Cp x Eti),
Where Ct refers to the cost of the trip ti of the ith
truck and Cp is the predefined coefficient for the
penalty term. Eti is an indication of the quantity of constraint violation in each truck, or:
Eti = Dti - Li if Dti - Li > 0
0 Otherwise,
Where Dti is the total services demands by truck i and Li is the capacity of the services for truck i. The
fitness function is the main part of any evolutionary algorithm and if it’s poorly designed then the
evolutionary process will not run in the intended manner. This Salt Routing Optimisation system
combines this evolutionary computational approach with XRWIS (neXt generation Road Weather
Information System) which predicts the road surface temperature by taking into account factors
such as sky-view factor, altitude and landuse. [Handa et al 2005]
The only possible problem with this system is that the routes will be completely different for each
time the trucks need to salt the road network, so once an optimised solution is found; it is
improbable that that solution will be suitable for later use. This means whenever the need for an
optimised route arises, the algorithm will have to run which could take a significant amount of time.
3.2 Adaptive Traffic Control
This is a problem involving the flow of traffic and offers a solution that uses evolutionary computing
in a multi-agent system. The project actually uses a machine learning technique based on an LCS
(Learning Classifier System), which is a type of evolutionary algorithm. Learning Classifier Systems
evolve rules that consist of conditions and actions (similar to if... then... clauses). The Learning
Classifier System is used with a microscopic simulation model SIGSIM, which is used to simulate the
movement of vehicles in a signal-controlled road network (i.e. traffic lights). The Learning Classifier
System is used to develop signal control strategies and test the efficiency of the solution by using
data and measures of performance generated by SIGSIM simulations. The fitness function for this
project takes into account parameters such as: delay to vehicles, capacity of critical streams and
junctions, queue lengths, and characteristics of the signal timings.
The experiment was setup in SIGSIM as a four way junction crossroads junction with eight lanes.
Traffic goes in one direction and goes straight at the junction
Figure 3.2 4-arm junction, vehicles travelling ahead only.
Figure 3.3 shows the results gathered by this experiment, and how two implementations of the
Learning Classifier System compares to another three kinds of Adaptive Traffic Control, System D
Vehicle Actuated Control (V/A in the table) and normal and optimised fixed time (F/T).
Figure 3.3 Mean rates of delay over simulation run (after warmup)
Looking at the table it is clear that Learning Classifier System has competitive results when
compared to both F/T’s and the V/A systems. However scalability seems to be an issue with the LCS,
for even spread 200 and 400 streams the LCS systems hold up and give similar results to the others;
however it seemed to give slightly worse results for the 600 and no data is given for the 800.
Also in the real world, traffic at a cross road junction such as the one in the simulation would not
have an even distribution (i.e. the traffic flow would not be equal in all four directions). The
simulation was run with uneven traffic flow, however only one of these has results for the LCS and it
seem to not work as well as the V/A and optimised F/T systems. [Sha’Aban et al. 2008]
3.3 Evolutionary Computation Applied to Urban Traffic Optimisation
An important area of traffic management is the optimisation of traffic light cycles; getting the lights
at junctions to cycle in such a way that offers the best throughput for certain times of the day. This
paper used a similar technique to that used in the Adaptive Traffic Control paper above; a Genetic
Algorithm to optimise a microscopic simulator, which they claim is a more accurate estimation of
future traffic. The set up for the system was in three main sections; a genetic algorithm acting on a
CA (Cellular Automaton) all working on a Beowulf Cluster of computers. The GA was used for the
optimisation process, with the CA being used as the basis for the traffic simulator that runs inside
the GA.
The GA itself had some interesting optimisation criteria (parts of the fitness function). They were
mainly interested in how many vehicles left the system rather than the amount of vehicles in the
system itself. This makes a lot more sense when they justify it; “During the traffic simulation many
new vehicles are created as if they were arriving at the inputs... many vehicles reach their
destination point easily...” Other criteria included the Mean Elapsed Time (MET) or how long each
vehicle spent inside the system , the standard deviation of these values, a linear combination of the
MET and standard deviation and a linear combination of the MET and the number of vehicles that
left the system. There was also a traffic network mean occupancy density; the network was divided
into small sections and the number of vehicles in each section was that density
Minimising the MET however gave undesired effects; it ended up giving good values but for the
wrong reasons. Chromosomes that blocked the network faster were favoured, resulting in “false”
optimisation cycle combinations. The chromosomes related to the combinations of the traffic lights
and the aim was to optimise the duration of a predetermined cycle of light settings. They chose to
use Gray Encoding, which is a form of bit string encoding used a lot in evolutionary computing. This
is because when mutation occurs, or when a bit changes the stage length value increases/decreases
by one unit only. (Stage length refers to the light being in a certain state for a certain amount of
time) There are many methods to choosing which individuals go through to the next generation in
evolutionary computation. This system combined Elitism and Truncation, Elitism is when the
generation is run through the fitness function and sorted in terms of fitness and only the top section
are cloned to the next generation. Truncation is where the individuals are again ranked by fitness
but then split into two section, one survives and the other is discarded. [Sanchez et al. 2008]
3.4 Decentralised Car Traffic Control Using Message Propagation
This paper had a very interesting approach to solving the traffic problem in society today, optimising
throughput at junctions and reducing the overall travel time of vehicles on the road. The system
works on the idea of message propagation between different road elements; such as road junctions,
traffic lights and even the vehicles themselves. The system used a genetic algorithm to help optimise
itself using a fitness function. The GA actually generates the parameters for the simulation rather
than optimising the running of the simulation. The prototype system consisted of 2 types of vehicles,
normal and emergency (such as fire engines, ambulances and police cars) and 2 types of message
speed-up and slow-down. The main system that the paper focuses on has much more functionality
for example re-routing traffic in times of congestion. Also the messages were changed from speed-
up/slow-down to go/stop in the terms of traffic lights rather than vehicle speed. This helped
optimise traffic in congested conditions.
The size of the simulation was very large compared to others; the system’s area was modelled as a
20 x 20 node system which represented a 2km x 2km city. Each node was an intersection with a set
of traffic light attached to it, and lines connected them were lanes with a direction, with 8 lanes
connected to each node (i.e. a crossroad junction). The simulation consisted of ~15,000 vehicles
travelling along ~1,500 interconnected lanes. The simulation started with a small number of vehicles
in random places in the system, which then targeted a location on the opposite side and deduced an
optimal route. This provides congestion to some routes from the start, the vehicles loop through
completing routes and starting new ones until the simulation is over. The fitness of a solution is
determined by the throughput of the system, or the most number of completed journeys in a fixed
amount of time (half an hour of virtual activity). The vehicles re-evaluate their route whenever they
wish and use a path finding algorithm that has parameters from the GA.
The message propagation was between the traffic junctions to request signal changes depending on
the current traffic situation, this happened in both directions. Lanes can send messages forward to
request signal changes, for example if there is a lot of stationary traffic in that lane. Backwards
message propagation is also used by lanes in cases where the lane is congested, effectively closing
the lane. Nodes cycle a token between inbound lanes to grant access to outbound lanes.
There were five parameters that the GA generated for use in the simulation;
• Evaluation Trigger, the probability that a vehicle will re-evaluate its route.
• Response Threshold, the probability that a node will respond positively to a signal change
request.
• Request Threshold, the probability that a lane will raise a signal change request.
• Request Limit, The amount of requests a node can ignore before a change has to be made
• Phase, the minimum distance from an intersection that a vehicle must be before the lane
passes the token.
One problem with this solution is that the parameters had seven set values that were used by the
system as shown in the table below. This forces restrictions on to the system, which maybe could be
better if they were not there and the five parameters could take a ranged approach but have no
fixed values.
Figure 3.4 Parameter variables
All the parameters are indirectly interlinked; the message propagation part of the system has effects
on the routing and re-evaluating part. As traffic builds up lanes restrict access to themselves and
pass the access on to other lanes. This causes the journey times of the vehicles to increase and
increasing the chance that they will re-evaluate their routes and travel through a less congested
area.
The GA was set up with 49 initial individuals, which is evaluated for reproduction, applying cross-
over and mutation to result in two child genes. The offspring are run through the simulation and the
next generation is selected using roulette wheel selection (roulette wheel selection is where every
individual has survival probability proportional to its fitness). The system was set up to test only
unique genomes; no gene sequence will be evaluated twice. If a gene sequence performs better
than the worst candidate then it is added to the list and the worst is dropped from the “gene-pool”.
The genes for the GA are the five parameters mentioned above and have seven values, which results
in 16,807 gene sequences. The gene-pool always contains at the most the 49 best solutions and if no
additions are made the mutation rate is increased and two new individuals are added.
The results related to 158 generations through the GA, 3315 simulations were performed and the
optimum was found at generation 133. The system created a 13% variation between the worst and
best journeys and according to [Kelly et al.] “Given that each of the 1520 lanes in the 2km*2km block
begins filled to 50% capacity, and there are 15,200 vehicles vying for scarce resources, 13%
enhancement can be considered a good result.”
Figure 3.5 Global results
The choice of starting off with such a congested system makes sense, as [Kelly et al.] wished to show
that message propagation and re-routing could help ease congestion in an urban environment.
Out of the four papers this one seems the most promising and realistic. The parts of the system all
work together and there seems to be a lot of potential for future work. Cars that speak wirelessly to
traffic lights and self control the traffic between them seems like an idea that could be a reality in
the not so distant future [Kelly et al. 2007].
4 Ant Colony Optimisation
Throughout the world, optimal traffic routing is becoming more and more important as the number
of cars on the roads increases. As a result, road systems which were designed for capacities much
smaller than those observed today are unable to cope. In 2008 Rod Eddington estimated that for the
UK, “unless we take action, congestion would cost us an extra £22 billion in wasted time by 2025”
[National Archives, 2008]. One approach to dealing with the increase in traffic is to develop
innovative ways to plan routes. Current research is particularly aimed at the nature inspired
technique of Ant Colony Optimisation.
Ant Colony Optimisation takes advantage of the stigmergic behaviour of ants to find approximate
solutions to difficult optimisation problems. This form of optimisation is inspired by the simple trail-
laying and -following behaviour of ants and was studied by Goss et. al. [1989]. Similarly to Cellular
Automata, Ant Colony Optimisation takes advantage of the simple behaviour of individual agents in
a large population which produces global behaviours much more complex than the rules of the
individual.
The main behaviour used to solve optimisation problems is the route finding technique that ants use
to optimise the shortest route between their nest and a food source. In nature, ants wander
randomly around and upon finding food, return to their nest while laying a down pheromone trail. If
another ant finds this trail there is a chance they will stop wandering randomly and follow the trail. If
they find food, they will return to the nest and reinforce the trail with their own pheromones.
Goss et. al. [1989] produced an experiment that demonstrated this ability in scientific setting. Their
experiment consisted of two trays: one containing a colony of ants and the second containing a food
source. The only path between the two trays was a bridge designed to provide the ants with a range
of routes, but only one shortest route and with no bias to a particular route (as shown in Figure 4.1) .
They found that the shortest route was developed using the following process:
1. The first ants arrive, choose randomly and travel along their chose branch, marking with
pheromone as they go.
2. After 20 s (the time it takes to travel the shortest path to the food), the short path will be
marked not only by ants arriving from the nest, but also by the first ants returning to the
Figure 4.1 (a) the design of the bridge between food and nest, photos taken at 4 (b) and 8 (c) minutes
after the placement of the bridge [Goss et. al. 1989].
nest that chose the short branch.
3. At this point the short branch accumulates an advantage over the long branch at both ends,
an effect which is amplified by the autocatalytic nature of the choose-and-mark process.
Thus, the short branch becomes rapidly preferred.
Whilst the ants do not communicate directly, they alter the environment to notify the rest of the
colony of the location of a food source. This behaviour is called stigmergy and it is a fundamental
part of the problem solving technique.
In computer science this method provides an excellent basis for an optimisation algorithm. The use
of multiple agents with simple rules makes modelling complex behaviour much simpler, and the
stigmergic behaviour of the optimisation technique allows the current status of the model to be
stored at each time step. For these reasons and due to the simplicity of the optimisation technique it
is often used to solve a range of Vehicle Routing Problems.
4.1 Vehicle Routing Problems Vehicle Routing Problems requires the determination of an optimal set of routes for a fleet of
vehicles that needs to serve a set of customers [Balserio, Loiseau and Ramonet, 2011]. In addition to
find the optimal route, these problems can have a range of constraints including limits on the
capacity of the vehicles, time windows for the customers to be served, and limits on the time a
driver can work or on the lengths of the routes.
A study by Balserio, Loiseau and Ramonet [2011] dealt with a vehicle routing problem where a set of
customers must be served by a fleet of vehicles located in a single depot. The problem was modelled
on a large metropolitan where travel times vary. That is, the time taken to travel between two
points does not depend solely on the distance: traffic fluctuations seriously affect travel speeds,
resulting in great variations in travel times. Other constraints included allowing the clients to impose
time windows during which they can be served. A vehicle can arrive before the beginning of the
window but is not permitted to begin serving them until the start of the window or after the end.
The capacity of the vehicles was also limited in this study.
In order to solve the vehicle routing problem with this range of constraints, Balserio, Loiseau and
Ramonet [2011] use a Multiple Ant Colony System. This uses a pair of ant colonies to optimise the
two main solutions. The first colony (ACS-TIME) optimises the best total time solution with the given
number of vehicles v, whilst the colony ACS-VEI tries to minimise the number of vehicles by
minimizing the number of unrouted clients in solutions with v-1 vehicles. These are optimised in
order of priority: (1) ACS-VEI and (2) ACS-TIME. A solution that employs fewer vehicles is better than
another that uses more in spite of total time. Only when two solutions have an equal number of
routes are their times compared. This objective is typical of Vehicle Routing Problems because the
fixed cost of drivers and vehicles usually has a “greater impact than per-mile transportation costs”
[Balserio, Loiseau and Ramonet, 2011].
4.1.1 Representation and Evaluation
Ant Colony optimisation is typically represented by a graph in which nodes are connected by edges.
Each edge has a probability that it will be travelled by an ant and this probability represents the
pheromone trail lain down by the ants. In this study, the probability pij that an ant will move to client
j from the current node i is computed base on two values: a heuristic value ηij that yields closer
clients more attractive, and the pheromone level τij. The formula for calculating the probability is:
where α and β are parameters determining the relative importance of η versus τ, and Ni denotes the
set of feasible nearest neighbours of node i. Neighbours are feasible if (i) they are not visited, (ii)
their demand does not exceed the remaining capacity of the route, and (iii) they can be reached
before the end of their time window [Balserio, Loiseau and Ramonet, 2011]. The heuristic value ηij
varies between ACS-VEI and ACS-TIME. Both take into account the distance to the customer, and the
urgency of the delivery (based on the time remaining until the end of a customer’s service window).
ACS-VEI also increases the attractiveness of clients that are not frequently served in order to
produce more complete solutions.
Balserio, Loiseau and Ramonet [2011] observe that “a problem with Ant Colony Optimisation
algorithms for routing problems is that, as constraints tighten, ants tend to generate infeasible
solutions with unrouted clients.” For this reason they added a set of insertation techniques that
added unrouted clients into the solutions found by the Ant Colony Optimisation, provided the
addition of a new client did not violate any of the problem constraints. This works in a similar way to
the mutation and crossover techniques used in genetic algorithms. Improvements were made to the
routes produced by the Ant Colony Optimisation to improve the performance of the optimisation.
The performance of the algorithm was tested against the results of other solutions to the vehicle
routing with time windows problem which have been developed since 1987. By using the same set
of parameters used in original collection of result the algorithm developed by Balserio, Loiseau and
Ramonet [2011] produced matching best solutions to 40 out of 56 benchmark problems. In the time-
dependant framework they improved on all previously published results, leading the authors to
claim that their algorithm is “fairly robust” [Balserio, Loiseau and Ramonet, 2011].
However, one could argue that due to the centralised nature of the algorithm – the fact that all
routes are developed in advanced from a centralised computer, using a static representation of an
metropolitan environment that, in reality, varies greatly from minute to minute - this solution is not
that robust.
4.2 Time-dependant Travel Times In this digital age, it is possible to model road conditions down to the minute. An excellent example
is XRWIS used by Handa et.al. [2005] in the Salt Routing problem mentioned above, which allows
them to predict road temperatures with a spatial and temporal resolution of 20 meters every 20
minutes. Similarly, a study by Schmid and Doerer [2010] used data collected by a fleet of taxis in the
city of Vienna over several months to model the average velocity of traffic in the city 15 minute
intervals for a full day.
Although these studies do not deal specifically with Ant Colony Optimisation, they do lead the way in
using dynamic and up-to-date data to produce optimal solutions to routing problems. The study by
Schmid and Doerer [2010] aims to find the optimal position for a fleet of ambulances throughout the
day in the city of Vienna, in order to maximise the amount of population covered by the ambulance
service. The study not only takes into account the variation in travel times throughout the day
(Figure 4.2), but it also considers the fact that some ambulances may become unavailable
throughout the day. For this reason the study optimises the solution by moving the fleet around the
city throughout the day, depending on the traffic situation. Figure 4.2(a) shows how during the peak
hour the ambulances are clustered towards the city centre, in order to provide coverage in an area
where travel times are slow, however during the off-peak hour (Figure 4.2(b)) the distribution is less
clustered.
Schmid and Doerer [2010] develop two algorithms based on a Double Standard Model (DSM). The
first is the Myopic Double Standard Model (Myopic DSM) in which the ambulance location was
optimised individually for the average travel time in each time interval, without consideration for
travel-time variations throughout the day. The second algorithm is the Multi-period Double Standard
Model (mDSM) which optimises a solution for each time interval, taking into account the variation in
travel-time throughout the day. Figure 4.3 shows the results from this study. Favg shows the average
values for the objective function for each time interval, ravg shows the average number of vehicle
relocations. The most significant result of the study is relative change column for the ravg. This shows
how using time-dependant data improves the average number of vehicle locations by 50%, whilst
the overall quality of the solution only deteriorates by 0.36%.
Although this study takes into account the variations in travel time throughout the day, it is still not
considered robust. Due to its centralised nature the system would not be able to deal with any major
changes to the transport network, an accident for example, without the travel time being
recalculated and the optimisation algorithm run again. Although it uses nature inspired techniques
to optimise a solution, the solution does not adapt well to change in the system – a property that is
considered fundamental to good nature inspired solutions.
Figure 4.2 Spatial pattern of solution and traffic situation during peak and off-peak hours [Schmid and
Doerer, 2010].
Figure 4.3 Solutions for Myopic DSM vs. mDSM [Schmid and Doerner, 2010].
4.3 Dynamic Traffic Routing Truly robust solutions to routing problems are able to deal with dynamic spatial and temporal
variations in the environment. The key to success in natural ant colony route optimisation is the
decentralisation of the system. There is no single “overlord ant” controlling the route. It is the
stigmeric behaviour of all ants which allows the optimal solution to be found, regardless of a change
in the environment.
Two papers by Tatomir and Rothkrantz [2004, 2006] present a dynamic routing system called
Hierarchical Routing System. Using large numbers of small, autonomous sensors installed in the road
together with an Ad-Hoc network formed by wireless enabled devices in vehicles, the authors
present a system that mimics the stigmeric behaviour of an ant colony. The result is a hybrid
network shown in Figure 4.4 which provides road users with dynamic routing.
In order to test the viability of the system, Tatomi and Rothkrantz [2004, 2006] built a simulation
environment that guides vehicles between cities taking into account the travel time and the load on
the roads. As is typical when using Ant Colony Optimisation, the problem is represented using a
graph to model the road network. The hierarchical nature of the system allows a large area to be
split up into local sectors, with edges connecting these sectors representing motorways between
cities for example (Figure 4.5).
As demonstrated in the study by Henda et. al. [2005] a dynamic source of data allows better
solutions to be generated. In this system, Tatomi and Rothkrantz [2004, 2006] suggest that the
Figure 4.4 The hybrid routing system proposed by Tatomir and Rothkrantz [2006].
vehicles themselves provide data for the routing system by communicating their route and the time
it took to travel each section to the Hierarchical Routing System. This allows the travel time for each
edge to be stored in probability tables which a “smart vehicle,” as described in this study, will have
access to when planning a route.
A simulation environment was built to allow Tatomi and Rothkrantz [2006] to test the
responsiveness of the system to changes in travel time. The simulation used a skeleton road network
with four cities connected by motorways, and 4 vehicle categories:
1. Vehicles that have no communication with the routing system and are guided by a regular
route finding algorithm;
2. Vehicles that give information about the state of the traffic system but do not use it (e.g.
public transport);
3. Vehicles that give no information, but do use it for routing;
4. Vehicles that use the routing system and also provide it with information.
Categories one and two are called ‘standard vehicles’ and categories three and four are called ‘smart
vehicles.’ The simulation is run for a period of time to allow the system to “warm up” and collect
data about road use. After 2500 time steps an accident is generated between two cities by artificially
reducing the speed down to 20km/h on that edge. The results can be seen in Figure 4.6.
Figure 4.5 A network model used by Tatomir and Rothkrantz [2006].
Figure 4.6 The results of the Heirarchical Routing System for standard vehicles (left) and smart vehicles
(right) [Tatomir and Rothkrantz, 2006].
Although the trip time of the smart vehicles increases rapidly, it only increases the trip time by about
300 time steps, whilst the standard vehicle trip time increases by at least 1000 time steps. At 10000
time steps an additional delay is added to the road network. This only costs the smart vehicles an
additional ~150 units of time, whilst the standard vehicle trip time increases by a further 500 units.
At time 15000 the original block is released. It is clear that the standard vehicles respond most
rapidly to this change in the system, but once the smart vehicles notice the change in the system
they also begin to reuse this route.
5 Conclusion
Throughout the research in this area, it has become clear that current traffic systems are inefficient,
often over complicated and too centralised. A decentralised system that can respond dynamically to
the current traffic state is preferred, over one that has been carefully prepared, based on average
data observations, but is more rigid. This is due to the inherent random or naturally fluctuating
behaviour of traffic that is exhibited because of our human nature.
It is clear from these studies that the most robust solutions are those that take into account the
dynamic changes in the environment and are able to adapt to them on the fly. Implementing the
solution proposed by Tatomi and Rothkrantz [2006]would be expensive, but not difficult. Road side
sensors could be added with ease and the wireless units in the cars could be developed alongside
existing GPS solutions such as the TomTom or NavMan which are commonly used today. It is also
important to consider the economic impact of implementing this solution. The smart vehicles travel
times were seven times shorter than the standard vehicles and although this would not correlate to
similar reduction in the cost of congestion, the savings would still be considerable.
The dynamic traffic routing method described in the ant algorithm section, the SOTL method in the
cellular automata section and the message propagation method in the evolutionary algorithm
section are all decentralised and show the most promise. Future work should focus on developing
these ideas, that the cars and traffic lights should interact independently of a central control unit,
and more complex models be tested. It would also be interesting to develop all three good solutions
into a single solution and evaluate the effectiveness of multiple systems working together. The fact
that all these solutions use simple rules should not make it difficult to integrate them into a single
solution, however the cost of evaluation may be considerable.
Finally, although projects have been implemented on real road networks in some cases [Gershenson,
2005], these have largely been carried out on isolated or small areas. Implementation in a city or
over a large area, would advance this field significantly.
6 References
Balserio, S. R., Loiseau, I. and Ramonet, J., 2011. An Ant Colony algorithm hybridized with insertation
heuristics for the Time Dependen Vehicle Routing Problem with Time Windows. Computers
& Operation Research, 38, 954 – 966.
Biham, O, Middleton, A.A and Levine D, 1992. Self-organization and a dynamical transition in traffic-
flow models.
Boccara, N & Fuks, H, 1998. Cellular automaton rules conserving the number of active sites.
Adaptation and Self-Organizing Systems. Journal of Physics A: Mathematical and Theoretical.
Brockfeld, E et al 2001. Optimizing traffic lights in a cellular automaton model for city traffic. Physical
review E, Vol 64.
Gershenson, C., 2005. Self-organizing Traffic Lights. Complex Systems, 16, 29-53.
Goss, S., Aron, S., Deneubourg, J. L. and Pasteels, J. M., 1989. Self-organised Shortcuts in the
Argentine Ant. Naturewissenschaften, 76, 579 – 581.
Handa, H et al. Dynamic Salting Route Optimisation using Evolutionary Computation.
Kelly, M et al, 2007. Decentralised Car Traffic using Message Propagation Optimised with a Genetic
Algorithm.
Medina, J et al, 2008. Evolutionary Computation Applied to Urban Traffic Optimisation. Advances in
Evolutionary Algorithms.
Nagel, K and Schreckenberg, M., 1992. A cellular automaton model for freeway traffic. Journal de
physique I, page 2221-2229.
Nagel, K et al., 1998. Two-lane traffic rules for cellular automata: A systematic approach. Physical
Review E, Vol. 58 No. 2
National Archives, 2008. Delivering choice and reliability [Online]. Available at
<http://webarchive.nationalarchives.gov.uk/+/http://www.dft.gov.uk/press/speechesstate
ments/speeches/congestion> [Accessed 6 March 2011].
NetLogo, 2009. A model of city traffic based on elementary cellular automata [Online]. Available at
<http://turing.iimas.unam.mx/~cgg/NetLogo/trafficCA.html> [Accessed 5 March 2011].
Quartieri, J et al, 2010. Cellular Automata Application to Traffic Noise Control. Automatic Control,
Modelling & Simulation. 12th
WSEAS International Conference.
Schmid, V. and Doerner, K. F., 2010. Ambulence location and relocation problems with time-
dependent travel times. European Journal of Operational Research, 207, 1293 – 1301.
ScienceDaily, 2010. Self-Organizing Traffic Lights. ScienceDaily [Online]. Available at:
<http://www.sciencedaily.com/releases/2010/09/100915094416.htm> [Accessed 4 March
2011].
Sha’Aban, J et al, 2008. Adaptive Traffic Control using Evolutionary Algorithms.
Szklarski, J., 2010. Cellular automata model of self-organizing traffic control in urban networks.
Bulletin of the polish academy of sciences, Technical sciences, Vol 58, No. 3.
Tatomir, B. and Rothkrantz, L., 2004. Dynamic traffic routing using Ant Based Control. IEEE
International Conference on Systems, Man and Cybernetics.
Tatomir, B. and Rothkrantz, L., 2006. Hierarchical routing in traffic using swarm-intelligence. IEEE
Intelligent Transport Systems Conference, Toronto, Canada, September 17 – 20, 2006.
Wolfram, S, 1986. Theory and Applications of Cellular Automata. World Scientific, Singapore.