optimisation of the beer distribution game.pdf

8/13/2019 Optimisation of the Beer Distribution Game.pdf

1/8

Optimisation of the Beer Distribution Game with ComplexCustomer Demand Patterns

Hongliang Liu, Enda Howley and Jim Duggan

Abstract This paper examines a simulation of the BeerDistribution Game and a number of optimisation approaches tothis game. This well known game was developed at MIT in the1960s and has been widely used to educate graduate studentsand business managers on the dynamics of supply chains.This game o ff ers a complex simulation environment involvingmultidimensional constrained parameters. In this research wehave examined a traditional genetic algorithm approach tooptimising this game, while also for the rst time examininga particle swarm optimisation approach. Optimisation is usedto determine the best ordering policies across an entire supplychain. This paper will present experimental results for four com-plex customer demand patterns. We will examine the e ffi cacy of our optimisation approaches and analyse the implications of theresults on the Beer Distribution Game. Our experimental resultsclearly demonstrate the advantages of both genetic algorithmand particle swarm approaches to this complex problem. Wewill outline a direct comparison of these results, and present aseries of conclusions relating to the Beer Distribution Game.

I. Introduction

This paper examines the well known Beer DistributionGame (BDG), which is a commonly used supply chainanalysis tool. This simple game has been widely used foralmost ve decades to illustrate human decision making andthe concepts of supply chain management [1], [2]. The tradi-

tional game is normally played by four players, representingfour individuals, a retailer, a wholesaler, a distributor anda manufacturer. Each individual faces a decision makingchallenge involving how they manage their current stock inventories. Each participant in the game seeks to minimisetheir total cost by managing their inventories in the face of uncertain demand. It has been shown that this simple gameprovides complex and often non-linear dynamics due to feed-backs and time delays. It has been shown through simulationand also real life experiments that game participants ndit extremely di fficult to perform well in this game. Theirdecisions commonly result in large divergences which arefar from optimal behavior. These result in large oscillations,deterministic chaos and other forms of complex behavior [3].

The primary goal of this paper is to examine optimalsolutions for the BDG. This becomes increasingly di fficultwhen more complex customer demand patterns are examined.We dene optimality as the best set of strategies for theoverall efficiency of the supply chain, thereby, minimising thetotal costs for all participants of the game. Determining themost optimal ordering policy is always a di fficult problem for

Hongliang Liu, Enda Howley and Jim Duggan are with the De-partment of Information Technology, National University of Ireland,Galway (email: [email protected], [email protected], and

[email protected]).

any participant in the Beer Distribution Game. A number of possible approaches will be examined in this paper. We willpresent a particle swarm optimisation (PSO) approach. SuchPSO approaches have been successfully used in other similarcontexts to nd optimal solutions, yet, this is the rst time ithas been applied to the BDG. We will also examine optimalsolutions found using a genetic algorithm (GA) approach.This paper di ff ers from existing research which is primarilybased on very simple customer demand patterns [1], [4]. Thispaper will address a number of important research questions:

1) How e ff ective are PSO and GA approaches to optimis-ing the BDG?

2) What are the di ff erences between our PSO approachand other optimisation approaches to this problem?

3) What are the e ff ects of more complex customer de-mand patterns on the results?

4) What are the di ff erences between common and distinctordering policies among game participants?

These research questions will be referred to regularlythroughout this paper and answered directly in the Conclu-sions section. The following sections of this paper are struc-tured as follows. In Section II, we will discuss backgroundresearch and in Section III we will outline our experimental

setup. Section IV will provide a detailed examination of our experimental results. In Section V we will outline ourconclusions, while nally in Section VI we will brieysummarise the contributions of this paper and outline somefuture work.

II. Background Research

Our interpretation of the BDG is based on the specicationoutlined by Sterman [1]. Later sections will provide a de-tailed overview of Particle Swarm Optimisation, and GeneticAlgorithm search techniques. These sections will also outlineprevious optimisation approaches to the BDG.

A. Introduction to the Beer Distribution GameBDG is a classic supply chain optimisation problem [5].

It has been widely used in the domain of supply chainmanagement [5], [6], [7]. This game o ff ers a simpliedimplementation of common real world production and distri-bution systems. This system consists of four participants: Re-tailer, Distributor, Wholesaler and Manufacturer. As shownin Fig. 1, each participant has control and responsibility forits own inventory.

Retailer: The retailer receives orders from customersthough the customer demand. Subsequently the Re-tailer must order beer from the Wholesaler to replenish


2/8

CurrentInventory

Retailer

CurrentInventory

Wholesaler

Shipment

CurrentInventory

Distributor

Shipment

CurrentInventory

Manufacturer

Shipment

Customer Demand

ProductionDelay

Ordering Ordering Ordering

Fig. 1. The structure of the Beer Distribution Game

its inventory. These orders will then arrive after aspecied time delay and when the Wholesaler is capableof fullling those orders.

Wholesaler: The Wholesaler supplies beer from itsinventory to fulll its orders received. The Wholesalerorders and receives beer from the Distributor.

Distributor: The Distributor supplies beer to theWholesaler and receives beer from the Manufacturer.

Manufacturer: The Manufacturer brews beer in orderto maintain its own inventory and fulll orders from theDistributor.

These processes are e ff ected by a number of delays. Delaysrepresent shipping delays and order receiving delays. Theseinvolve the time it takes to receive, process, ship and deliverorders. In the case of the manufacturer these delays representthe amount of time required to replenish its inventory. Thesedelays are an intrinsic part of the BDG. Their presencecauses signicant challenges in attempting to maintain opti-

mal levels of inventory. Each week, every participant mustdecide how much to order from its respective supplier or howmuch to brew in order to meet current and future demands.It is common for each participant to order more (or less)than actually needed due to instabilities in the supply chain.Furthermore, demand information is often distorted in thesupply chain from the end customer to the manufacturer.This phenomenon is known as the bullwhip e ff ect [8].

As we have stated previously, the objective of participantsis to minimise cumulative costs over a 150 week periodby keeping inventories as low as possible while avoidingout-of-inventory conditions which cause backlogs. The BDGcommonly uses the following costs to penalise inventoryholding and backlogs. The cost of inventory holding is $0.5for each case of beer per week and the cost of backlogsis $2.0 for each case of beer per week. It is intuitive fora player to order more beer when inventory falls below adesired level. Similarly a player is likely to order less beerwhen stocks begin to accumulate.

B. The Beer Distribution Game Ordering Policy

Since all the game participants experience time delays, itbecomes quite di fficult to manage their inventory levels. Todeal with this challenge, Sterman proposes that the partici-pants adopt three ordering principles [1]. Stermans ordering

principles are as follows, and state that game participantsshould place su fficient orders to:

Satisfy Expected Demand: The participants shouldorder enough to satisfy the demand. However, to predictthe exact future customer demand is often a di fficulttask. In this paper, we use an adaptive expectationsformulation (see Equation (3)) to model future customerdemand.

Adjust Inventory Levels: There is a chance of predic-tion errors from the previous principle. Therefore, it isnecessary to adjust orders above or below the expectedorders. This serves to correct actual inventory levels inline with desired inventory levels.

Adjust for Orders Currently in the Supply Line:Orders currently in the supply line should be factoredinto future ordering decisions. Therefore, a participantshould be capable of ensuring a stable response to rapidchanges in customer demand. It is pointless to place

orders for items already ordered in a previous time step.The following equations formalise an intuitive ordering

policy based on the principles discussed above. We will referto this as the Stock Management Structure (SMS). First,orders must be nonnegative:

OP t = max(0 , OP

t ) (1)

In Equation (1), OP t represents the actual orders placed atweek t , whereas OP t represents the orders calculated throughthe ordering heuristic at week t . This is dened as follows:

OP t = E D t + ( INV

t OS L) (2)

In Equation (2), ED t represents the expected demand atweek t (see Equation (3)), INV t represents the discrepancybetween desired and actual inventory at week t (see Equation(4)), OS L represents the orders in the supply line; and represent the adjustment paraments for the inventory andthe supply line, respectively; The expected demand ( ED t ) inEquation (2) is expressed as:

ED t = CD t 1 + (1 ) ED t 1 (3)


3/8

In Equation (3), ED t and ED t 1 are the expected demandat week t and t 1; C D t 1 is the customer demand at week t 1, describes the rate at which the demand expectationsare updated. The parameter is typically set to 0 .25 [1]. Thediscrepancy between desired and actual inventory ( INV t ) inEquation (2) is formulated as follow:

INV t = Q INV t + BLt (4)

In Equation (4), Q represents the desired inventory level, INV t represents the current inventory and BLt represents thebacklogs at week t . It is clear from Equation (2), that theSMS comprises of three factors: expected demand, inventorylevel and orders in the supply line. In the BDG, Equation (1)is used to determine how much beer to produce in the caseof the manufacturer. Furthermore, in all other cases Equation(1) is used to determine how much beer to order from onesrespective supplier.

It has been proposed that two parameters are used by the

game participants to determine their orders [9].( ) This represents the discrepancy between desiredand actual inventory ordered. This parameter isusually represented in the range. (0 1)

( ) This represents the fraction of the supply line takeninto account. This parameter is usually representedin the range. (0 1) If = 1, the participantfactors in all orders in the supply line or conversely,if = 0, the participant factors in no orders in thesupply line.

The combination of the adjustment parameters ( , ) cor-responds to a set of behaviours for a given participant. Theseparameters are fundamental to our analysis of the BDG. Theywill provide the main basis for our PSO and GA optimisationapproaches. The following two experimental scenarios willbe examined in this paper.

Scenario A: all participants in the supply chain usethe same ordering policy (the same and values).Therefore there are the only two parameters involved.We will refer to this scenario as S a .

Scenario B: each participant uses di ff erent orderingpolicies (di ff erent and values). Therefore there areeight parameters involved as there are four individualsin the supply chain and each of these has two strategiesparameters ( and ). We will refer to this scenario asS b .

C. The Beer Distribution Game Objective Function

In the BDG, all the participants wish to minimize theirtotal cost. This objective can be formulated as:

totalCost =m

i= 1

(n

j= 1

(2.0 BL ji ) + 0.5 INV ji ) (5)

In Equation (5), m is the total number of weeks and n isthe total number of participants, in our case n = 4. The costof holding inventory is 2 .0 while the cost of maintaining a

backlog is 0 .5. BL ji is the jth participants backlog at week

i; INV ji is the jth participants inventory at week i.

We use equation (5) as the objective function and employboth our PSO and GA to determine the optimal and forthe entire supply chain.

D. Particle Swarm Optimization

Particle swarm optimization (PSO) was rst described in1995 by Kennedy and Eberhart [10], [11]. This approachwas rst inspired by the phenomena of bird ocking andsh schooling. It has been successfully applied to manyproblems including the economic dispatch [12], and alsoreactive power and voltage control [13]. However, PSO hasnot been widely used in the case of supply chain problems.

For each particle {Initialise it in the search space (it must satisfyall the constraints);

}

Do {For each particle {

Evaluate its position according the objectivefunction;

If the current position is better than the previ ous perso nal bes t posi tion ( t) , t henupdate ;}

Choose the particle with the best position of allthe particles according the objective value asthe ;

For each particle {Calculate its velocity according the equation(6);

Update its position according the equation (7);

}}while it does not reach a stop criteria(i.e.

maximum iterations)

pBe s pBe st

gBe st

Check if its position is still in the search space. If not , r einit ia lis e a new par ticle t hat sat is fiesthe constraints;

Fig. 2. The Modied PSO

In PSO, there is a population of solutions referred toas particles. Particles y around the D-dimensional solutionspace, and are evaluated according to a tness criteria aftereach iteration. The i-th particles position is representedas X i = ( x i1, x i2, . . . , x id , . . . , x iD ). The ying velocity fora particle i is represented as V i = (vi1, vi2, . . . , vid , . . . , viD ).In every iteration, each particles ying velocity is updatedaccording to the following two positions. The rst one is theposition at which its best tness has achieved so far. Thisposition is a personal best position and recorded as pBest .The second position is the best position obtained so far byall particles in the population (or by its local neighborhood,in the local version of PSO). This position is a global bestposition (or local best) and called gBest (or lBest ). After


4/8

updating velocity, each particle updates its position based onits velocity. The Equations (6) and (7) formalise how eachparticles velocity and position is updated. The inertia weightw in (6) has been shown to provide improved performancein a number of applications [14] and the c1 and c2 aretwo positive constants, rand () and Rand () are two randomfunctions in the range [0 , 1].

vid = w vid + c1 rand () ( pBest x id )+ c2 Rand () (gBest x id ) (6)

x id = x id + vid (7)

E. The modied PSO

Particle Swarm Optimisation has been shown to be veryeff ective for many optimization problems. For the BDGproblem, however, the PSO needs to be modied to dealwith the game constraints.

Our modication involves reinitialising unfeasible parti-cles. Therefore in each iteration, once a particle fails to

satisfy the game constraints, it is reinitialised and positionedrandomly in the valid search space. Therefore, this particlewill again satisfy all the game constraints while also retainingits memory of (a) its personal best position to date pBest ,and (b) the location of the global best position gBest . Thismodication only concerns those particles which break thegame constraints, therefore all other particles are updated asnormal as in the traditional PSO.

This modication is similar to that proposed by Hu andEberhart [15] which was applied to constrained non-linearoptimisation problems. Our modication benets the overalldiversity of particles, and maintains a good search capabilitysince the total number of particles does not reduce. Anoverview of the modied PSO is shown in Fig. 2 1 .

F. Genetic Algorithms

Genetic Algorithms (GAs) are inspired by evolutionarybiology such as selection, crossover (also called recombina-tion) and mutation. GAs have been successfully applied tooptimisation problems like wire routing, scheduling, adaptivecontrol, supply chain management, etc [16].

An implementation of a GA begins with a population of chromosomes that encode candidate solutions to a problem.For each generation G, each chromosome is evaluated andassigned a tness value. After evaluation, Selection is ap-

plied to the population according their tness value and anintermediate population is created. The algorithm appliesrecombination and mutation to create the next populationin G + 1. Once evaluation, selection, recombination andmutation is completed, this new population is then used andthe process repeats over successive generations. This processcontinues for a predened period of time (G), or until themost optimal solutions are found.

Previous research has applied a GA to the BDG problem.ODonnell et al. [17] and Lu et al. [18] have successfullyused GAs to reduce bullwhip e ff ect for the BDG. Strozzi

1Please note that our modication is shown in italic bold.

et al. [4] also used a GA to optimise the BDG orderingpolicies for an entire supply chain. However, these papersexamined simple customer demand patterns. In this paper,we will examine a series of more complex demand patternsand investigate the implications of these demand patterns.Furthermore, we will use a real-coded GA which Wright hasclaimed should increase e fficiency and precision [19]. The

previous GA approaches mentioned here use binary encodedGAs which di ff ers from our approach. Our real-coded GAuses the Java Genetic Algorithms Package (JGAP) [20].

III. E xperimental Setup

Our experiments examine a number of customer demandpatterns. These demand patterns use a common mean valuewhich is set to 8. Then we can compare the participantsbehaviour when they use the best policies obtained fromPSO or GA. As has been outlined by Yan and Woo, wecan model a number of alternative customer demand patterns[21]. These demand patterns are also illustrated in Fig. 3, and

less than 150 data points are used for clarity purpose. Thesedemand patterns are as follows: One Step Demand (OSD): This demand changes only

once in the period of this simulation. It is typically setthat the customer demand is four cases of beer per week until week 4 and then steps to eight cases of beer perweek.

Stationary Demand (SD): The mean demand remainsconstant over time. The distribution of the demandconforms to a normal distribution. We use a mean of 8 and the standard deviation of 2.

Uniform Demand (UD): This demand uctuates ran-domly and is generated using a uniform distribution inthe range of [0 , 16].

Cyclic Demand (CD): This pattern varies cyclicallyover time, usually in response to some seasonal e ff ect of season or the standard business cycle. The mean value of demand changes periodically. The cycle of the demandis 50 weeks. In every cycle of the rst 25 weeks, weuse normal distribution with the mean of 10 and thedeviation of 2; In the following 25 weeks, we use normaldistribution with the mean of 6 and the deviation of 2.

As explained we will examine four distinct customerdemand patterns. The One Step Demand pattern is the mostcommonly used demand pattern when analysing the BDG.These other three demand patterns are more realistic andreect a closer representation of real market dynamics. Forexample, demand for goods such as rice normally followsthe SD pattern, demand for Christmas ornaments normallyfollows the CD pattern. New product demand normallyfollows the UD pattern.

As mentioned previously we will examine two distinct ex-perimental scenarios. Scenario A: Where all individuals usethe same and strategies. We will then examine ScenarioB: Where all individuals can use di ff erent and values.These two distinct scenarios will be used to compare theperformance of our PSO and GA solutions. These scenarios


5/8

0

2

4

6

8

10

12

14

16

0 20 40 60 80 100 120 140

C u s t o m e r

D e m a n

d

Time(week)(a) One Step Demand

0

2

4

6

8

10

12

14

16

0 20 40 60 80 100 120 140

C u s t o m e r

D e m a n

d

Time(week)(b) Stationary Demand

0

2

4

6

8

10

12

14

16

20 40 60 80 100 120 140

C u s t o m e r

D e m a n

d

Time(week)(c) Uniform Demand

0

2

4

6

8

10

12

14

16

0 20 40 60 80 100 120 140

C u s t o m e r

D e m a n

d

Time(week)(d) Cyclical Demand

Fig. 3. The Simulation Results of Demand Patterns

fundamentally change the optimisation problem involved. InScenario A, all individuals use the same ordering policy andtherefore only two parameters must be optimised. In ScenarioB, four individuals use di ff erent policies and therefore eightparameters need to be optimised.

In the case of all these experiments, optimality is consid-ered as the collective optimal solution for the entire supplychain. This di ff ers from using individual optimality for anindividual participant in the BDG. For example, the mostoptimal solution for the Distributor may well be di ff erent, if we were to assume he is acting in a selsh manner. However,in this case the collective performance of the supply chainamong all its participants is our fundamental concern. There-fore we dene the most optimal solution as those parameterswhich provide the most optimal performance across the entiresupply chain. This optimal policy is formally speciedthrough the minimum cost as calculated through Equation(5).

A. Beer Distribution Game Parameters

The parameters involved in our BDG are as follows. Thetotal simulation length was 150 weeks. The and valuesexist in the range of [0 , 1]. The value used was 0 .25(see Equation (3)). Delays equal 4 weeks and the desiredinventory levels, Q = 17.

B. PSO Parameters

Our PSO implementation is based on an existing PSOimplementation by Hu and Eberthart [15]. Therefore, weadopt the following parameters which are specied by Hu

and Eberthart [15]. The population size used was N = 50 andthe total number of iterations was 500. The inertia weightwas w = 0.5 + rand ()/ 2.0, while the maximum allowedvelocity V max was set equal to range of the dimension. Eachdimension was set to V max = 1, since the and valueswere limited to the range of [0 , 1]. Finally the constants c1and c2 were set to 1 .49445.

C. GA Parameters

As mentioned earlier, our real-coded GA was implementedusing JGAP [20]. The population size was set to N = 50 andthe total number of iterations (Generations) used was = 500.The crossover rate was 0 .8 and the mutation rate was 0 .01.

TABLE IComparison Between PSO and GA in Scenario A

Customer Method Worst Best Demand

OSD PSO 1854.0 1772.0 1834.6 27.5

GA 1896.0 1791.0 1847.0 28.6

SD PSO 2527.0 2459.0 2472.8 15.0GA 2765.0 2459.0 2549.19 61.5

UD PSO 7489.5 7488.0 7490.2 0.5GA 8041.5 7488.0 7605.4 147.7

CD PSO 13533.0 12015.0 12579.2 700.4GA 13702.5 12020.0 12946.9 727.2

IV. E xperimental Results

In this section we will provide a series of detailed ex-perimental results, involving the BDG, and our di ff erentoptimisation approaches.


6/8

TABLE IIComparison results with enumeration methods in Scenario A

Customer Enumeration PSO GA No. No.Demand Method PSO GA

OSD 1772.0 1772.0 1791.0 8 0SD 2459.0 2459.0 2459.0 17 4UD 7488.0 7488.0 7488.0 43 6CD 12015.0 12015.0 12020 18 0

TABLE IIIThe best and for all participants in Scenario A

Customer Method Demand

OSD PSO 0.8835 0.8478GA 0.8866 0.8470

SD PSO 0.2039 0 .8132GA 0.2018 0.8132

UD PSO 0.0296 0 .7274GA 0.0300 0.7252

CD PSO 0.0249 0 .1349GA 0.0246 0.1323

A. Simulation Results in Scenario A

The data in Table I shows the results obtained by ourmodied PSO and GA. These results are from 50 individualruns of our PSO and GA. Data is shown representing themaximum cost (Worst), the minimum cost (Best), the averagecost ( ) and standard deviation ( ) for 50 individual runs.Compare the results obtained by PSO and GA, we observethat the results from the PSO are marginally better than theresults from the GA. Only two best results are the same whenthe participants face the SD and UD. So it seems that PSO

provides a better solution for the BDG in S a .In order to clarify our results from S a , we compare theresults obtained from using our PSO and GA implementa-tions with the results we have obtained from a bruteforcesearch of the , strategy space. This corresponds to anenumeration method, and therefore provides us with theoptimal solutions for this problem. Here, we used a searchresolution of (0 .0001) in S a . This resolution is high enoughto obtain the global optimum due to the fact that there wereno further improvements when higher resolutions were used.However, the bruteforce method was not feasible in S b dueto the size of the search space. Table II shows the optimalresults from the enumeration method, the best results fromthe PSO and GA and the number of hits to the optimal in50 individual runs of the PSO and GA which we record as(No. PSO) and (No. GA). From Table II, we can see thatthe PSO nd the optimal results in all customer demandswhile GA does not nd all the optimal results. However,the results obtained by GA are very close to the optimalwhen the participants face the OSD and CD that GA doesnot obtain the optimal results. Furthermore, the PSO hits theoptimal much more times than the GA. It should be notedthat these di ff erences do not indicate a clear benet for usinga PSO over a GA approach. Instead they clearly show thediff erences in the specic case and for the specic parameters

used in our PSO and GA algorithms.Our results in Table I and II show that the most opti-

mal strategies di ff er signicantly when alternative customerdemand patterns are used. This means that the demandpatterns have a signicant e ff ect on the overall performanceof the supply chain. Total costs vary signicantly acrossthe various customer demand patterns. We believe this cor-

relates strongly with the degree of complexity involved ininterpreting each of these demand patterns. It is plausible toargue that where complexity is represented as C that C OS D

optimisation of the beer distribution game.pdf

Documents