quantitative method for business

11SlideSlide©© 2009 South2009 South--Western, a part of Western, a part of CengageCengage LearningLearning

Slides by

JohnLoucksSt. Edward’s

University


Chapter 17 Chapter 17 Markov ProcessesMarkov Processes

Transition ProbabilitiesTransition ProbabilitiesSteadySteady--State ProbabilitiesState ProbabilitiesAbsorbing StatesAbsorbing StatesTransition Matrix withTransition Matrix with SubmatricesSubmatricesFundamental MatrixFundamental Matrix


Markov ProcessesMarkov Processes

Markov process modelsMarkov process models are useful in studying the are useful in studying the evolution of systems over repeated trials or sequential evolution of systems over repeated trials or sequential time periods or stages.time periods or stages.•• the promotion of managers to various positions the promotion of managers to various positions

within an organizationwithin an organization•• the migration of people into and out of various the migration of people into and out of various

regions of the countryregions of the country•• the progression of students through the years of the progression of students through the years of

college, including eventually dropping out or college, including eventually dropping out or graduatinggraduating


Markov ProcessesMarkov Processes

Markov processes have been used to describe the Markov processes have been used to describe the probability that:probability that:•• a machine that is functioning in one period will a machine that is functioning in one period will

function or break down in the next period.function or break down in the next period.•• a consumer purchasing brand A in one period will a consumer purchasing brand A in one period will

purchase brand B in the next period.purchase brand B in the next period.


Example: Market Share AnalysisExample: Market Share Analysis

Suppose we are interested in analyzing the market Suppose we are interested in analyzing the market share and customer loyalty for Murphyshare and customer loyalty for Murphy’’ss

FoodlinerFoodliner

and and

AshleyAshley’’s Supermarket, the only two grocery stores in a s Supermarket, the only two grocery stores in a small town. We focus on the sequence of shopping trips small town. We focus on the sequence of shopping trips of one customer and assume that the customer makes of one customer and assume that the customer makes one shopping trip each week to either Murphyone shopping trip each week to either Murphy’’ss

FoodlinerFoodliner

or Ashleyor Ashley’’s Supermarket, but not both.s Supermarket, but not both.



We refer to the weekly periods or shopping trips We refer to the weekly periods or shopping trips as the as the trials of the processtrials of the process. . Thus, at each trial, the Thus, at each trial, the customer will shop at either Murphycustomer will shop at either Murphy’’ss

FoodlinerFoodliner

or or

AshleyAshley’’s Supermarket. The particular store selected in s Supermarket. The particular store selected in a given week is referred to as the a given week is referred to as the state of the systemstate of the system

in in

that period. Because the customer has two shopping that period. Because the customer has two shopping alternatives at each trial, we say the system has two alternatives at each trial, we say the system has two states. states.

State 1. The customer shops at MurphyState 1. The customer shops at Murphy’’ss

FoodlinerFoodliner..

State 2. The customer shops at AshleyState 2. The customer shops at Ashley’’s Supermarket.s Supermarket.



Suppose that, as part of a market research study, we Suppose that, as part of a market research study, we collect data from 100 shoppers over a 10collect data from 100 shoppers over a 10--week period. week period. In reviewing the data, suppose that we find that of all In reviewing the data, suppose that we find that of all customers who shopped at Murphycustomers who shopped at Murphy’’s in a given week, s in a given week, 90% shopped at Murphy90% shopped at Murphy’’s the following week while s the following week while 10% switched to Ashley10% switched to Ashley’’s. s.

Suppose that similar data for the customers who Suppose that similar data for the customers who shopped at Ashleyshopped at Ashley’’s in a given week show that 80% s in a given week show that 80% shopped at Ashleyshopped at Ashley’’s the following week while 20% s the following week while 20% switched to Murphyswitched to Murphy’’s.s.


Transition ProbabilitiesTransition Probabilities

Transition probabilitiesTransition probabilities govern the manner in which govern the manner in which the state of the system changes from one stage to the the state of the system changes from one stage to the next. These are often represented in a next. These are often represented in a transition transition matrixmatrix. .



A system has a A system has a finite Markov chainfinite Markov chain with with stationary stationary transition probabilitiestransition probabilities if:if:•• there are a finite number of states,there are a finite number of states,•• the transition probabilities remain constant from the transition probabilities remain constant from

stage to stage, andstage to stage, and•• the probability of the process being in a particular the probability of the process being in a particular

state at stage state at stage n+n+1 is completely determined by the 1 is completely determined by the state of the process at stage state of the process at stage nn

(and not the state at (and not the state at

stage stage nn--1). This is referred to as the 1). This is referred to as the memorymemory--less less propertyproperty..




ppijij

==

probability of making a transition from state probability of making a transition from state iiin a given period to state in a given period to state jj

in the next periodin the next period

pp1111

pp1212

0.9 0.10.9 0.1P P = = = =

pp2121

pp2222

0.2 0.80.2 0.8


State ProbabilitiesState Probabilities

P = .9(.9) = .81P = .9(.9) = .81

P = .9(.1) = .09P = .9(.1) = .09

P = .1(.2) = .02P = .1(.2) = .02MurphyMurphy’’ss

MurphyMurphy’’ss



AshleyAshley’’ss



.9.9

.9.9

.1.1

.2.2

.8.8

.1.1P = .1(.8) = .08P = .1(.8) = .08P = .1(.8) = .08




State Probabilities for Future PeriodsState Probabilities for Future PeriodsBeginning Initially with a MurphyBeginning Initially with a Murphy’’s Customers Customer

State Probabilities for Future PeriodsState Probabilities for Future PeriodsBeginning Initially with an AshleyBeginning Initially with an Ashley’’s Customers Customer


SteadySteady--State ProbabilitiesState Probabilities

The The state probabilitiesstate probabilities at any stage of the process can at any stage of the process can be recursively calculated by multiplying the initial be recursively calculated by multiplying the initial state probabilities by the state of the process at stage state probabilities by the state of the process at stage nn..The probability of the system being in a particular The probability of the system being in a particular state after a large number of stages is called a state after a large number of stages is called a steadysteady--state probabilitystate probability. .



Steady state probabilitiesSteady state probabilities can be found by solving the can be found by solving the system of equations system of equations ΠΠPP = = ΠΠ together with the together with the condition for probabilities that condition for probabilities that ΣπΣπii = 1. = 1. •• Matrix Matrix PP

is the transition probability matrixis the transition probability matrix

•• Vector Vector ΠΠ

is the vector of steady state probabilities.is the vector of steady state probabilities.



Let Let ππ11

= long run proportion of Murphy= long run proportion of Murphy’’s visitss visitsππ22

= long run proportion of Ashley= long run proportion of Ashley’’s visitss visitsThen, Then,

.9 .1 .9 .1 [[ππ11

ππ22

] = [] = [ππ11

ππ22

]].2 .8 .2 .8

continued . . .continued . . .




..99ππ11

+ + .2π.2π22

= = ππ11

(1)(1)..11ππ11

+ + .8π.8π22

= = ππ22

(2)(2)ππ11

+ + ππ22

= 1 (3)= 1 (3)

Substitute Substitute ππ22

= 1 = 1 --

ππ11

into (1) to give:into (1) to give:

ππ11

= = ..99ππ11

+ .2(1 + .2(1 --

ππ11

) = 2/3 = .667) = 2/3 = .667Substituting back into (3) gives:Substituting back into (3) gives:

ππ22

= 1/3 = .333.= 1/3 = .333.



Thus, if we have 1000 customers in the system, the Thus, if we have 1000 customers in the system, the Markov process model tells us that in the long run, Markov process model tells us that in the long run, with steadywith steady--state probabilities state probabilities ππ

11

==

.667 and.667 and

ππ

22

==

.333,.333, 667 customers will be Murphy667 customers will be Murphy’’s and 333 customers s and 333 customers

will be Ashleywill be Ashley’’s.s.2

31

32

31

3





SSuppose Ashleyuppose Ashley’’s Supermarket is contemplating an s Supermarket is contemplating an advertising campaign to attract more of Murphyadvertising campaign to attract more of Murphy’’s s customers to its store. Let us suppose further that customers to its store. Let us suppose further that AshleyAshley’’s believes this promotional strategy will increase s believes this promotional strategy will increase the probability of a Murphythe probability of a Murphy’’s customer switching to s customer switching to AshleyAshley’’s from 0.10 to 0.15. s from 0.10 to 0.15.



Revised Transition ProbabilitiesRevised Transition Probabilities


Revised SteadyRevised Steady--State ProbabilitiesState Probabilities

..8585ππ11

+ + .2.200ππ22

= = ππ11

(1)(1)..1515ππ11

+ + .8.800ππ22

= = ππ22

(2)(2)ππ11

+ + ππ22

= 1 (3)= 1 (3)

Substitute Substitute ππ22

= 1 = 1 --

ππ11


ππ11

= = ..8585ππ11

+ .20(1 + .20(1 --

ππ11

) = .57) = .57Substituting back into (3) gives:Substituting back into (3) gives:

ππ22

= .43.= .43.



Suppose that the total market consists of 6000 Suppose that the total market consists of 6000 customers per week. The new promotional strategy will customers per week. The new promotional strategy will increase the number of customers doing their weekly increase the number of customers doing their weekly shopping at Ashleyshopping at Ashley’’s from 2000 to 2580. s from 2000 to 2580.

If the average weekly profit per customer is $10, If the average weekly profit per customer is $10, the proposed promotional strategy can be expected to the proposed promotional strategy can be expected to increase Ashleyincrease Ashley’’s profits by $5800 per week. If the cost s profits by $5800 per week. If the cost of the promotional campaign is less than $5800 per of the promotional campaign is less than $5800 per week, Ashley should consider implementing the week, Ashley should consider implementing the strategy.strategy.



Example: NorthExample: North’’s Hardwares Hardware

Henry, a persistent salesman, calls North'sHenry, a persistent salesman, calls North'sHardware Store once a week hoping to speak withHardware Store once a week hoping to speak withthe store's buying agent, Shirley. If Shirley does notthe store's buying agent, Shirley. If Shirley does notaccept Henry's call this week, the probability sheaccept Henry's call this week, the probability shewill do the same next week is .35. On the other hand,will do the same next week is .35. On the other hand,if she accepts Henry's call this week, the probabilityif she accepts Henry's call this week, the probabilityshe will not do so next week is .20. she will not do so next week is .20.



Transition MatrixTransition Matrix

Next Week's CallNext Week's CallRefuses AcceptsRefuses Accepts

ThisThis

Refuses .35 Refuses .35 .65 .65 Week'sWeek's

CallCall

Accepts .20 Accepts .20 .80 .80


SteadySteady--State ProbabilitiesState ProbabilitiesQuestionQuestion

How many times per year can Henry expect to How many times per year can Henry expect to talk to Shirley?talk to Shirley?

AnswerAnswerTo find the expected number of accepted calls To find the expected number of accepted calls per year, find the longper year, find the long--run proportion run proportion (probability) of a call being accepted and (probability) of a call being accepted and multiply it by 52 weeks.multiply it by 52 weeks.




SteadySteady--State ProbabilitiesState ProbabilitiesAnswerAnswer

(continued)(continued)

Let Let ππ11

= long run proportion of refused calls= long run proportion of refused callsππ22

= long run proportion of accepted calls= long run proportion of accepted callsThen, Then,

.35 .65 .35 .65 [[ππ11

ππ22

] = [] = [ππ11

ππ22

]].20 .80 .20 .80




SteadySteady--State ProbabilitiesState ProbabilitiesAnswer (continued)Answer (continued)

.35π.35π11

+ + .20π.20π22

= = ππ11

(1)(1).65π.65π11

+ + .80π.80π22

= = ππ22

(2)(2)ππ11

+ + ππ22

= 1 (3)= 1 (3)

Solve for Solve for ππ11

and and ππ22

..





SteadySteady--State ProbabilitiesState ProbabilitiesAnswer (continued)Answer (continued)

Solving using equations (2) and (3). (Equation 1 is Solving using equations (2) and (3). (Equation 1 is redundant.) Substitute redundant.) Substitute ππ11

= 1 = 1 --

ππ22


.65(1 .65(1 --

ππ22

) + ) + .80π.80π22

= = ππ22

This gives This gives ππ22

= .76471. Substituting back into = .76471. Substituting back into equation (3) gives equation (3) gives ππ11

= .23529. = .23529. Thus the expected number of accepted calls per Thus the expected number of accepted calls per

year is:year is:(.76471)(52) = 39.76 or about 40(.76471)(52) = 39.76 or about 40


State ProbabilityState ProbabilityQuestionQuestion

What is the probability Shirley will accept What is the probability Shirley will accept Henry's next two calls if she does not accept his Henry's next two calls if she does not accept his call this week?call this week?




State ProbabilityState Probability

AnswerAnswerP = .35(.35) = .1225P = .35(.35) = .1225

P = .35(.65) = .2275P = .35(.65) = .2275

P = .65(.20) = .1300P = .65(.20) = .1300RefusesRefuses

RefusesRefuses

RefusesRefuses

RefusesRefuses

AcceptsAccepts

AcceptsAccepts

AcceptsAccepts

.35.35

.35.35

.65.65

.20.20

.80.80

.65.65P = .65(.80) = .5200P = .65(.80) = .5200P = .65(.80) = .5200


State ProbabilityState ProbabilityQuestionQuestion

What is the probability of Shirley accepting What is the probability of Shirley accepting exactly one of Henry's next two calls if she accepts exactly one of Henry's next two calls if she accepts his call this week?his call this week?




State ProbabilityState ProbabilityAnswerAnswer

The probability of exactly one of the next two calls The probability of exactly one of the next two calls being accepted if this week's call is accepted can be being accepted if this week's call is accepted can be found by adding the probabilities of (accept next week found by adding the probabilities of (accept next week and refuse the following week) and (refuse next week and refuse the following week) and (refuse next week and accept the following week) = and accept the following week) =

.13 + .16 = .29.13 + .16 = .29


Absorbing StatesAbsorbing States

An An absorbing stateabsorbing state is one in which the probability that is one in which the probability that the process remains in that state once it enters the the process remains in that state once it enters the state is 1.state is 1.If there is more than one absorbing state, then a If there is more than one absorbing state, then a steadysteady--state condition independent of initial state state condition independent of initial state conditions does not exist.conditions does not exist.


Transition Matrix with Transition Matrix with SubmatricesSubmatrices

If a Markov chain has both absorbing and If a Markov chain has both absorbing and nonabsorbingnonabsorbing states, the states may be rearranged so states, the states may be rearranged so that the transition matrix can be written as the that the transition matrix can be written as the following composition of four following composition of four submatricessubmatrices: : II,, 00, , RR, , and and QQ::

II 00

RR QQ


Transition Matrix with Transition Matrix with SubmatricesSubmatrices

II

= an identity matrix indicating one always = an identity matrix indicating one always remains in an absorbing state once it is reachedremains in an absorbing state once it is reached

00

= a zero matrix representing 0 probability of = a zero matrix representing 0 probability of transitioning from the absorbing states to the transitioning from the absorbing states to the nonabsorbingnonabsorbing

statesstates

RR

= the transition probabilities from the = the transition probabilities from the nonabsorbingnonabsorbing

states to the absorbing statesstates to the absorbing states

QQ

= the transition probabilities between the = the transition probabilities between the nonabsorbingnonabsorbing

statesstates


The vice president of personnel at The vice president of personnel at JetairJetair

AerospaceAerospacehas noticed that yearly shifts in personnel can behas noticed that yearly shifts in personnel can bemodeled by a Markov process. The transition matrix is:modeled by a Markov process. The transition matrix is:

Next YearNext YearSame Pos. Promotion Retire Quit FiredSame Pos. Promotion Retire Quit Fired

Current YearCurrent YearSame Position .55 .10 .05 Same Position .55 .10 .05 .20 .10.20 .10

Promotion .70 .20 0 Promotion .70 .20 0 .10 0.10 0Retire Retire 0 0 1 0 00 0 1 0 0

Quit Quit 0 0 0 1 00 0 0 1 0Fired Fired 0 0 0 0 10 0 0 0 1

Example: Example: JetairJetair

AerospaceAerospace



AerospaceAerospace

Transition MatrixTransition Matrix

Next YearNext YearRetire Quit Fired Same PromotionRetire Quit Fired Same Promotion

Current YearCurrent YearRetire Retire 1 0 0 0 01 0 0 0 0Quit Quit 0 1 0 0 00 1 0 0 0Fired Fired 0 0 1 0 00 0 1 0 0

Same Same .05 .20 .10 .55 .10.05 .20 .10 .55 .10Promotion 0 .10 0 .70 Promotion 0 .10 0 .70 .20.20


Fundamental MatrixFundamental Matrix

The The fundamental matrixfundamental matrix, , NN, is the inverse of the , is the inverse of the difference between the identity matrix and the difference between the identity matrix and the QQmatrix.matrix.

NN

= (= (II

--

Q Q ))--11


Fundamental MatrixFundamental Matrix

--1 1 --11

1 0 .55 .10 1 0 .55 .10 .45 .45 --.10 .10 N N = (= (II

--

Q Q ) ) --1 1 = = ==

0 1 .70 .20 0 1 .70 .20 --.70 .80 .70 .80


AerospaceAerospace



AerospaceAerospace

Fundamental MatrixFundamental MatrixThe determinant, The determinant, dd

= = aa1111

aa2222

--

aa2121

aa1212= (.45)(.80) = (.45)(.80) --

((--.70)(.70)(--.10) = .29 .10) = .29

Thus, Thus,

.80/.29 .10/.29 2.76 .34.80/.29 .10/.29 2.76 .34NN

= = = =

.70/.29 .45/.29 2.41 1.55.70/.29 .45/.29 2.41 1.55


NRNR

MatrixMatrix

The The NRNR matrixmatrix is the product of the fundamental (is the product of the fundamental (NN) ) matrix and the matrix and the R R matrix. matrix. It gives the probabilities of eventually moving from It gives the probabilities of eventually moving from each each nonabsorbingnonabsorbing state to each absorbing state. state to each absorbing state. Multiplying any vector of initial Multiplying any vector of initial nonabsorbingnonabsorbing state state probabilities by probabilities by NRNR gives the vector of probabilities gives the vector of probabilities for the process eventually reaching each of the for the process eventually reaching each of the absorbing states. Such computations enable absorbing states. Such computations enable economic analyses of systems and policies.economic analyses of systems and policies.



AerospaceAerospace

NR NR MatrixMatrixThe probabilities of eventually moving to the The probabilities of eventually moving to the

absorbing states from the absorbing states from the nonabsorbingnonabsorbing

states are states are given by:given by:

2.76 .34 2.76 .34 .05 .20 .10 .05 .20 .10 NRNR

= = xx

2.41 1.55 0 .10 0 2.41 1.55 0 .10 0



AerospaceAerospace

NR NR Matrix (continued)Matrix (continued)

Retire Quit FiredRetire Quit Fired

Same .14 .59 .28 Same .14 .59 .28 NRNR

= =

Promotion .12 .64 Promotion .12 .64 .24.24



AerospaceAerospace

Absorbing StatesAbsorbing StatesQuestionQuestion

What is the probability of someone who was just What is the probability of someone who was just promoted eventually retiring? . . . quitting? . . . promoted eventually retiring? . . . quitting? . . . being fired? being fired?



AerospaceAerospace

Absorbing States (continued)Absorbing States (continued)AnswerAnswer

The answers are given by the bottom row of the The answers are given by the bottom row of the NRNR

matrix. The answers are therefore:matrix. The answers are therefore:

Eventually Retiring = .12Eventually Retiring = .12Eventually Quitting = .64Eventually Quitting = .64Eventually Being Fired = .24Eventually Being Fired = .24


End of Chapter 17End of Chapter 17

quantitative method for business

Documents

south2009 south

market share analysisexample

market share analysiswe

market share analysissuppose

customer loyalty

weekly periods

various positions

murphyone shopping trip