markov chain models in economics, management and finance

Markov Chain Models in Economics,Management and Finance

Intensive Lecture Course

in High Economic School, Moscow Russia

Alexander S. Poznyak

Cinvestav, Mexico

April 2017

Alexander S. Poznyak (Cinvestav, Mexico) Markov Chain Models April 2017 1 / 42

4-th Lecture Day

Bargaining

(Negotiation)


BargainingDe�nition of barganing

De�nitionBargaining is a negotiation with the aim to arrive at an agreementbetween parties �xing obligations that each promises to carry out andestablishing the terms of a sale or exchange of goods or services.


BargainingIllustrative photo

Figure: People bargaining in a traditional Indonesian pasar malam (night market)in Rawasari, Central Jakarta.

So, the prase: " to reach a bargain with the antique dealer over some product"can be translated as "Zaklyuchit�sdelku s prodavtzom antikvariata naschetnekotorogo tovara".


BargainingMain features of bargaining

Bargaining or haggling (sporit�) is a type of negotiation ("peregovorov") inwhich the buyer and seller of a good or service debate the price and exactnature of a transaction.

Bargaining is an alternative pricing strategy to �xed prices. Optimally, if itcosts the retailer nothing to engage and allow bargaining, the retailer candivine the buyer�s willingness to spend.

Hanggling has largely disappeared in markets where the cost to haggleexceeds the gain to retailers.

The status quo of the classical bargaining problem (or equilibrium) is anagreement reached between all interested parties. It is clear that studyinghow individual parties make their decisions is insu¢ cient for predicting whatagreement will be reached.


BargainingBargaining as a game

Bargaining games refer to situations where two or more players must reachagreement regarding how to distribute an object or monetary amount. Eachplayer prefers to reach an agreement (favouring their interests) in thesegames, rather than abstain from doing so.J. Nash (The Bargaining Problem // Econometrica.1950. Vol. 18. P.155-162) de�nes a classical bargaining problem as being a set of jointallocations of utility, some of which correspond to what the players wouldobtain if they reach an agreement, and another that represents what theywould get if they failed to do so. The Nash bargaining solution is thebargaining solution that maximizes the product of an agent�s utilities on thebargaining set.A bargaining game for two players is de�ned as a tuple (F , x , ddis ) where

F =nJ1�x (1), x (2)

�, J2

�x (1), x (2)

�ois the set of possible joint utility

allocations (possible agreements), x 2 Xadm and ddis is the disagreementpoint (where participants return if they did not obtain agreement.


BargainingNash barganing solution

There exists several possible solutions depending on an additional axiomatics.

ProblemBy the Nash barganing solution (NBS) we will understand the solution of thebargaining game (F , x , dis ) for two players possessing the following 4 properties:

- Invariant to a¢ ne transformations or Invariant to equivalent utilityrepresentations.- Pareto optimality.- Independence of irrelevant alternatives (if from the begginning delete irrelevantalternatives, then the solution of the problem remain unchable).

- Symmetry, that is, the players have the same utility functions J1�x (1), x (2)

�= J2

�x (1), x (2)

�and such that J1

�x (2), x (1)

�= J2

�x (1), x (2)

�implying

that in the disagreement point they obtain the same pro�t (meaning that this

point is exactly in the "bisectrice"): i.e., J1�d (1)dis , d

(2)dis

�= J2

�d (1)dis , d

(2)dis

�.


BargainingThe Nash�s theorem

Theorem (J.Nash, 1950)

The Nash barganing solution (NBS) is the point x�Nash maximizing the function(the Nash product)

PNash�x (1), x (2)

�=h

J1�x (1), x (2)

��J1

�d (1), d (2)

�i hJ2�x (1), x (2)

�� J2

�d (1), d (2)

�ion the Pareto set XPareto :

XPareto :=nx (1), x (2) :

�x (1) (λ) , x (2) (λ)

�=

arg maxx (1)2X (1)adm , x

(2)2X (1)adm

hλJ1

�x (1), x (2)

�+ (1� λ) J2

�x (1), x (2)

�i)0 � λ � 1


BargainingThe NBS as a bargaining process

NBS can be explained as the result of the following bargaining process:

A current agreement, say�x (1), x (2)

�, is on the table.

One of the players, say player 2, can raise an objection. An objection is an

alternative agreement�x 0(1), x 0(2)

�, .Probably, the alternative agreement is

better for player 2 (J2�x 0(1), x 0(2)

�> J2

�x (1), x (2)

�) and worse for

player 1 (J1�x 0(1), x 0(2)

�> J1

�x (1), x (2)

�. Suppose that raising such an

objection has some probability of ending the negotiation. This probability pcan be selected by player 2 (e.g, by the amount of pressure he puts on player

1 to agree). The objection is e¤ective only if pJ2�x 0(1), x 0(2)

�>

J2�x (1), x (2)

�, i.e, player 2 prefers the alternative agreement

�x 0(1), x 0(2)

�with a chance p, over the original agreement

�x (1), x (2)

�for sure.


BargainingThe NBS as a bargaining process (continuation)

Player 1 can then raise a counter-objection by claiming that for him

pJ1�x (1), x (2)

�> J1

�x 0(1), x 0(2)

�. This means that player 1 prefers to

insist on the original agreement even if this might blow up the negotiation.Player 1 prefers the original agreement with a chance p, over the alternativeagreement for sure.

An agreement is NBS if, for every objection raised by one of the players,there is a counter-objection by the other player, that is,

pJ2�x 0(1), x 0(2)

�> J2

�x (1), x (2)

�J1�x 0(1), x 0(2)

�> pJ1

�x (1), x (2)

�It is an agreement which is robust to objections.


BargainingThe NBS as a bargaining process: illustration

Figure: Nash barganing solution.


BargainingThe generalization of the Nash�s theorem for the multi-participants case

TheoremThe Nash barganing solution (NBS) is the point

x�Nash=�x�(1)Nash, x

�(1)Nash, ..., x

�(1)Nash

�maximizing the function ( the Nash product)

x�Nash = maxx2XPareto

PNash (x) = arg maxλ2SN

PNash (x (λ))

PNash (x) =N∏i=1[Ji (x)�J i (d)] , SN :=

�0 � λi ,

N∑i=1

λi= 1�

on the Pareto set XPareto :

XPareto :=

(Ji (x (λ)) , i = 1,N : x (λ) = arg max

x (i )2X (i )adm ,

N∑i=1

λiJi (x)

)


BargainingAutomated bargaining

FactA bargaining situation is complex (but this takes place practically always) and�nding the Nash barganning equilibrium is di¢ cult using game theory.Evolutionary computation methods have been designed for automated bargaining,and demonstrated e¢ cient and e¤ective for approximating Nash equilibrium.


BargainingKalai�Smorodinsky bargaining solution

Independence of Irrelevant Alternatives can be substituted with a Resourcemonotonicity axiom (Kalai, Ehud & Smorodinsky, Meir (1975). "Othersolutions to Nash�s bargaining problem". Econometrica. 43 (3): 513�518).This leads to the socalled Kalai�Smorodinsky bargaining solution: it isthe point which maintains the ratios of maximal gains. In other words, ifplayer 1 could receive a maximum of

J�i := maxx (1)2X (1)adm , x

(2)2X (1)adm

Ji�x (1), x (2)

�(i = 1, 2)

with player 2�s help (and vice versa), this de�nes the Utopia Point(J��1 , J

��2 ), then the Kalai�Smorodinsky bargaining solution would yield the

point on the Pareto frontier such that

x�K�S =�x 2 XPareto :

J1(x (1),x (2))�J1(d (1),d (2))J2(x (1),x (2))�J2(d (1),d (2))

=J ��1 �J1(d (1),d (2))J ��2 �J2(d (1),d (2))

�Alexander S. Poznyak (Cinvestav, Mexico) Markov Chain Models April 2017 14 / 42

BargainingKalai�Smorodinsky bargaining solution: illustrative �gure

Figure: The Kalai-Smorodinskii solution.


BargainingKalai�Smorodinsky solution as an Optimization Problem

Generalization for N-participants

The Kalai-Smorodinsky rule, say x�K�S , chooses the maximum individuallyrational payo¤ pro�le at which each agent�s payo¤ gain from disagreement J�i hasthe same proportion to his aspiration ("utopia") payo¤�s gain J��i fromdisagreement J�i . Formally,

x�K�S = arg maxx (i )2X (i )adm ,i=1,N

�mini=1,N

Ji (x (1),...,x (N ))�J �iJ ��i �J �i

�


BargainingKalai�Smorodinsky solution as an Optimization Problem

Simplex-format of the KS-solution representation

Lemma

fα := mini=1,N

fi = minλ2SN

N∑i=1

λi fi , λi =

�1 if i = α0 if i 6= α

In view of that

x�K�S = arg maxx (i )2X (i )adm ,i=1,N

minλ2SN

N∑i=1

λiJi (x (1),...,x (N ))�J �i

J ��i �J �i


BargainingKalai�Smorodinsky solution as an Optimization Problem: numerical algorithm

λ (n+ 1) = πS

8>>><>>>:λ (n)� γλ

0BBB@J1(x (1)(n),...,x (N )(n))�J �1

J ��1 �J �1...

JN (x (1)(n),...,x (N )(n))�J �NJ ��N �J �N

1CCCA9>>>=>>>;

x (n+ 1) = πXadm

8>>><>>>:x (n) + γx

0BBB@rJ1(x (1)(n),...,x (N )(n))

J ��1 �J �1...

rJN (x (1)(n),...,x (N )(n))J ��N �J �N

1CCCA9>>>=>>>;


BargainingEgalitarian bargaining solution of Kalai: the photo of E.Kalai

Figure: Ehud Kalai (2007): born Dec. 7, 1942; nationality: United States andIsrael, �eld: Game theory and economics.


BargainingEgalitarian Kalai bargaining solution

The egalitarian bargaining solution (Kalai, Ehud (1977), "Proportional solutionsto bargaining situations: Intertemporal utility comparisons". Econometrica. 45(7): 1623�16307) is a third solution which drops the condition of scale invariancewhile including both the axiom of Independence of irrelevant alternatives, and theaxiom of resource monotonicity. It is the solution which attempts to grant equalgain to both parties.

Figure: Kalai egalitarian solution.Alexander S. Poznyak (Cinvestav, Mexico) Markov Chain Models April 2017 20 / 42

BargainingComparison table

NamePareto

OptimalitySymmetry

Scale

invariance

Irrelevant

independence

Resource

Monotonicity

Nash (1950)Maximizing the product

of surplus utilities

+ + + + NO

Kalai-Smor.(1975)

Equalizing the ratios

of maximal gains

+ + + NO +

Kalai (1977)Equal payoofs

on Pareto set

+ + NO + +

Table: Table of Comparison


BargainingExample of the Nash and Kalai�Smorodinsky bargaining solutions (1)

Example

Alice and George are businessmen and they have to choose between threeoptions a, b and c , that give them the following monetary revenues:24 a b c

Alice $60 $50 $30George $80 $110 $150

35They can also mix these options in arbitrary fractions("predpochteniya"=how often), e.g, they can choose option a for a fractionp1 of the time, option b for fraction p2, and option c for fraction p3, suchthat p1+p2+p3= 1. Hence, the set F of feasible agreements is the convexhull of a(60, 80) and b(50, 110) and c(30, 150):



The disagreement point is de�ned as the point of minimal utility: this is$30 for Alice and $80 for George, so d = (30, 80).For both Nash and KS solutions, we have to normalize the agents�utilitiesby subtracting the disagreement values, since we are only interested in thegains that the players can receive above this disagreement point. Hence, thenormalized values are:24 a b c

Alice $30 $20 $0George $0 $30 $70

35



Figure: Possible values of payo¤s.Alexander S. Poznyak (Cinvestav, Mexico) Markov Chain Models April 2017 24 / 42


The Nash bargaining solution maximizes the product of normalized utilities onthe Pareto set:

x�Nash = arg maxx2XParetoh

J1�x (1), x (2)

��J1

�d (1), d (2)

�i hJ2�x (1), x (2)

��J2

�d (1), d (2)

�i= arg max

p1,p2,p3[30p1+20p2+0p3] [0p1+30p2+70p3]

The maximum on the Pareto frontier is attained when p1= 0, p2 = 7/8 andp3 = 1/8 (i.e, option b is used 87.5% of the time and option c is used in theremaining time). The utility-gain of Alice is $17.5 and of George - $45.


BargainingExample: the Nash barganing solution (5)

Figure: Nash bargaining solution.


BargainingExample: the Kalai-Smorodinskii barganing solution (6)

The KS bargaining solution equalizes the relative gains - the gain of eachplayer relative to its maximum possible gain on the Pareto set satisfying therelation

30p1+20p2+0p30p1+30p2+70p3

=3070

Here, the maximum on the Pareto frontier attained when p1 = 0,p2= 21/26 and p3= 5/26. The utility-gain of Alice is $20 and of George- $42.Note that both solutions are Pareto-superior to the "random-dictatorial"solution - the solution that selects a dictator at random and lets him/herselects his/her best option. This solution is equivalent to letting x = 1/2,y = 0 and z = 1/2, which gives a utility of only $15 to Alice and $35 toGeorge.


BargainingExample: the Kalai-Smorodinskii barganing solution (7)

Figure: KS solution.


BargainingExample: the Kalai egalitarian barganing solution (8)

It is reached at the point (30, 30).

Figure: Kalai egalitarian solution.Alexander S. Poznyak (Cinvestav, Mexico) Markov Chain Models April 2017 29 / 42

Barganing using Markov chain modelsPayo¤ representation in Markov model bargaining problem

For l = 1,N the payo¤ Jl (x1, ..., xN ) in multi-participants bargainingcorresponds to

U l�c1, ..., cN

�:=

N1,M1

∑i1,k1

...NN ,MN

∑iN ,kN

W l(i1,k1,...,iN ,kN )

N∏l=1c l(il ,kl )

W l(i1,k1,...,iN ,kN )

=N1∑j1...NN∑jNU l(i1,j1,k1,...,iN ,jN ,kN )

N∏l=1

πl(il ,jl jkl )

c l :=hc l(il ,kl )

iil=1,Nl ;kl=1,Ml

is a matrix with elements

c l(il ,kl ) = dl(kl jil )P

l�s l=s(il )

�


Barganing using Markov chain modelsAdmissible sets representation in Markov model bargaining problem

The admissible sets�x (l)2 X (l)adm

�in Markov model bargaining problem

corresponds to

c l2 C ladm :=

8>>>>>>><>>>>>>>:

kc (l)il ,kl k: c (l)il ,kl� 0,Nl∑il=1

Ml

∑kl=1

c (l)il ,kl= 1

Ml

∑kl=1

c (l)il ,kl=Nl∑il=1

Ml

∑kl=1

π(l)il ,klc ln(i l , k l )

Nl ,Ml

∑il ,kl

ql(jl jil ,kl )cl(il ,kl )

= 0 (for continuos-time MCh.)

9>>>>>>>=>>>>>>>;P l�s l=s(il )

�=

Ml

∑klc l(il ,kl ), d l(kl jil ) =

c l(il ,kl )Ml∑kl

c l(il ,kl )

since in the ergodic caseMl

∑klc l(il ,kl ) > 0 for all l = 1,N . The individual aim of

each player is U l (c l ) ! maxc l2C ladm

.Alexander S. Poznyak (Cinvestav, Mexico) Markov Chain Models April 2017 31 / 42

Barganing using Markov chain modelsTwo-person KS-bargaining problem: example (1)

N1 = N2 = 5, and the number of actions M1 = M2 = 2

U l = U l�c1, c2

�:=

N1,M1

∑i1,k1

N2,M2

∑i2,k2

W l(i1,k1,i2,k2)

c1(i1,k1)c2(i2,k2)

, l = 1, 2

We are interested in the point (U1,U2) which is better than the disagreementpoint

�U1�,U2�

�, that is,

U l > U l�


Barganing using Markov chain modelsTwo-person KS-bargaining problem: example (2)

U1(i ,j j1)=

26666410 8 13 7 611 19 6 8 109 7 13 19 514 9 15 12 1612 14 9 8 10

377775 , U1(i ,j j2)=

26666412 9 7 10 1516 0 9 14 618 10 16 9 412 16 9 8 1311 9 13 17 10

377775

U2(i ,j j1)=

2666647 9 12 6 1018 12 7 9 105 14 8 11 1610 13 8 14 1016 9 12 10 8

377775 , U2(i ,j j2)=

2666645 17 6 9 118 9 14 12 810 13 9 1 1812 18 15 9 417 13 9 5 6

377775


Barganing using Markov chain modelsTwo-person KS-bargaining problem: transition rate matrices (1)

The transition rate matrices for each player are de�ned as follows

q1(i ,j j1)=

266664�0.5366 0.0888 0.0611 0.1893 0.14090.0416 �0.5689 0.0588 0.1331 0.09420.2358 0.1929 �0.3784 0.1878 0.20840.0942 0.1929 0.1244 �0.5963 0.05700.1649 0.0942 0.1342 0.0861 �0.5005

377775

q1(i ,j j2)=

266664�0.8049 0.1333 0.0916 0.2839 0.21140.0624 �0.8534 0.0881 0.1996 0.14120.3538 0.2894 �0.5677 0.2817 0.31270.1414 0.2894 0.1867 �0.8944 0.08550.2474 0.1414 0.2012 0.1292 �0.7508

377775


Barganing using Markov chain modelsTwo-person KS-bargaining problem: transition rate matrices (2)

q2(i ,j j1)=

266664�0.5434 0.1635 0.1649 0.2965 0.08710.1090 �0.9561 0.2965 0.0878 0.22640.1005 0.1996 �1.0011 0.1648 0.08770.1641 0.3679 0.3194 �0.8518 0.15710.1697 0.2251 0.2202 0.3027 �0.5585

377775

q2(i ,j j2)=

266664�0.2329 0.0701 0.0707 0.1271 0.03740.0467 �0.4097 0.1271 0.0376 0.09700.0431 0.0856 �0.4290 0.0706 0.03760.0703 0.1577 0.1369 �0.3651 0.06730.0727 0.0964 0.0944 0.1298 �0.2393

377775Π(t j d) = Π(0)eQ (d )t= eQt :=

∞

∑t=0

tnQ n(d )n! !

t!∞Π(d)


Barganing using Markov chain modelsTwo-person KS-bargaining problem: disagreement point as Nash equilibrium (1)

c1�=

2666640.1214 0.07880.1214 0.07880.1159 0.08250.1265 0.07540.1185 0.0807

377775 c2�=

2666640.1141 0.08170.1091 0.09350.1091 0.09360.1095 0.09250.1134 0.0836

377775


Barganing using Markov chain modelsTwo-person KS-bargaining problem: disagreement point as Nash equilibrium (2)

The mixed strategies obtained for the players are as follows

d1� =

2666640.6065 0.39350.6062 0.39380.5842 0.41580.6264 0.37360.5948 0.4052

377775 d2� =

2666640.5828 0.41720.5384 0.46160.5382 0.46180.5423 0.45770.5757 0.4243

377775With the strategies calculated the resulting utilities in the disagreement point foreach player ψl�(c1, c2) are as follows:

J�1 := J�1 (c1�, c2�) = 126.8675, J�2 := J�2 (c

1�, c2�) = 121.5986


Barganing using Markov chain modelsTwo-person KS-bargaining problem: KS-solution (1)

Given γx = γλ = 0.7 we obtain the convergence

to the point

c1K�S =

2666640.1046 0.09560.1045 0.09570.0987 0.09960.1100 0.09210.1015 0.0977

377775 , c2K�S =

2666640.1002 0.09600.0956 0.10680.0955 0.10690.0960 0.10580.0995 0.0977

377775Alexander S. Poznyak (Cinvestav, Mexico) Markov Chain Models April 2017 38 / 42

Barganing using Markov chain modelsTwo-person KS-bargaining problem: KS-solution (2)

The players strategies are

d1K�S =

2666640.5224 0.47760.5220 0.47800.4977 0.50230.5443 0.45570.5094 0.4906

377775 , d2K�S =

2666640.5106 0.48940.4722 0.52780.4720 0.52800.4756 0.52440.5045 0.4955

377775and the resulting utilities in the bargaining KS-solution for each player are asfollows:

J1,K�S := J1(c1, c2) = 128.1883, J2,K�S := J2(c1, c2) = 122.1602

The utilities on the utopia point for the bargaining problem are for each player asfollows:

J��1 = 135.7439, J��2 = 126.0856


Barganing using Markov chain modelsTwo-person KS-bargaining problem: illustrating �gure

Figure: KS-solution for Markov Chain Model.


ConclusionWhich topics we have discussed today?

4-rd Lecture Day: Bargaining (Negotiation)� De�nition of Bargaining and the main properties of bargaining.� Bargaining as a game.� Nash�s bargaining solution.� Generalization for N-participants.� Kalai-Smorodinsky bargaining solution.� Expression of KS solution as a result of Optimization Problem.� Egalitarian Kalai-solution.� Comparison table.� Bargaining for Markov Chain models.� Illustrative example.


ConclusionNext lecture

Next Lecture Day:

Partially Observable Markov Chain Models and Tra¢ c OptimizationProblem

Thank you for your attention! See you soon!


markov chain models in economics, management and finance

Documents