© cognitive radio technologies, 2007 1 supermodular games

1 © Cognitive Radio Technologies, 2007

Supermodular Games


Terminology


Partial Ordering

A relation is R is an order on a set S if the following three properties are satisfied for all x, y, z S: reflexive (x R x) transitive (x R y R z x R z) antisymmetric (x R y R x x = y)

The order is only partial order if there are some elements a, b S such that neither a R b nor b R a.

A set S taken together with a partial order is a partially-ordered set.


Our relation of interest

Let x and y denote two vectors in KR

Let x y if xk yk for all k = 1,2,…,K

Let x > y if x y and there exists some k such that xk > yk

Example:x = (1,0,0,3), y = (1,2,2,3) y

xy > x

x = (1,0,4,3), y = (1,2,2,3) No relation


Two More Relations

“meet of x and y” 1 1 2 2(min , ,min , , ,min , )K Kx y x y x y x y

“join of x and y” 1 1 2 2{max , ,max , , ,max , }K Kx y x y x y x y

Example:

x = {1,0,0,3}, y = {1,2,2,3}

x = {1,0,4,3}, y = {1,2,2,3}

1,0,0,3x y

The meet of x and y is the infimum of x and y

The join x and y is the supremum of x and y

1,2,2,3x y

1,0,2,3x y 1,2,4,3x y

Note: if y x then x y x x y y


Sublattice

Consider Si to be a subset (maybe convex) of . Form S as imRm

i N iS S R where1

n

kk

m m

Definition Sublattice

A set is a sublattice if it is a partially ordered () subset of and if the operations and are closed on S. (i.e. if s, s* S then s s* S and s s* S)

mR

S = {(0,0), (0,0.5), (0.5,0), (1,0), (0,1)}Sublattice?

noS = {(0,0), (0,1), (1,0), (1,1)} yes

Sublattice property – Every bounded sublattice has a greatest and least element.


Increasing Differences

Definition

ui(si, s-i) has increasing differences in (si, s-i) if, for all

, and ,i i i i i is s S s s S such that andi i i is s s s

, , , ,i i i i i i i i i i i iu s s u s s u s s u s s

In other words, an increase in the strategies of i’s rivals increasesthe value of playing a high strategy for player i.


Supermodular Function

, , , , ,i i i i i i i i i i i i i i i i iu s s u s s u s s s u s s s s s S ui(si, s-i) is supermodular in si if for each s-i

Definition

Note if Si is single-dimensional, this is satisfied with equality.


Supermodular games


Equivalent Formulations

A game is supermodular if

2

0 , ,i

i j

u si j N s S

s s

A game is supermodular if Si is a sublattice of mR for all i and

,i i i iu s s u s s u s u s s s S


Best Response Properties

Stated without proof


Nash Equilibrium Existence

(Tarski) If A is a non-empty, compact sub-lattice of m and f : A A is non-decreasing, then f has a fixed point in A.

Note upper-semi-continuity

Would have to jump down across y=x, but that violatesUSC.

i N iBR a BR a

has a fixed point (NE)A

A


NE Properties

(Topkis) A supermodular game for which each Ai is compact and each ui is u.s.c. in ai for each a-I, then the set of pure strategy NE is non-empty and contains greatest and least elements

and , respectivelya a

(Vives) Further the set of NE form a sub-lattice which is nonempty and complete.


NE Uniqueness


More NE Properties

Definition Best Response DynamicAt each stage, one player iN is permitted to deviate from ai to

some randomly selected action bi Ai iff

, ,i i i i i i i i iu b a u c a c b A and ,i i i iu b a u a

(Milgrom and Roberts) A best response dynamic played on a supermodular game with compact action spaces and u.s.c. objective functions converges to a region bounded the greatest and least elements in the set of NE.

If the NE is unique, then the best response dynamic converges to theNE.


Convergence of Adaptive Dynamics

The corollaries to Theorem 8 in [Milgrom_90] show that a smooth supermodular game following an adaptive dynamic process with any timing converges to a region bounded by the Nash equilibrium lattice .


Ad-hoc power control

Network description Each radio attempts to

achieve a target SINR at the receiving end of its link.

System objective is ensuring every radio achieves its target SINR

Gateway

ClusterHead

ClusterHead

Gateway

ClusterHead

ClusterHead

2ˆk kk N

J

p


Generalized repeated gamestage game

Players – N Actions – Utility function

Action space formulation 2

ˆj j ju o

max0,j jP p

2

10 10\

ˆ 10log 10logj j jj j kj k jk N j

u p g p g p N

gjk fraction of power transmitted by j that can’t be removed by receiving end of radio j’s linkNj noise power at receiving end of radio j’s link


Model identification & analysis

Supermodular game– Action space is a lattice– Implications

NE exists Best response converges Stable if discrete action space

Best response is also standard– Unique NE– Solvable (see prelim report)– Stable (pseudo-contraction) for infinite action

spaces

2

\

2000

ln 20

j kj

j kj kj k j

k N j

u p g

p pp g p N

ˆˆ jk

j jj

B p

p


Validation

Noiseless Best Response Noisy Best Response

Implies all radios achieved target SINR


Comments on Designing Networks with Supermodular Games

Scales well– Sum of supermodular functions is a supermodular function– Add additional action types, e.g., power, frequency,

routing,..., as long as action space remains a lattice and utilities are supermodular

Says nothing about desirability or stability of equilibria

Convergence is sensitive to the specific decision rule and the ability of the radios to implement it


Potential Games

time

(

)

Existence of a function (called the potential function, V), that reflects the change in utility seen by a unilaterally deviating player.

Cognitive radio interpretation:– Every time a cognitive radio

unilaterally adapts in a way that furthers its own goal, some real-valued function increases.


Exact Potential Games

Definitions, Existence, Basic Properties Examples Path Properties


Exact Potential Games

, , , , , ,i i i i i i i i i i i i i i iP a a P b a u a a u b a a b A a A

Definition Exact Potential Game

A normal form game whose objective functions are structured such that there exists some function P: A which satisfies the following property for all players:

In other words it must be possible to construct a single-dimensionalfunction whose change in value is exactly equal to the change in value of the deviating player.


Example Potential Game (1/2)

a1

b1

a2 b2

1,1 0, 0

3, 30, 0

Coordination Game

1 2

1 2

1 2

1 2

1 ,

0 ,

0 ,

3 ,

a a a

a a bV a

a b a

a b b

u1(a1,a2) - u1(b1,a2) = 1 = V(a1,a2) - V(b1,a2)

u2(a1,a2) – u2(a1,b2) = 1 = V(a1,a2) - V(a1,b2)

u1(b1,b2) - u1(a1,b2) = 3 = V(b1,b2) - V(a1,b2)

u2(b1,b2) – u2(b1,a2) = 3 = V(b1,b2) - V(b1,a2)

Note: V is not unique.Consider V’ = V + c where c is a constant.

Also note the relationbetween CG Prop. 2 and V


Example Potential Game (2/2)

a1

b1

a2 b2

4,2 -1, 1

2, 13,-2

1 2

1 2

1 2

1 2

1 ,

0 ,

0 ,

3 ,

a a a

a a bV a

a b a

a b b

u1(a1,a2) - u1(b1,a2) = 1 = V(a1,a2) - V(b1,a2)

u2(a1,a2) – u2(a1,b2) = 1 = V(a1,a2) - V(a1,b2)

u1(b1,b2) - u1(a1,b2) = 3 = V(b1,b2) - V(a1,b2)

u2(b1,b2) – u2(b1,a2) = 3 = V(b1,b2) - V(b1,a2)

The Same Potential!!The Same NE!

Coordination Game(In Equilibriums)


Comments on Second Example

a1

b1

a2 b2

1,1 0, 0

3, 30, 0

Coordination Game

a1

b1

a2 b2

4,2 -1, 1

2, 13,-2

Second Game

a1

b1

a2 b2

3,1 -1, 1

-1,-23,-2

Dummy Game

As we shall see, this is a property of all exact potential games.

Also a potential function for an exact potential game is always equal to the characteristic function (plus a constant) of its constituent coordination game.


EPG Property 1

A game G = <N, {Ai}iN , {ui}iN> is an exact potential game iff there exist functions {ci}iN and {di}iN such that

(Voorneveld)

•ui = ci + di

•<N, {Ai}iN , {ci}iN> is a coordination game•<N, {Ai}iN , {di}iN> is a dummy game

Outline of proof: if: The characteristic function of the coordination game is an exact potential function of GOnly if: Let P be an exact potential of G. Clearly P forms a coordination game. Now consider a game with objective fcns given by ui – P. As the value of deviating in this game is now 0 at all points, this is a dummy game.


EPG Property 2

The NE of an exact potential game are coincident with the NE ofits constituent coordination game.

Outline of Proof

Any unilateral deviation in a dummy game yields the same payoff. Adding a dummy game D to another game G preserves G’s NE. All exact potential games can be expressedAs the sum of a coordination game and a dummy game (EPG Property 1). Therefore the NE of the potential game mustbe the same as the NE of the coordination game.


EPG Property 3

For an EPG, the maximizers of the EPF are NE of the EPG.

Outline of Proof

(Voorneveld)

The NE of an EPG are the NE of its coordination game (CG).By CG Property 3, the maximizers of its characteristic functions(V) are NE.All EPF can be expressed as V constant c.Since the addition of the constant does not change whichtuples yield maximum payoffs, the maximizers of the EPF are coincident with the maximizers of V, thus coincident with theNE of the CG, thus coincident with the NE of the EPG.


EPG Property 4

(Voorneveld)Let the EPG be finite (finite action space, finite player set), then theEPG has at least one pure-strategy NE.

Note these conditions mean that the EPF must have at least onemaximum. By EPG Property 4, this must be a NE.

Outline of Proof


Continuous Action Sets (1/2)

i

i i

P u

a a

22 2ji

i j i j i j

uP u

a a a a a a

If objective functions are twice differentiable then a game is a EPG iff

Let G be a game in which the strategy sets are closed intervals of . Suppose the objective functions are continuously differentiable. A function P is a potential iff P is continuously differentiable and

for every i N

for every i, j N

EPG Property 5 (Shapley)

EPG Property 6 (Shapley)


Continuous Action Sets (2/2)

EPG Properties 1-4 also hold for continuous closed action sets.

Proofs follow in exactly the same manner.


Vector Operations

Consider the set of EPG {EPG1, EPG2,…,EPGK}, with player setN and action space A, and objective functions , andpotential functions . Form a new game, G, with player set N, action space A, and objective functions given by

1 2, , , Ki i iu u u

1 2, , , KP P P

1 1 2 2G K Ki i i iu u u u c . Then G is an EPG with an EPF

given by 1 1 2 2 K KP P P P

Note: this means that the set of EPG formed from a particular N andA form a vector topological space (closed under addition and scalarmultiplication).


Common EPG


Exact Potential Game Forms

Many exact potential games can be recognized by the form of the utility function


Coordination – Dummy Game

As previously stated, all EPG are formed from the sum of a coordination game and a dummy game so this is only here for completeness.

Consider a game G = <N, (Ai)iN , {ui}iN> such thatui = ci + di where <N, (Ai)iN , {ci}iN> is a coordination game with characteristic function V(a) and <N, (Ai)iN , {di}iN> is a dummy game.

This game has an EPF given by V(a)


Bilateral Symmetric Interaction Game

Introduced in Ui

, ,ij i j ji j iw a a w a a:ij i jw A A

,i j i ja a A A

\

,i ij i j i ij N i

u a w a a h a

1

1

,i

ij i j i ii N j i N

P a w a a h a

A strategic form game where each player’s objective function isa sum of bilateral symmetric interaction (BSI) terms. A BSI term

such that

. The objective function is expressed as

for every

An EPF for this game is given by


Self-Motivated Game

Note this is not really a game (no interaction), but it is often encountered as a component of more complex games.

i i iu a h a

i ii N

P a h a

A strategic form game where each player’s objective function isa function solely of their own action, i.e.

This has an EPF given by


Cournot Oligopoly (1/2)

i i k ik N

u b b B b cb i N

22

1ji

i j j i

uui j N

b b b b

Cournot oligopoly characterized by real interval action sets and objective function given by

Note that

So a potential exists


Cournot Oligopoly (2/2)

2

\i i k i i i

k N i

u b b b Bb b cb i N

Now rewrite the objective function as

Note that this is just a BSI game. So a potential can be written as

1

2

1

i

i k i i ii N j i N

P a b b Bb b cb


Prisoners’ Dilemma

a1

b1

a2 b2

w, w x, y

z, zy, x

a1

b1

a2 b2

x-z

0

a1

b1

a2 b2

z-x+y, z-x+y z, z-x+y

z, zz-x+y, zx-z

x-z+w-y


Ordinal Potential Games

Definitions, Existence, Properties Examples Applications


Ordinal Potential Games

, , , , , ,i i i i i i i i i i i i i i iP a a P b a u a a u b a a b A a A

Definition Ordinal Potential Game (OPG)

A normal form game whose objective functions are structured such that there exists some function P: A which satisfies the following property for all players:

In other words it must be possible to construct a single-dimensionalfunction where the sign of the change in value is the same as the sign of the change in value of the deviating player.

sgn , , sgn , , , ,i i i i i i i i i i i i i i iP a a P b a u a a u b a a b A a A

Note that an EPG also satisfies this definition.


Example Ordinal Potential Game

Not a Coordination Game

1 2

1 2

1 2

1 2

0 ,

3 ,

1 ,

2 ,

a a a

a a bP a

a b a

a b b

sgn(u1(a1,a2) - u1(b1,a2)) = - = sgn(P(a1,a2) - P(b1,a2))

sgn(u2(a1,a2) – u2(a1,b2)) = - = sgn(P(a1,a2) - P(a1,b2))

sgn(u1(b1,b2) - u1(a1,b2)) = - = sgn(P(b1,b2) - P(a1,b2))

sgn(u2(b1,b2) – u2(b1,a2)) = + = sgn(P(b1,b2) - P(b1,a2))

Note: P is not unique.Consider P’ = c2P + c1

a1

b1

a2 b2

0,0 1,1

0,12,0


Properties shared with EPG

For an OPG, the maximizers of the OPF are NE of the OPG.

An OPG has at least one pure-strategy NE.

An finite OPG has FIP.

An OPG with continuous bounded action sets has AFIP.

A repeated game with the same OPG stage also converges witha better response dynamic.

(Shapley)


Cycles

1 1

0, 0

k m mi m i mm

I u u a u a

Consider a cycle , the sum of the changes in value seen by the deviating players in an OPG is not always 0.

a1

b1

a2 b2

0,0 1,1

0,12,0

= ((a1, a2), (b1, a2), (b1, b2), (a1, b2), (a1, a2))

I(, u) = 2 + 1 + 1 – 1 = 3


Weak Improvement Cycles

Non-deteriorating pathA path is an non-deteriorating path if for all k 1

1k ki k i ku a u a

Weak improvement cycleA finite non-deteriorating path = (a0, a1,…,ak) where ak = a0

No known simple necessary and sufficient condition like the secondderivative condition of EPG.

(Voorneveld)All OPG lack weak improvement cycles.


Properties not shared with an EPG

The set of OPG is not a vector space.

a1

b1

a2 b2

0,0 1,1

0,12,0

a1

b1

a2 b2

1,2 1,0

0,10,0

a1

b1

a2 b2

1,2 2,1

0,22,0

a1

b1

a2 b2

0 3

21

a1

b1

a2 b2

3 2

10

Improvement Cycle= ((a1, a2), (b1, a2), (b1, b2), (a1, b2))

Still closed under scalar multiplication though.


Ordinal Transformations


Ordinal Transformation

Definition Ordinal TransformationsAn ordinal transformation is a one-to-one mapping of the utility

functions {ui} to a new set of utility functions {ui’} in such a way that the ordinality of the preference and indifference relationships for all players are maintained. This can be restated as.

' ', , , , , ,i i i i i i i i i i i iu a a u b a u a a u b a a b A i N

' ', , , , , ,i i i i i i i i i i i iu a a u b a u a a u b a a b A i N


Ordinal Transformation

Example

a1

b1

a2 b2

0,0 1,1

0,12,0

a1

b1

a2 b2

-1,3 0,8

-1,87,3

u1(a1, a2) = u1(b1, b2) < u1(a1, b2) < u1(b1, a2)

u2(a1, a2) = u2(b1, a2) < u2(a1, b2) < u2(b1, a2)

u1(a1, a2) = u1(b1, b2) < u1(a1, b2) < u1(b1, a2)

u2(a1, a2) = u2(b1, a2) < u2(a1, b2) < u2(b1, a2)

a1

b1

a2 b2

0 3

21

Note that both games have an OPF, and bothadmit the same OPF


Common Ordinal Transformations

Monotonic Transformations

Linear

Logarithmic

' , 0, 0i iu au c a c

' log , 0, 0i iu au c a c


OT Property 1

An ordinal transformation of an ordinal potential game is itself an ordinal potential game.

Since an OT preserves the ordering of all preference relationships,if the original game lacks weak improvement cycles, then thetransformed game must also lack weak improvement cycles.

Proof Outline


OT Property 2

If an ordinal transformation of a game yields an ordinal potential game, then the original game must also be an ordinal potential game.

Since an OT preserves the ordering of all preference relationships, ifthe transformed game lacks weak improvement cycles, then so mustthe original game.

Proof Outline


OT Property 3

An ordinal transformation of an exact potential game is an ordinal potential game (will remain an EPG if OT is linear).

Proof OutlineAn EPG is also an OPG. Apply the OT property 1.


Generalized Ordinal Potential Games


Optimality

If ui are designed so that maximizers of V are coincident with your design objective function, then NE are also optimal.

(*) Can also introduce cost function to utilities to move NE.

In theory, can make any action tuple the NE

– May introduce additional NE– For complicated NC, might as well

completely redesign ui

* *

0i i

V a NC a

a a

*i iu a u a NC a

V

a


Path Properties


EPG Property 7

(Shapley)

Consider a game with finite action sets. The following are equivalent:

(1) The game is an EPG.(2) I(, u) = 0 for every finite closed path (3) I(, u) = 0 for every finite simple closed path (4) I(, u) = 0 for every finite simple closed path of length 4

where 1 1

0,

k m mi m i mm

I u u a u a

and i(m) is the unique deviating player at step m


EPG Property 8

(Shapley)

Every finite EPG has FIP.

Finite Improvement Path Property (FIP) All improvement paths in the game are finite.

Proof OutlineFor any = (a0, a1,…), V(a0) < V(a1) < …Since A is finite, must be finite

Note that all NE of a finite EPG must be a terminal point in a finiteimprovement path.


- Improvement Path

Consider a game with continuous, bounded action sets and >0.A path is an -improvement path if for all k 1

1k ki k i ku a u a

Approximate Finite Improvement Path Property (AFIP)If for every >0, every -improvement path is finite.

EPG Property 9

Every EPG with continuous bounded action sets has AFIP.

EPG Property 10Every EPG with bounded action sets possesses an -equilibrium point. (A point from which there are no improvement deviations greater than or equal to )


Convergence Properties of EPG

EPG Property 11

Because they satisfy FIP, all repeated games where each stage is the same finite EPG and all players are myopic converge to a NE of the EPG when play follows a better response dynamic.

(Shapley)

EPG Property 12

Because they satisfy AFIP, all repeated games where each stage is the same bounded infinite EPG and players are myopic convergeto a -equilibrium point a when play follows a better response dynamic.

(Shapley)


Implications of Monotonicity

Monotonicity implies – Existence of steady-states (maximizers of V)– Convergence to maximizers of V for numerous combinations of

decision timings decision rules – all self-interested adaptations Does not mean that that we get good performance

– Only if V is a function we want to maximize


Token Economies

Pairs of cognitive radios exchange tokens for services rendered or bandwidth rented

Example: – Primary users leasing spectrum to secondary users

D. Grandblaise, K. Moessner, G. Vivier and R. Tafazolli, “Credit Token based Rental Protocol for Dynamic Channel Allocation,” CrownCom06.

– Node participation in peer-to-peer networks T. Moreton, “Trading in Trust, Tokens, and Stamps,” Workshop on

the Economics of Peer-to-Peer Systems, Berkeley, CA June 2003. Why it works – it’s a potential game when there’s no

externality to the trade– Ordinal potential function given by sum of utilities


Lyapunov Stability


Direct Method for Continuous Systems

(from potential game chapter/report)


Application of Continuous Direct Method

[Slade1994] – directional updates

For differentiable exact potential games, negative of potential function is the Lyapunov function

This is more concisely shown in [Anderson2004]

[Slade94] M. E. Slade, “What Does an Oligopoly Maximize?” Journal of Industrial Economics 58, pp. 45–61. 1994.

[Anderson2004] S. Anderson, J. Goeree, C. Holt, “Noisy Directional Learning and the Logit Equilibrium,” Scandinavian Journal of Economics, (106) September 2004, pp. 581-602.


Direct Method for Discrete Time Systems

Theorem 3.4 in A. Medio, M. Lines, Nonlinear Dynamics: A Primer, Cambridge University Press, Cambridge, UK, 2001.


Better Response Stability

Assume f is any better response process implemented on a generalized ordinal potential game with potential V.

Any fixed point of f is Lyapunov stable. Proof: Lyapunov function


Continuous Better Response Stability

Assume f is any better response process that converges to an isolated potential maximzer for a potential game then f is Lyapunov stable

Proof: Lyapunov function


Best Response Stability

Assume f is any best response process implemented on a generalized best response potential game with potential V.

Any NE s Lyapunov stable. Proof: Lyapunov function


How potential games handle the shortcomings

Steady-states– Finite game NE can be found from maximizers of V.

Optimality– Can adjust exact potential games with additive cost function

(that is also an exact potential game)– Sometimes little better than redesigning utility functions

Game convergence – Potential game assures us of FIP (and weak FIP)– DV satisfy Zangwill’s (if closed)

Noise/Stability– Isolated maximizers of V have a Lyapunov function for

decision rules in DV

Remaining issue:– Can we design a CRN such that it is a potential game for

the convergence, stability, and steady-state identification properties

– AND ensure steady-states are desirable?


Comments on Network Design Options

Approaches can be combined– Policy + potential– Punishment + cost adjustment– Cost adjustment + token economies

Mix of centralized and distributed is likely best approach Potential game approach has lowest complexity, but cannot be

extended to every problem Token economies requires strong property rights to ensure

proper behavior Punishment can also be implemented at a choke point in the

network

© cognitive radio technologies, 2007 1 supermodular games

Documents