Game Theory & Cognitive Radio

part A

Hamid Mala

2/28

Presentation Objectives

1. Basic concepts of game theory

2. Modeling interactive Cognitive Radios as a game

3. Describe how/when game theory applies to cognitive radio.

4. Highlight some valuable game models.

3/28

Interactive Cognitive Radios

Adaptations of one radio can impact adaptations of others

Interactive Decisions

Difficult to Predict Performance

4/28

Interactive Cognitive Radios

Scenario: Distributed SINR maximizing power control in a single cluster.

Final state : All nodes transmit at maximum power.

(1) the resulting SINRs are unfairly distributed (the closest node will have a far superior SINR to the furthest node)

(2) battery life would be greatly shortened.

Power

SINR

5/28

traditional analysis techniques

Dynamical systems theory optimization theory contraction mappings Markov chain theory

6/28

Research in a nutshell

Applying game theory and game models (potential and supermodular) to the analysis of cognitive radio interactions

– Provides a natural method for modeling cognitive radio interactions

– Significantly speeds up and simplifies the analysis process

– Permits analysis without well defined decision processes

Game Theory

Definition, Key Concepts

8/28

Exaple Same color winneropposite color winner

$ = card number of winner

9/28

Exaple Same color winneropposite color winner

$ = card number of winner

10/28

Exaple

(2,-2) (-8,8)

(-1,1) (7,-7)

Matrix representation

Girl’s strategiesBoy’s strategies

Pay-off function

11/28

Games

A game is a model (mathematical representation) of an interactive decision situation.

Its purpose is to create a formal framework that captures the relevant information in such a way that is suitable for analysis.

Different situations indicate the use of different game models.

1. A set of 2 or more players, N2. A set of actions for each player, Ai

3. A set of utility functions, {ui}, that describe the players’ preferences over the outcome space

Normal Form Game ModelNormal Form Game Model

12/28

An action vector from which no player can profitably unilaterally deviate.

, ,i i i i i iu a a u b a An action tuple a is a NE if for every i N

for all bi Ai.

Definition

Nash Equilibrium

13/28

Friend

Foe

Friend Foe

500,500 0,1000

0,01000,0

(Friend, Friend)?? No

(Friend, Foe)?? Yes

(Foe, Friend)?? Yes

(Foe, Foe)?? Yes

Friend or Foe Example

Modeling and Analysis Review

15/28

Modeling a Network as a Game

Network GameNodes

Power Levels

Algorithms

Players

Actions

Utility Functions

Structure of game is taken from the algorithm and the environment

[Laboratoire de Radiocommunications et de Traitement du Signal]

void update_power(void){/*Adjusting power level*/int k;

}

16/28

Modeling Review

The interactions in a cognitive radio network can be represented by the tuple:

<N, A, {ui}, {di},T>

Timings:– Synchronous– Round-robin– Random– Asynchronous

Dynamical System

17/28

1. Steady state characterization

2. Steady state optimality3. Convergence4. Stability5. Scalability

a1

a2

NE1

NE2

NE3

a1

a2

NE1

NE2

NE3

a1

a2

NE1

NE2

NE3

a1

a2

NE1

NE2

NE3

a3Steady State Characterization Is it possible to predict behavior in the system? How many different outcomes are possible?

Optimality Are these outcomes desirable? Do these outcomes maximize the system target parameters?

Convergence How do initial conditions impact the system steady state? What processes will lead to steady state conditions? How long does it take to reach the steady state?

Stability How does system variations impact the system? Do the steady states change? Is convergence affected?

Scalability As the number of devices increases, How is the system impacted? Do previously optimal steady states remain optimal?

Key Issues in Analysis

18/28

How Game Theory Addresses These Issues

Steady-state characterization– Nash Equilibrium existence– Identification requires side information

Steady-state optimality– In some special games

Convergence– in some cases

Stability, scalability– No general techniques– Requires side information

19/28

Nash Equilibrium Identification

Time to find all NE can be significant Let tu be the time to evaluate a utility function. Search Time:

Example: – 4 player game, each player has 5 actions.– NE characterization requires 4x625 = 2,500 tu

Desirable to introduce side information.

N

u iT t N A

20/28

Example(1) : The Cognitive Radios’ Dilemma

Example : The Cognitive Radios’ Dilemma

Frequency domain representation of waveforms

The Cognitive Radios’Dilemma in Matrix

NE=?

Two cognitive radios Each radio can implement two different waveforms

low-power narrowband higher power wideband

21/28

Repeated Games and Convergence

Repeated Game Model– Consists of a sequence

of stage games which are repeated a finite or infinite number of times.

– Most common stage game: normal form game.

Finite Improvement Path (FIP)– From any initial starting

action vector, every sequence of round robin better responses converges.

Weak FIP– From any initial starting

action vector, there exists a sequence of round robin better responses that converge.

22/28

Better Response Dynamic

During each stage game, player(s) choose an action that increases their payoff, presuming other players’ actions are fixed.

Converges if stage game has FIP.

a

b

A B

1,-1

-1,1

0,2

2,2

23/28

Best Response Dynamic

During each stage game, player(s) choose the action that maximizes their payoff, presuming other players’ actions are fixed.

converge if stage game has weak FIP.

a

b

A B

1,-1 -1,1

1,-1-1,1

C

0,2

1,2

c 2,12,0 2,2

24/28

Supermodular Games

Key Properties– Best Response (Myopic) Dynamic Converges– Nash Equilibrium Generally Exists

Why We Care– Low level of network complexity

How to Identify2

0, , ,i

i j

ui j N a A

a a

25/28

Supermodulaar Games

NE Existence: have at least one NE.

NE Identification: all NE for a game form a lattice. While this does not particularly aid in the process of initially identifying NE, from every pair of identified

Convergence: have weak FIP, so a sequence of best responses will converge to a NE.

Stability: if the radios make a limited number of errors or if the radios are instead playing a best response to a weighted average of observations from the recent past, play will converge.

26/28

Example : outer loop power control

Parameters– Single Cluster– Pi = Pj = [0, Pmax] i,j N

– Utility target SINR

Supermodular – best response convergence

i\ \

, 1 absii i j i i j j v

j N i j N i

u p p h p h p N

27/28

Summary

When we use game theory to model and analyse interactive CRs, it should address :

– steady state existense and identification– convergence– stability– desirability of steady states

Supermodular games : to some extent

Questions?

Game Theory & Cognitive Radiopart B

Mahdi Sadjadieh

30/28

Overview

Potential Game Model Type of Potential Game Example of Exact Potential Game FIP and Potential Games How Potential Games handle the

shortcomings Physical Layer Model Parameters and

Potential Game

31/28

Potential Game Model

Identification

NE Properties (assuming compact spaces)

– NE Existence: All potential games have a NE

– NE Characterization: Maximizers of V are NE

Convergence– Better response algorithms

converge.

Stability– Maximizers of V are stable

Design note: – If V is designed so that its

maximizers are coincident with your design objective function, then NE are also optimal.

, , , ,i i i i i i i i i iu b a u a a V b a V a a , , , ,i i i i i i i i i iu b a u a a V b a V a a

Existence of a potential function V such that

32/28

Potential Games

Existence of a function (called the potential function, V), that reflects the change in utility seen by a unilaterally deviating player.

E1

E2

E3

E4

GPGGOPG (Gilles)OPG GPG (finite A)

33/28

Potential Games

34/28

Ordinal Potential Game Identification

Lack of weak improvement cycles [Voorneveld_97]

FIP and no action tuples such that

Better response equivalence to an exact potential game [Neel_04]

, , ,i i i i i i i iu a a u b a b a

Not an OPG

An OPG

35/28

Ordinal Potential Game Identification

Lack of weak improvement cycles [Voorneveld_97]

FIP and no action tuples such that

Better response equivalence to an exact potential game [Neel_04]

, , ,i i i i i i i iu a a u b a b a

Not an OPG

An OPG

36/28

Other Exact Potential Game Identification Techniques

Linear Combination of Exact Potential Game Forms [Fachini_97]– If <N,A,{ui}> and <N,A,{vi}> are EPG, then <N,A,

{ui + vi}> is an EPG

Evaluation of second order derivative [Monderer_96]

22

, ,ji

i j i j

uui j N a A

a a a a

37/28

Exact Potential Game Forms

Many exact potential games can be recognized by the form of the utility function

38/28

Example Identification

Single cluster target SINR

Better Response Equivalent

\

ˆ

1/

i ii

k kk N i

g pu

K g p

p

2

'

\

ˆ /i k k i ik N i

u K g p g p

p

' 2 2

\

2

\

ˆ2 /

ˆ2 /

ˆ /

i i i i i

i k i kk N i

k kk N i

u g p K g p

K g g p p

K g p

p

2 2

ˆ2 /

ˆ2 /

n

i k i ki N i k

i i i ii N

V K g g p p

g p K g p

p

Dummy game

BSI game

Self-motivated game

39/28

FIP and Potential Games

GOPG implies FIP ([Monderer_96]) FIP implies GOPG for finite games

([Milchtaich_96]) Thus we have a non-exhaustive search

method for identifying when a CRN game model has FIP.

Thus we can apply FIP convergence (and noise) results to finite potential games.

40/28

Steady-states

As noted previously, FIP implies existence of NE

41/28

Optimality

If ui are designed so that maximizers of V are coincident with your design objective function, then NE are also optimal.

(*) Can also introduce cost function to utilities to move NE.

In theory, can make any action tuple the NE

– May introduce additional NE– For complicated NC, might as well

completely redesign ui

* *

0i i

V a NC a

a a

*i iu a u a NC a

V

a

42/28

Convergence in Infinite Potential Games

-improvement path– Given >0, an -improvement path is a path such that for all

k1, ui(ak)>ui(ak-1)+ where i is the unique deviator at step k.

Approximate Finite Improvement Property (AFIP)– A normal form game, , is said to have the approximate finite

improvement property if for every >0 there exists an such that the length of all -improvement paths in are less than or equal to L.

[Monderer_96] shows that exact potential games have AFIP, we showed that AFIP implies a generalized -potential game.

43/28

Convergence Implications

44/28

How potential games handle the shortcomings

Steady-states– Finite game NE can be found from maximizers of V.

Optimality– Can adjust exact potential games with additive cost function

(that is also an exact potential game)– Sometimes little better than redesigning utility functions

Game convergence – Potential game assures us of FIP (and weak FIP)– DV satisfy Zangwill’s (if closed)

Noise/Stability– Isolated maximizers of V have a Lyapunov function for

decision rules in DV

Remaining issue:– Can we design a CRN such that it is a potential game for

the convergence, stability, and steady-state identification properties

– AND ensure steady-states are desirable?

More Examples

46/28

Physical Layer Model Parameters

47/28

Assume that there is a radio network wherein each radio can alter their power.Assume each radio reacts to some separable function of SINR, e.g. log ratio

Each radio would also like to minimize power consumption

SINR Power Control Games

,1 , ,2 ,\ ,

,i i i

i

i i i i i j j v ij i ij N i

u a f p f p N c p

Decentralized Power Control Using a dB Metric

,1 ,,ii i i i i

i N

P a f p c p

Thus game is a potential game and convergence is assured and we can quickly find steady states.

48/28

Example Power Control Game

Parameters– Single Cluster– DS-SS multiple access– Pi = Pj = [0, Pmax] i,j N

– Utility target BER

target\

0\

, 1 abs BER i ii i j

j N ij j

j N i

h pu p p Q

Rh p N

W

Also a potential game.

49/28

Snapshot inner + outer loop power control

Parameters– Single Cluster– DS-SS multiple access– Pi = Pj = [0, Pmax] i,j N

– Utility target SINR

Supermodular – best response convergence

i\ \

, 1 absii i j i i j j v

j N i j N i

u p p h p h p N

50/28

Game Models, Convergence, and Complexity

Determining the kind of game required to accurately model a RRM algorithm yields information about what updating processes are appropriate and thus indicates expected network complexity.

In [Neel04] the following relation between power control algorithms, game models, and network complexity was observed.

51/28

Summary

Distributed dynamic resource allocations have the potential to provide performance gains with reduced overhead, but introduce a potentially problematic interactive decision process.

Game theory is not always applicable. Can generally be applied to distributed radio

resource management schemes.

Questions?

53/28

Example:Exact Poential Game

return

54/28 return

55/28

Example : Ordinal Poential Game

return

56/28

Example : Generalized Ordinal Poential Game

return

57/28

Exact Potential Game Forms

Many exact potential games can be recognized by the form of the utility function