game theory for wireless networks€¦ · pareto optimality; subgame perfection; ... game theory...
TRANSCRIPT
Game theory for wireless networks Georg-August University Göttingen
Game theory for wireless networks
static games;
dynamic games;
repeated games;
strict and weak dominance;
Nash equilibrium;
Pareto optimality;
Subgame perfection;
…
Georg-August University Göttingen Game theory for wireless networks
Outline
1 Introduction
2 Static games
3 Dynamic games
4 Repeated games
2
Georg-August University Göttingen Game theory for wireless networks
Brief introduction to Game Theory
Proper operation of wireless networks requires appropriate rule enforcement mechanisms to prevent or discourage malicious or selfish behavior
– Discouraging malicious or selfish behavior can benefit from game theoretic modeling
Classical applications: economics, but also politics and biology
Example: should a company invest in a new plan, or enter a new market, considering that the competition may make similar moves?
Most widespread kind of game: non-cooperative (meaning that the players do not attempt to find an agreement about their possible moves)
3
Georg-August University Göttingen Game theory for wireless networks
Classification of games
Non-cooperative Cooperative
Static Dynamic (repeated)
Strategic-form Extensive-form
Perfect information Imperfect information
Complete information Incomplete information
4
Cooperative
Imperfect information
Incomplete information
Georg-August University Göttingen Game theory for wireless networks
Classification of games
5
non-cooperative games: individuals cannot make binding agreements and the unit of
analysis is the individual. In cooperative game theory, binding agreements between
players are allowed and the unit of analysis is the group or coalition.
Static games: each player makes a single move and all moves are made simultaneously.
Perfect info: each player has a perfect knowledge of the whole previous actions of
other players at any moment it has to make a new move.
Complete info: each player knows who the other players are, what are their possible
strategies and what payoff will result of each player for any combination of moves.
Georg-August University Göttingen Game theory for wireless networks
Cooperation in self-organized wireless networks
6
S1
S2
D1 D2
Usually, the devices are assumed to be cooperative.
But what if they are not?
Georg-August University Göttingen Game theory for wireless networks
Assumptions and terminology
7
We will consider two players and non-cooperative games in four simple
game examples related to wireless network operations:
• Forwarder’s dilemma
• Joint packet forwarding game
• Multiple access game
• Jamming game
The players wish to transmit or receive data packets coping with limited
transmission resources which can make them to have selfish or malicious
behavior.
Si: Strategy of player i (e.g. forward the packet, drop the packet, etc.)
S-i: Strategy of the opponents of player i (as we will have only two players i
and j, S-i will be Sj)
Having any strategy Si the players would get some benefit and have to pay
some cost
Ui: Utility or payoff of player i expresses the benefit of him given a strategy
minus the cost it has to incur: Ui = bi - ci
Users controlling the devices are rational, i.e. try to maximize their benefit.
Georg-August University Göttingen Game theory for wireless networks
Outline
1 Introduction
2 Static games
3 Dynamic games
4 Repeated games
8
Georg-August University Göttingen Game theory for wireless networks
Example 1: The Forwarder’s Dilemma
9
?
?
Blue Green
Forwarder’s Dilemma:
• Two players: Blue and Green
• Green has one data packet to send to its receiver (R1) via Blue and also
blue has one data packet to send to its receiver (R2) via Green.
• If Blue forwards the packet for Green to R1, it must pay the transmission
cost of c and Green will get the benefit of 1.
• The dilemma:
• each player is tempted to drop the other player’s packet to save its
energy
• but the other player may reason in the same way -- > they could do
better by forwarding for each other
R1 R2
Georg-August University Göttingen Game theory for wireless networks
From a problem to a game
game formulation: G = (P,S,U)
– P: set of players
– S: set of strategy functions
– U: set of payoff functions
strategic-form representation of the game
(each cell corresponds to one possible combination of strategies of the players and is presented as (Ublue , Ugreen))
10
• Benefit for the source of a packet reaching the destination: 1
• Cost of packet forwarding for the forwarder: c (0 < c << 1)
(1-c, 1-c) (-c, 1)
(1, -c) (0, 0)
Blue
Green
Forward
Drop
Forward Drop
Georg-August University Göttingen Game theory for wireless networks
Solving the Forwarder’s Dilemma (1/2)
' '( , ) ( , ), ,i i i i i i i i i iu s s u s s s S s S
iu U
i is S
is
11
Strict dominance: strictly best strategy, for any possible strategy of the other
player(s)
where: payoff function of player i
strategies of all players except player i
In this game, Drop strictly dominates Forward from the point of view of both
players:
Because 1-c<1 and –c<0
Therefore the solution of the game is (D,D), i.e. both players choose the
strategy drop
Forward
Drop
Drop
Strategy strictly dominates strategy S’i if
(1-c, 1-c) (-c, 1)
(1, -c) (0, 0)
Blue Green
Forward
Georg-August University Göttingen Game theory for wireless networks
Solving the Forwarder’s Dilemma (2/2)
12
Solving the game by iterative strict dominance:
iteratively eliminate strictly dominated strategies
(1-c, 1-c) (-c, 1)
(1, -c) (0, 0)
Blue Green
Forward
Drop
Forward Drop
This is the lack of trust between the players that leads
to this suboptimal solution (green is never sure that Blue will
forward for it, if it forwards for blue (as they play simultaneously) and vice
versa.)
Drop strictly dominates Forward Dilemma
(F,F) would result in a better outcome for both players BUT }
Georg-August University Göttingen Game theory for wireless networks
Example 2: The Joint Packet Forwarding Game
13
?
Blue Green Source Dest
?
For this game there is no strictly dominated strategies !
• A sender sends its packets to its receiver and two players (Blue and Green)
should forward the packets for it.
• Reward for packet reaching the destination: 1 for both players
• Cost of packet forwarding: c (0 < c << 1)
(1-c, 1-c) (-c, 0)
(0, 0) (0, 0)
Blue
Green
Forward
Drop
Forward Drop
Georg-August University Göttingen Game theory for wireless networks
Weak dominance
14
? Blue Green Source Dest
?
'( , ) ( , ),i i i i i i i iu s s u s s s S
with strict inequality for at least one s-i
Solving the game by Iterative weak dominance:
iteratively eliminate weakly dominated strategies
(1-c, 1-c) (-c, 0)
(0, 0) (0, 0)
Blue
Green
Forward
Drop
Forward Drop For Green, Drop is weakly dominated
by Forward (so the second column is
eliminated); then in the remaining parts
of the table Blue would do better to
Forward.
The result of the iterative weak
dominance is not unique in general.
Weak dominance: Strategy s’i is weakly dominated by strategy si if
Georg-August University Göttingen Game theory for wireless networks
Nash equilibrium (1/2)
15
Nash Equilibrium: a game strategy such that no player can increase its
payoff by deviating unilaterally
(1-c, 1-c) (-c, 1)
(1, -c) (0, 0)
Blue
Green
Forward
Drop
Forward Drop
E1: The Forwarder’s
Dilemma: (Drop, Drop) is a Nash
Equilibrium; if Green deviates to
(D,F) its payoff decreases (from
0 to –c) and similarly if blue
deviates to (F,D) its payoff
decreases (from 0 to –c)
E2: The Joint Packet
Forwarding game: There are 2 Nash Equilibria for
this game: (F,F) and (D,D)
(1-c, 1-c) (-c, 0)
(0, 0) (0, 0)
Blue
Green
Forward
Drop
Forward Drop
Georg-August University Göttingen Game theory for wireless networks
Nash equilibrium (2/2)
16
* * *( , ) ( , ),i i i i i i i iu s s u s s s S
iu U
i is S
where: payoff function of player i
strategy of player i
( ) arg max ( , )i i
i i i i is S
b s u s s
The best response of player i to the profile of strategies s-i is
a strategy si such that:
Nash Equilibrium = Mutual best responses
Caution! Many games have more than one Nash equilibrium
Strategy profile s* constitutes a Nash equilibrium if, for each player i,
Georg-August University Göttingen Game theory for wireless networks
Example 3: The Multiple Access game
17
Benefit for the source of a
successful transmission: 1
Cost of transmission: c
(0 < c << 1)
• For this game:
o There is no strictly dominating strategy
o There are two Nash equilibria
(0, 0) (0, 1-c)
(1-c, 0) (-c, -c)
Blue
Green
Quiet
Transmit
Quiet Transmit
Time-division channel
Two players have packets to send to their receivers but
through the same channel; if in a given time slot both
transmit packets there will be a collision and no packet
will be delivered to its destination
If one player transmits and the other stays quiet in that
time slot, the packet will be received by its receiver and
its source will get some benefit
Georg-August University Göttingen Game theory for wireless networks
Mixed strategy Nash equilibrium
(1 )(1 ) (1 )blueu p q c pqc p c q
18
objectives
– Blue: choose p to maximize ublue
– Green: choose q to maximize ugreen
(1 )greenu q c p
1 , 1p c q c
p: probability of transmit for Blue
q: probability of transmit for Green
is a mixed strategy Nash equilibrium
Green: If p<1-c --> 1-c-p is positive; Green chooses q=1
If p>1-c --> 1-c-p is negative; Green chooses q=0
If p=1-c 1-c-p=0; Green’s benefit not affected by its move
Blue: If q<1-c --> 1-c-q is positive; Blue chooses p=1
If q>1-c --> 1-c-q is negative; Blue chooses p=0
If q=1-c 1-c-q=0; Blue’s benefit not affected by its move
Georg-August University Göttingen Game theory for wireless networks
Example 4: The Jamming game
19
Transmitter: has a data packet to send to
its receiver at each time slot
Jammer: is interested in preventing the
transmitter from successful transmission
by sending packets on the same channel
Transmitter:
benefit of 1 for successful transmission
loss of -1 for jammed transmission
Jammer:
benefit of 1 for successful jamming
loss of -1 for missed jamming
There is no pure-strategy Nash equilibrium
two channels:
C1 and C2
(-1, 1) (1, -1)
(1, -1) (-1, 1)
Blue
Green
C1
C2
C1 C2
transmitter
jammer
1 1,
2 2p q is a mixed strategy Nash equilibrium
p: probability of transmit on
C1 for Blue
q: probability of transmit on
C1 for Green
Georg-August University Göttingen Game theory for wireless networks
Theorem by Nash, 1950
20
Theorem:
Every finite strategic-form game has a mixed-
strategy Nash equilibrium.
Georg-August University Göttingen Game theory for wireless networks
Efficiency of Nash equilibria
21
E2: The Joint
Packet Forwarding
game (1-c, 1-c) (-c, 0)
(0, 0) (0, 0)
Blue
Green
Forward
Drop
Forward Drop
• How to choose between several Nash equilibria ?
• One way is to compare strategy profiles using the concept of
Pareto-optimality.
Pareto-optimality: A strategy profile is Pareto-optimal if it is not possible to
increase the payoff of any player without decreasing the payoff of another
player.
• Strategy profile s is Pareto-superior to strategy profile s’ if for any player i:
With strict inequality for at least one player.
Georg-August University Göttingen Game theory for wireless networks
Back to the examples
In Forwarder’s dilemma game: – The Nash Equilibrium (D,D) is not Pareto-Optimal
– Strategy profiles (F,F), (F,D) and (D,F) are Pareto-Optimal but not Nash Equilibria.
In Joint Packet forwarding game: – (F,F) and (D,D) are Nash Equilibria; out of them only (F,F) is Pareto-Optimal.
In Multiple Access game: – (T,Q) and (Q,T) are Nash Equilibria and Pareto-Optimal.
In Jamming game: – There exists no pure-strategy Nash Equilibrium (D,D) but all pure-strategy profiles are
Pareto-Optimal.
22
Georg-August University Göttingen Game theory for wireless networks
Back to the examples
In Multiple Access game with mixed-strategy: – The mixed-strategy Nash Equilibrium (p=1-c, q=1-c) results in the payoff (0,0);
– Hence, both pure-strategy Nash Equilibria are Pareto-superior to it.
It can be shown in general that there does not exist a mixed strategy profile that is Pareto-superior to all pure strategy profiles. Because each mixed strategy of a player I is a linear combination of her pure strategies with positive coefficients that sum up to one.
23
Georg-August University Göttingen Game theory for wireless networks
Outline
1 Introduction
2 Static games
3 Dynamic games
4 Repeated games
24
Georg-August University Göttingen Game theory for wireless networks
Extensive-form games
usually to model sequential decisions
game represented by a tree
Example 3 modified: the Sequential Multiple Access game: – Blue plays first, then Green plays.
– Blue: leader; Green: follower
25
Green
Blue
T Q
T Q T Q
(-c,-c) (1-c,0) (0,1-c) (0,0)
Reward for successful
transmission: 1
Cost of transmission: c
(0 < c << 1)
Green
Time-division channel
Georg-August University Göttingen Game theory for wireless networks
Strategies in dynamic games
The strategy defines the moves for a player for every node in the game, even for those nodes that are not reached if the strategy is played.
26
Green
Blue
T Q
T Q T Q
(-c,-c) (1-c,0) (0,1-c) (0,0)
Green strategies for Blue:
T, Q
strategies for Green:
TT, TQ, QT and QQ
TQ means that player p2 transmits if p1 transmits and remains quiet if p1 remains quiet.
There are three pure-strategy Nash Equilibria: (T,QT), (T,QQ) and (Q,TT)
Blue: T Green’s best move Q; Blue: Q Green’s best move T
Georg-August University Göttingen Game theory for wireless networks
Backward induction
Solve the game by reducing from the final stage
Green’s best moves: Q if Blue moves T and T if Blue moves Q
Blue can calculate its best move knowing this argument:
– Blue’s best move is T (comparing (1-c,0) to (0,1-c))
So backward induction starts from the last stage and ends at the root
– The continuous line from a leaf of the tree to the root would be the solution
27
Green
Blue
T Q
T Q T Q
(-c,-c) (1-c,0) (0,1-c) (0,0)
Green
Backward induction solution: h={T, Q}
Georg-August University Göttingen Game theory for wireless networks
Subgame perfection
In fact extends the notion of Nash equilibrium
28
Green
Blue
T Q
T Q T Q
(-c,-c) (1-c,0) (0,1-c) (0,0)
Green Stackelberg equilibria using
backward induction :
(T, QT) and (T, QQ)
Stackelberg equilibrium (subgame perfect equilibrium ): A strategy
profile s is a Stackelberg equilibrium for a game with p1 as the leader
and p2 as the follower if p1 maximizes her payoff subject to the
constraint that player p2 chooses according to her best response
function.
Georg-August University Göttingen Game theory for wireless networks
Outline
1 Introduction
2 Static games
3 Dynamic games
4 Repeated games
29
Georg-August University Göttingen Game theory for wireless networks
Repeated games
repeated interaction between the players (in stages)
move: decision in one interaction
strategy: defines how to choose the next move, given the previous moves
history: the ordered set of moves in previous stages
– most prominent games are history-1 games (players consider only the previous stage)
initial move: the first move with no history
finite-horizon vs. infinite-horizon games
stages denoted by t
30
Georg-August University Göttingen Game theory for wireless networks
Utilities: Objectives in the repeated game
finite-horizon (finite no. of stages ) vs. infinite-horizon games
myopic vs. long-sighted repeated games
1i iu u t
0
T
i i
t
u u t
31
0
i i
t
u u t
Myopic games : players maximize their next stage payoff
long-sighted finite games: players maximize their payoff
for the whole finite duration T
long-sighted infinite: the period of maximizing the
payoff is not finite
payoff with discounting: 0
t
i i
t
u u t
0 1 is the discounting factor
Georg-August University Göttingen Game theory for wireless networks
Strategies in the repeated game
usually, history-1 strategies (strategy depend only on the previous stage), based on different inputs:
– others’ behavior:
– others’ and own behavior:
– payoff:
1i i im t s m t
1 ,i i i im t s m t m t
32
1i i im t s u t
• Example strategies in the Forwarder’s Dilemma:
Blue (t) initial move
F D strategy name
Green (t+1) F F F AllC (always cooperate)
F F D Tit-For-Tat (TFT) (repeat the opponent’s move)
D D D AllD(always defect)
D D F Anti-TFT(opposite to the opponent’s move)
Georg-August University Göttingen Game theory for wireless networks
Analysis of the Repeated Forwarder’s Dilemma (1/3)
Blue strategy Green strategy
AllD AllD
AllD TFT
AllD AllC
AllC AllC
AllC TFT
TFT TFT
Blue payoff Green payoff
0 0
1 -c
1/(1-ω) -c/(1-ω)
(1-c)/(1-ω) (1-c)/(1-ω)
(1-c)/(1-ω) (1-c)/(1-ω)
(1-c)/(1-ω) (1-c)/(1-ω)
33
infinite game with discounting: usually used to model finite games
in which the players are not aware of the duration of the game
• By decreasing values of future payoffs:
0
t
i i
t
u u t
0< ω <1
Georg-August University Göttingen Game theory for wireless networks
Analysis of the Repeated Forwarder’s Dilemma (2/3)
AllC receives a high payoff with itself and TFT, but
AllD exploits AllC
AllD performs poor with itself
TFT performs well with AllC and itself, and
TFT retaliates the defection of AllD
34
Blue strategy Green strategy
AllD AllD
AllD TFT
AllD AllC
AllC AllC
AllC TFT
TFT TFT
Blue payoff Green payoff
0 0
1 -c
1/(1-ω) -c/(1-ω)
(1-c)/(1-ω) (1-c)/(1-ω)
(1-c)/(1-ω) (1-c)/(1-ω)
(1-c)/(1-ω) (1-c)/(1-ω)
TFT is the best strategy if ω is high (ω ≈ 1)!
Georg-August University Göttingen Game theory for wireless networks
Analysis of the Repeated Forwarder’s Dilemma (3/3)
Blue strategy Green strategy Blue payoff Green payoff
AllD AllD 0 0
TFT TFT (1-c)/(1-ω) (1-c)/(1-ω)
35
Theorem: In the Repeated Forwarder’s Dilemma, if both
players play AllD, it is a Nash equilibrium.
Theorem: In the Repeated Forwarder’s Dilemma, both
players playing TFT is a Nash equilibrium as well.
The Nash equilibrium sBlue = TFT and sGreen = TFT is
Pareto-optimal (but sBlue = AllD and sGreen = AllD is not) !
Georg-August University Göttingen Game theory for wireless networks
(1, 1) (-1, 2)
(2, -1) (0, 0)
Country 1
Country 2
Reduce
military
investment
Increase
military
investment
Reduce military
investment
Increase military
investment
Payoffs:
2: I have weaponry superior to the one of the opponent
1: We have equivalent weaponry and managed to reduce it on both sides
0: We have equivalent weaponry and did not managed to reduce it on both sides
-1: My opponent has weaponry that is superior to mine
An Example beyond Engineering
36
Georg-August University Göttingen Game theory for wireless networks
Who is malicious? Who is selfish?
37
Both security and game theory backgrounds are useful in many cases !!
Harm everyone: viruses,…
Selective harm: DoS,… Spammer
Cyber-gangster:
phishing attacks,
trojan horses,…
Big brother
Greedy operator
Selfish mobile station
Georg-August University Göttingen Game theory for wireless networks
Conclusion
Game theory can help modeling greedy behavior in wireless networks
Four simple examples were used to explain how to capture the wireless networking problem in a corresponding game
Game theory has been recently applied to wireless communication, but discipline still is in its infancy
38