evolutionary non-cooperative spectrum sharing game: long ...mainak/papers/faisal-wcmc.pdfmuhammad...

WIRELESS COMMUNICATIONS AND MOBILE COMPUTINGWirel. Commun. Mob. Comput. (2016)

Published online in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/wcm.2674

RESEARCH ARTICLE

Evolutionary non-cooperative spectrum sharing game:long-term coexistence for collocated cognitiveradio networks†

Muhammad Faisal Amjad1*, Mainak Chatterjee2, Omar Nakhila2 and Cliff C. Zou2

1 National University of Sciences and Technology, Islamabad, Pakistan2 Department of EECS, University of Central Florida, Orlando, FL, U.S.A.

ABSTRACT

Collocated cognitive radio networks (CRNs) employ coexistence protocols to share the spectrum when it is not being usedby the licensed primary users. These protocols work under the assumption that all spectrum bands provide the same levelof quality of service, which is somewhat simplistic because channel conditions as well as the licensee’s usage of allocatedchannels can vary significantly with time and space. These circumstances dictate that some channels may be consideredbetter than others; therefore, CRNs are expected to have a preference over the choice of available channels. Because allCRNs are assumed to be rational and select the best available channels, it can lead to an imbalance in contention fordisparate channels, degraded quality of service, and an overall inefficient utilization of spectrum resource. In this paper, weanalyze this situation from a game theoretic perspective and model the coexistence of CRNs with heterogeneous spectrumas an evolutionary anti-coordination spectrum-sharing game. We derive the evolutionarily stable strategy (ESS) of thegame by proving that it cannot be invaded by a greedy strategy. We also derive the replicator dynamics of the proposedevolutionary game, a mechanism with which players can learn from their payoff outcomes of strategic interactions andmodify their strategies at every stage of the game and subsequently converge to ESS. Because all CRNs approach ESSbased solely upon the common knowledge payoff observations, the evolutionary game can be implemented in a distributedmanner. Finally, we analyze the game from the perspective of fairness using Jain’s fairness index under selfish behaviorfrom CRNs. Copyright © 2016 John Wiley & Sons, Ltd.

KEYWORDS

cognitive radio networks; coexistence; evolutionary game theory; evolutionarily stable strategy

*Correspondence

Muhammad Faisal Amjad, National University of Sciences and Technology, Islamabad, Pakistan.E-mail: [email protected]

1. INTRODUCTION

The Federal Communications Commission (FCC) madeTV white space (TVWS) channels in the 54–698 MHz fre-quency range available [1] for secondary unlicensed accessafter the TV broadcast was switched from analog to digi-tal signal in 2009. Opening up of the TVWS for unlicenseduse was the result of a realization that the gap between thedemand and supply of wireless spectrum resource is everincreasing and that fixed spectrum allocation is causing itssevere underutilization [2]. Strict requirements are how-ever placed on the secondary users (SUs) of the spectrum,

†At the time of submission of this paper, Muhammad F. Amjad was

a PhD candidate at the Department of EECS, University of Central

Florida, USA.

which is otherwise allocated to licensees called primaryusers (PU), to continuously sense the spectrum and vacateit when the presence of the PU is detected and not to causethem any interference. This type of spectrum access is intu-itively called dynamic spectrum access (DSA). Cognitiveradio network (CRN) is a paradigm that meets preciselythese communication requirements and utilizes DSA toenable secondary, unlicensed access to TVWS spectrumbands in an opportunistic and non-interfering basis [1].

Dynamic spectrum access allows CRNs to ensure thattheir use of spectrum does not cause interference to PUs,while at the same time, all spectrum opportunities are uti-lized to the maximum. Within a CRN, the decision to selecta specific channel for DSA is usually made by a centralentity such as its base station or in case of an ad hoc CRN,an algorithm that enables all SUs to reach a consensus

Copyright © 2016 John Wiley & Sons, Ltd.

Evolutionary non-coop spectrum sharing game: long-term coexistence for collocated CRNs M. F. Amjad et al.

for choosing specific channel in a distributed manner. TheIEEE 802.22 wireless regional area network [3] is an exam-ple of CRNs with very large transmission ranges (from30 to 100 km) in which the base station controls all theoperation of the CRN including the choice of spectrumbands for communication. Regardless of how a decisionto select a specific channel is made, every entity withinthe CRN is bound to abide by that decision. On the otherhand, there may be multiple collocated CRNs within ageographical region, all of which compete for access tothe same set of available channels. Sharing of spectrumby collocated CRNs is called self-coexistence in the con-text of CRNs, which employ coexistence protocols such asthe Institute of Electrical and Electronics Engineers 802.22standard’s coexistence beacon protocol. However, withoutany controlling entity, fair distribution of heterogeneousspectrum resources is nontrivial in the case of multiplecollocated CRNs as they may be independently ownedand operated by different service providers. This bringsus to the definition of this paper’s problem statement forlong-term coexistence with heterogeneous spectrum, in thefollowing section.

1.1. Problem definition

Coexistence protocols employed by collocated CRNs workunder the assumption that all spectrum bands afford thesame level of quality of service and do not take into consid-eration the fact that these channels can be heterogeneous.The heterogeneity of channels can be in the sense that theymay vary in their characteristics such as signal-to-noiseratio or bandwidth. Similarly, a channel whose PU remainsidle for most of the time may be more attractive for a CRNas compared with a channel whose PU remains mostlyactive. This would entail that some channels can be consid-ered better than others and therefore can have an associatedquality parameter. As a result, CRNs are expected to have apreference over the set of available channels for secondaryaccess. Without any incentive for altruism, all CRNs wouldwant to gain access to the highest-quality channels result-ing in a conflict among rational entities. Therefore, in theabsence of any centralized enforcement mechanism, evolu-tion of a strategy that would ensure long-term coexistencewith fair distribution of heterogeneous spectrum resourcesamong collocated CRNs is a challenge and is the problemstatement for this paper.

1.2. Game theoretic approach

Game theory provides an elegant means to model strate-gic interaction between agents, which may or may notbe cooperative in nature. It has been applied to numer-ous areas of research involving conflict, competition, andcooperation in multi-agent systems, which also encom-pass wireless communications. Therefore, by leveragingthe mechanisms of game theory, we model the long-termsharing of heterogeneous spectrum by CRNs as an evolu-tionary anti-coordination spectrum-sharing game in which

collocated CRNs in a given region are its players. Thepayoff for every player in the game is determined bythe quality of the spectrum band to which it is able togain access.

In this paper, we present a detailed analysis on the evo-lutionary stability as well as fairness of the solution. Forany system with non-cooperative entities, it is likely thatthere will be some associated inefficiency. However, itis worth pointing out that fairness is the primary objec-tive of our proposed evolutionary heterogeneous spec-trum sharing game. We also confirm our findings throughdetailed simulations.

1.3. Contribution

In this paper, we have formulated an evolutionary spectrumsharing anti-coordination game and propose its solutionthat is stable even with the presence of greedy strategy,robust under changing network conditions, and at the sametime, results in fair distribution of the spectrum resources.Specifically, we have made the following contributions:

� As potential solutions for the heterogeneous spec-trum sharing game, we have derived the game’s pureand mixed strategy Nash equilibria (NE) (PSNE andMSNE, respectively).

� To show that the game’s strategy in MSNE is evolu-tionarily stable strategy (ESS), we prove that it cannotbe invaded by a greedy strategy and is robust underchanging network conditions.

� We have derived replicator dynamics of the proposedevolutionary game, a mechanism with which play-ers can learn from their payoff outcomes of strategicinteractions and modify their strategies at every stageof the game and subsequently converge to ESS.

� Finally, we have presented a fairness analysis ofthe proposed evolutionary game using Jain’s fairnessindex.

The rest of this paper is organized as follows: Section 2provides on overview of the existing work on varioussolutions for spectrum access and sharing. Section 3presents the underlying system model and assumptions forour proposed evolutionary spectrum sharing game, whileSection 4 presents the formulation of our proposed evolu-tionary game along with its replicator dynamics. Section 5presents the fairness analysis of the evolutionary game, andsimulation results are presented in Section 6. Section 7concludes the paper.

2. RELATED WORK

In this section, we provide an overview of some of theworks carried out in the domain of self-coexistence inCRNs as well as application of some of the game theo-retic solution concepts in the context of communicationnetworks.

Wirel. Commun. Mob. Comput. (2016) © 2016 John Wiley & Sons, Ltd.DOI: 10.1002/wcm

M. F. Amjad et al. Evolutionary non-coop spectrum sharing game: long-term coexistence for collocated CRNs

A spectrum sharing mechanism is proposed in [4] inwhich the PUs lease their licensed spectrum bands to SUsin return for their cooperation in relaying PUs’ traffic. Thework proves that the spectrum sharing game converges to aStackelberg equilibrium. Authors of [5] have developed anauction-based algorithm for joint allocation of resources,that is, source and relay nodes power profiles and sub-carrier assignment for amplify and forward orthogonalfrequency division multiple-access (OFDMA) systems. Itis based on a one-shot auction where each user submitsbids for all subcarriers at once based on the Shapley value.

Authors of [6] have applied the evolutionary game theo-retic concepts in order to make SUs of a CRN to participatein collaborative spectrum sensing in a decentralized man-ner. SUs learn through strategic interactions at every stageof the game, and the learning behavior is modeled with thehelp of replicator dynamics. A game theoretic approachbased on correlated equilibrium has been proposed in[7] for multi-tier decentralized interference mitigation intwo-tier cellular systems. Authors of [8] propose a multi-cell resource allocation game for efficient allocation ofresources in OFDMA systems based on throughput, inter-cell interference, and complexity. The subcarriers are con-sidered as players of the game, while the base station actsas the provider of external recommendation signal neededfor achieving correlation of strategies of players.

Authors of [9] model the competition among multiplefemtocell base stations for spectrum resource allocationin an OFDMA LTE downlink system as a static non-cooperative game. The correlated equilibrium of the gameis derived through a distributed resource block accessalgorithm, which is a variant of the no-regret learningalgorithm. CRNs with SUs having variable traffic charac-teristics are considered in [10] to tackle the problem ofdistributed spectrum sensing by modeling it as a coop-erative spectrum sensing game for utility maximization.The authors have proposed another variant of the no-regret learning algorithm called neighborhood learning,which achieves correlated equilibrium for the spectrumsensing game. In contrast to the no-regret learning algo-rithm, neighborhood learning is not completely distributedand requires some coordination among players to achievebetter performance.

Correlated equilibrium has been employed in [11] for aP2P file sharing non-cooperative game to jointly optimizeplayers’ expected delays in downloading files. Not upload-ing files for others causes an increase in file downloadtime for all players, which, in turn, forces even the non-cooperative players to cooperate. The authors of [12] tacklethe self-coexistence problem of finding a mechanism thatachieves a minimum number of wasted time slots for everycollocated CRN to find an empty spectrum band for com-munications. To do so, they employ a distributed modifiedminority game under incomplete information assumption.

Different punishment strategies have been employedin [13] that form part of a Gaussian interference game ina one-shot game as well as an infinite horizon repeatedgame to enforce cooperation. Spectrum sharing is however

considered within the context of a single CRN. Evolution-ary game theory is applied in [14] to solve the problem in ajoint context of spectrum sensing and sharing within a sin-gle CRN. Multiple SUs are assumed to be competing forunlicensed access to a single channel. SUs are consideredto have half-duplex devices so they cannot sense and accessa channel simultaneously. Correlated equilibrium has beenproposed in [15] as a solution for efficient coexistence bycollocated CRNs with heterogeneous channels.

Utility graph coloring is used to address the problem ofself-coexistence in CRNs in [16]. Allocation of spectrumfor multiple overlapping CRNs is carried out using graphcoloring in order to minimize interference and maximizespectrum utilization using a combination of aggregation,fragmentation of channel carriers, broadcast messages, andcontention resolution. The authors of [17] achieve cor-related equilibrium with the help of no-regret learningalgorithm to address. the problem of network congestionwhen a number of SUs within a single CRN contendfor access to channels using a CSMA-type MAC proto-col. They model interactions of SUs within the CRN asa prisoners dilemma game in which payoffs for the play-ers are based on aggressive or non-aggressive transmissionstrategies after gaining access to idle channels.

3. SYSTEM MODEL ANDASSUMPTIONS

3.1. System model

As shown in Figure 1, we consider a region where col-located and overlapping CRNs coexist and compete witheach other for secondary access to the licensed spectrumbands. We model the entire TVWS spectrum band that isavailable for unlicensed use by CRNs as a set of K D1, 2, : : : , k channels. The spectrum band is heterogeneousby virtue of the quality of a channel, which is determinedby the probability Pk with which PUs access their licensedchannels. Because knowledge of PU’s spectrum alloca-tion/activity is mandated by the FCC for CRNs [1,3], it ispublically available through online databases [18,19] andalso sensed by CRNs at regular intervals, and players cancalculate current values of Pk based on past observations.Higher Pk for a given channel k means it is of a lower qual-ity and vice versa, and CRNs compete to access the bestquality channels. Gaining access to higher-quality chan-nel results in higher payoff uk, while lower-quality channelyields lower payoff for CRNs where payoff uk D 1 � Pkfrom gaining access to channel k. Because every CRNis required to sense for the presence of PU on the spec-trum, all of them can calculate Pk, and hence, the payoffsof all the channels are considered common knowledgeas they would be the same for every player. CRNs needto gain access to a channel in every time slot, which isalso called a channel detection time slot [3]. Players areassumed to be rational and non-cooperative; that is, theydo not share a common goal and therefore do not coop-erate with each other. It is in every CRN’s interest to



Figure 1. (a) Collocated cognitive radio networks competing for (b) Heterogeneous channels. The channels of the spectrum band mayvary in quality with respect to availability, bandwidth, or signal-to-noise ratio.

gain access to the channels with minimum PU activity,that is, minimum value of Pk. When two or more CRNsselect the same channel for access in a given time slot,a contention/collision situation arises and that particulartime slot’s spectrum opportunity is wasted. Having pay-offs for selecting a specific channel derived from commonknowledge such as Pk is an intuitive choice and makesdistributed implementation of our proposed frameworkpossible. It is worth mentioning that any positive valuefor payoff derived from any other parameter, for exam-ple, quality of service or bandwidth, can be used insteadof Pk without affecting our analysis and the outcomes.Table 1 provides definitions of notations and acronymsused in this paper. As demonstrated subsequently, the num-ber of collocated CRNs does not play any part in thegame model because an evolutionary game is concernedwith the evolution of strategies, associated payoffs, andtheir stability.

3.2. Assumptions

Following are the underlying assumptions for the workpresented in this paper:

� Time: A single medium access control superframeconstitutes one time slot. Every CRN needs to gainaccess to a channel for which it contends with allother collocated CRNs in every time slot.

� Spectrum opportunity and wastage: A given timeslot’s spectrum opportunity that arises owing to theabsence of its PU may result in a collision and there-fore be wasted if two or more CRNs select the samechannel for access.

� Knowledge about PU activity: In addition to the FCCmandated continuous spectrum sensing to detect PUs’activity, CRNs are also required to periodically accessonline databases such as that in [18,19] in orderto gain up-to-date information about licensed PUsoperating in a given region.

� Channel quality: The amount of PU activity, band-width, and signal-to-noise ratio that, for the purposeof this paper collectively determine a channel’s qual-

Table I. Notations and acronyms.

Notation Definition

K Set of available channelsA Set of available actions (selecting channels)U Set of channels’ utilitiesak CRN’s action of selecting channel kuk CRN’s utility for gaining access to channel k,

uk may be any positive numberai action/strategy played by player ia��i Best actions/strategies played by players

other than player ia�i Action/strategy of player i which is the best

response (PSNE) to a��i

Op ESS probability distribution over set ofchannels (in MSNE)

p0 A mutant strategy that is greedier than ESSstrategy

EUk Expected utility from accessing channel kt Current timeCDT Channel detection time (one time slot)ESS Evolutionarily stable strategyPU Primary userSU Secondary userNE Nash equilibriumPSNE Pure strategy Nash equilibriumMSNE Mixed strategy Nash equilibrium

ity can be learnt from online databases and measuredthrough spectrum sensing over a period of time.Because of the fact that all contending CRNs are col-located in a given region, it is reasonable to assumethat a given channel’s quality is common knowledge.

� Non-cooperative behavior: All CRNs are indepen-dent as they do not share a common goal and thereforedo not cooperate with each other. Being rational abouttheir choices [20], every player has a clear prefer-ence of selecting the best available channel before thestart of every time slot. As a consequence of assumingrational behavior from them, players aim to maximizeonly their own payoffs. Therefore, if every player triesto access the best channel in a given time slot, it



will result in a collision and the spectrum opportunitybeing wasted.

� Payoffs‡: Players§ that eventually gain access tohigher-quality channels will gain higher payoffs ascompared with the players that end up with lower-quality channels. In the subsequent section, we showthat our proposed spectrum sharing game can beimplemented solely on the basis of a CRN’s commonknowledge payoff observations.

4. EVOLUTIONARYANTI-COORDINATION SPECTRUMSHARING GAME

In this section, we first present the basics of evolutionarygame theory followed by formulation of our proposed evo-lutionary spectrum sharing game. Next, we derive solutionsfor the game for a two-channel scenario and extend it for aK-channel scenario with replicator dynamics.

4.1. Evolutionary game theory: basics

Evolutionary game theory formalizes the way in whichvarious strategies of a population mix interact while com-peting against each other. As a result of such competitions,relative fitness of strategies can be determined based uponthe payoffs that the strategies bring. An incumbent strat-egy of a population may be invaded by a mutant strategyif, on average, the mutant strategy can bring higher pay-offs than the incumbent strategy. A strategy that cannot beinvaded by a mutant strategy is said to be an evolutionar-ily stable strategy or ESS. In this paper, we consider theaction of selecting a specific channel as a CRN’s strategyand need to determine which strategies are fair and stablefor the long term. To that end, we derive the PSNE andMSNE as the game’s solutions and prove that MSNE isESS; that is, MSNE cannot be invaded by a mutant strategythat is greedier than MSNE. In addition to being evolution-arily stable, MSNE of the game is also fair because of itsdefinition, which is presented subsequently.

4.2. Game formulation

The heterogeneous spectrum sharing anti-coordinationgame presented in this paper is a non-cooperative repeatedgame with perfect information because

� Being rational players, CRNs compete for the bestchannels available in the spectrum band and are inter-ested only in maximizing their own utility. Therefore,CRNs are not bound to cooperate with each other.

� Utilities are common knowledge because the qualityof various network parameters can be measured byevery CRN. Also, every CRN can tell which channels

‡We use the terms utility and payoff interchangeably.§Similarly, we use the terms CRNs and players interchangeably.

other CRNs were able to gain access to in the past;hence, they know other CRNs’ payoffs.

The evolutionary heterogeneous spectrum sharing gameis represented as G D h.K/, .A/, .U/i where K D

f1, 2, : : : , kg denotes the set of available channels. Everyplayer in the game has the same action space representedby A D fa1, a2, : : : , akg, and there is a bijection betweenthe sets A and K. The set of utilities of the channels isrepresented as U D fu1, u2, : : : , ukg. Strategy ak meansselecting channel k for communication, and a player gainsa payoff of uk if it selected channel k and no other playerselected the same channel for a given time slot. The payofffor players’ playing strategies ak and aj when competingagainst each other is denoted by the ordered pair u.ak, aj/ 2

U and is a function of an individual channel’s qualitygiven by

u.ak, aj/ D

�.uk, uj/when k ¤ j.0, 0/when k D j

uk, uj > 0 (1)

where the first element of the ordered pair u.ak, aj/ rep-resents the payoff for player that selected channel k andthe second element for player that selected channel j. Forthe sake of clarity and ease in analysis and without anyloss of generality, we assume that uk > uj,8u 2 Rk

�0.Also, we initially consider a two-channel game, that is,a game with two heterogeneous channels, and derive itsPSNE and MSNE as potential solutions. Later, we considerthe K-channel scenario where K D jKj, in Section 4.5and derive the replicator dynamics of the proposed evo-lutionary game. Replicator dynamics is a mechanism withwhich players can learn from their payoff outcomes ofstrategic interactions and modify their strategies at everystage of the game to converge to ESS. The game repre-sented by Equation (1) can also be represented in strategicform as Table II, which shows the payoffs for two play-ers selecting channels k or j. Because uk > uj, it is inevery CRN’s interest to choose channel k instead of chan-nel j for a larger payoff. However, when the players selectthe same channel, it results in a collision, the spectrumopportunity being wasted and both players end up with apayoff of 0. On the other hand, if both players select dif-ferent channels, then their payoffs reflect the quality ofthe channel to which they are able to gain access, hencethe name anti-coordination game. As shown in Table II,this game is the reverse of the classic Battle of the Sexesgame and is classified as an anti-coordination game where

Table II. Strategic form representationof evolutionary heterogeneous spectrumsharing game with deterministic strategies

ak and aj .

ak aj

ak .0, 0/ .uk , uj/

aj .uj , uk / .0, 0/



it is in both players’ interest not to end up selecting thesame strategy.

4.3. Pure and mixed strategy Nashequilibria for the evolutionary spectrumsharing game

In this section, we first derive the PSNE followed byMSNE, which are the two potential solutions that areconsidered for our evolutionary spectrum sharing anti-coordination game.

Definition 1. The PSNE [20,21] of the spectrum sharinggame is an action profile a� 2 A of actions, such that

u�a�i , a��i

�� u

�ai, a��i

�,8i 2 N (2)

where � is a preference relation over payoffs of strate-gies a�i and ai. The aforementioned definition means thatfor a�i to be a PSNE, it must satisfy the condition that noplayer i has another strategy that yields a higher payoff thanthe one for playing a�i , given that every other player playstheir equilibrium strategy a��i.

Lemma 1. Strategy pairs .ak, aj/ and .aj, ak/ are PSNEof the anti-coordination game in Table II.

Proof. Assume player 1 to be the row player and player2 to be the column player in Table II. From Equation (1),it follows that both uk and uj are positive values, andtherefore, the payoffs for strategy pairs .ak, aj/ and .aj, ak/

are greater than the payoffs for strategy pairs .ak, ak/ and.aj, aj/. Consider the payoff for strategy pair .ak, aj/ inTable II. Given that the player playing strategy aj contin-ues to play this strategy, then from Definition 1 for PSNE,it follows that the player playing strategy ak does not haveany incentive to change its choice to aj; that is, it willreceive a smaller payoff of 0 if it unilaterally switched to aj.Therefore, .ak, aj/ is a PSNE. The same argument can beapplied to prove that the strategy pair .aj, ak/ is the secondPSNE of this game.

Definition 2. The MSNE [20,21] of the spectrum sharinggame is a probability distribution Op over the set of actionsA for any player such that

Op D�p1, p2, : : : , pjKj

�2 RjKj�0 , and

jKjXjD1

pj D 1 (3)

which makes the opponents indifferent about the choiceof their strategies by making the payoffs from all of theirstrategies equal.

Let ˛ be the probability with which player 1 plays strat-egy ak and ˇ D .1 � ˛/ be the probability of playingstrategy aj, and then from the payoffs in Table III, theexpected utility EU2.ak/ of player 2 for playing strategy akis given by

EU2.ak/ D ˛u.ak, ak/C ˇu.aj, ak/ D ˛.0/C ˇuk (4)

Table III. Strategic form representation of evolutionaryheterogeneous spectrum sharing game with probabilis-

tic strategies ak and aj .

Op Prob..ak / D ˛ Prob..aj/ D ˇ

Prob..ak / D ˛ .0, 0/ .uk , uj/

Prob..aj/ D ˇ .uj , uk / .0, 0/

Similarly, the expected utility EU2.aj/ of player 2 forplaying strategy aj is given by

EU2.aj/ D ˛u.ak, aj/C ˇu.aj, aj/ D ˛uj C ˇ.0/ (5)

According to Definition 2, player 2 will be indifferentabout the choice of strategies when the expected utilitiesfrom playing strategies ak and aj are equal, that is,

EU2.ak/ D EU2.aj/ (6)

Substituting (4) and (5) in (6), we have ˇ.uk/ D ˛.uj/.Therefore,

˛ Duk

uk C uj(7)

ˇ D 1 � ˛ Duj

uk C uj(8)

The MSNE for the heterogeneous spectrum sharinggame is given by the distribution Op D f˛,ˇg ofEquations (7) and (8), which means that when both play-ers select strategies ak and aj with probabilities ˛ and ˇ,respectively, then their opponents will be indifferent aboutthe outcomes of the play. This means that all CRNs ina given region form a polymorphic population in whichevery CRN mixes for its choice of available channelsaccording to the probability distribution Op, which is theMSNE for our evolutionary channel sharing game. Theprobability distribution Op also represents the proportionsof the population adopting different strategies at any givenstage of the game. To generalize, expected utility for everyplayer i in a K-channel heterogeneous spectrum sharinggame is given as follows:

EUm D

jKjXmD1

um.pm,8i, m 2 K (9)

where pm represents the probability of a CRN selectingchannel m and all other CRNs not selecting channel m.

4.4. Evolutionary stabilityof the game’s equilibria

To determine if the game’s solutions derived in precedingsections can be invaded by a mutant strategy that is greed-ier, we analyze its evolutionary stability with the help ofDefinition 3 as follows:



Definition 3. For a strategy Op to be ESS, it must satisfythe following conditions [22]:

(1) u .Op, Op/ � u�p0, Op

�and

(2) if u .Op, Op/ D u�p0, Op

�then u

�Op, p0

�> u.p0, p0/

where Op is the strategy played by the population and cantherefore be termed as the population’s incumbent strat-egy, while p0 is a mutant strategy that competes with theincumbent strategy. According to the first condition of Def-inition 3, an incumbent strategy (i) must be a symmetricNE and (ii) must perform at least as good against itself asit does against a mutant strategy. According to the secondcondition of Definition 3, if an incumbent strategy is nota strict NE, then the incumbent strategy must do strictlybetter against a mutant than a mutant strategy does againstitself. Now we analyze both PSNE and MSNE derived inthe preceding section according to Definition 3 to see ifthey are evolutionarily stable.

4.4.1. Evolutionary stability of pure strategy

Nash equilibria.

Earlier, we proved that the strategies .ak, aj/ and .aj, ak/

are the PSNE of our evolutionary game. If two playersplay the same strategy, that is, play Op, Op and are in equi-librium, then it is said to be a symmetric NE. Clearly, thePSNE of our game are not symmetric NE and by condi-tion (1) of Definition 3, u .Op, Op/ < u

�p0, Op

�. Therefore, the

PSNE is not evolutionarily stable according to Definition 3.Another aspect of the PSNE is that it is always unfair forthe player that selected the lower-quality channel, thereforemaking it impractical as a long-term strategy for CRNs’channel selection.

4.4.2. Evolutionary stability of mixed strategy

Nash equilibria.

With no PSNE for our evolutionary game as ESS,we now determine if the MSNE that we derived inEquations (7) and (8) is an ESS according to Definition 3.To do so, we first calculate u .Op, Op/, that is, see how theincumbent strategy Op fares against itself and then determinethe payoff of a mutant strategy p0 against the incumbentstrategy. Consider the payoff matrix in Table III wherethe players select strategies ak and aj with the probabilitydistribution of the incumbent strategy Op D f˛,ˇg, then

u .Op, Op/ D ˛ˇ.uk C uj/ (10)

In Equation (10) earlier, we have determined the pay-off of incumbent strategy Op when it competes against itself,that is, u .Op, Op/. Now consider a mutant strategy p0 Df˛C ı,ˇ � ıg, which is greedier than the incumbent strat-egy Op and assume that it selects the higher-quality channelk with a higher probability, that is, ˛ C ı, and selectsthe lower-quality channel j with lower probability, thatis, ˇ � ı, where ı is a small positive number that rep-resents the increase in greediness/probability of a mutantstrategy to select a higher-quality channel. Because of theexistence of two competing strategies, we now calculate

u�p0, Op

�, that is, the utility of the mutant strategy against

the incumbent strategy:

H) u�p0, Op

�D ˛ˇ.uk C uj/ � ı.˛uk � ˇuj/ (11)

Because uk > uj as assumed in Section 4.2, we know that˛uk is greater than ˇuj, and therefore, the second term ofEquation (11) is positive. From Equations (10) and (11),we have u .Op, Op/ > u

�p0, Op

�. Because u .Op, Op/ is strictly

greater than u�p0, Op

�, we do not need to check for the sec-

ond condition of Definition 3, and we conclude that theincumbent strategy Op does strictly better than the mutationp0, which will die out in the evolutionary game. Hence, ourMSNE cannot be invaded by the greedier mutation p0 andis therefore an ESS.

It is pointed out that derivation of MSNE becomesintractable when the number of channels is greater than2. To expand our analysis for a K-channel scenario, wenow introduce the concept of replicator dynamics in thefollowing section.

4.5. Replicator dynamics andK-channel scenario

In the aforementioned section, we have shown that theMSNE of our proposed evolutionary game framework isevolutionarily stable. Evolutionary stability has providedus with a means to evaluate how the channel selectionstrategies perform in the long run when the CRNs do notcooperate with each other. This concept is somewhat staticin nature because it does not demonstrate the dynamicswith which the strategies evolve and converge to an equilib-rium state. replicator dynamics explain how players evolvetheir behaviors by learning through strategic interactionsat every stage/generation of the game to reach the equilib-rium state, which is also evolutionarily stable. In order toshow the dynamics and to extend our analysis to the K-channel scenario, we now derive the replicator dynamics ofour evolutionary heterogeneous spectrum sharing game.

From Section 4.3, let Op D fp1, p2, : : : , pkg andPjKjjD1 pj D 1, where Op represents the strategy of select-

ing channel k with probability pk. Alternatively, we canalso think of pk as the proportion of population that selectchannel k at any given time. Furthermore, let u0 be the ini-tial fitness of every CRN, and the average payoff of CRNsselecting channel k at a given stage of the game be repre-sented by the set U D fu1, u2, : : : , uKg. Then payoff for aCRN selecting channel k can be calculated as

uk D u0 C

jKjXjD1

pku.ak, aj/,8k, j 2 K (12)

where u.ak, aj/ is the fitness of a CRN that selects chan-nel k in a pairwise competition against a CRN that selectschannel j. Let Nu be the total average payoff of the entireCRN population at any given time. Then Nu for the entirepopulation of CRNs is given by

Nu DkX

nD1

pnun,8n 2 K (13)



and probability p0k of a CRN selecting channel k for thenext stage/time slot of the game is given by

p0k D pk Cpk .uk � Nu/

Nu(14)

Equations (12)–(14) are the replicator dynamics of ourevolutionary spectrum sharing game. The idea behind thereplicator dynamics is that if selecting channel k in the cur-rent time slot results in a higher average fitness for theCRNs that selected it than the overall fitness of the entireCRN population, then the proportion of CRNs selectingchannel k in the next time slot will increase. In Defini-tion 2 of Section 4.3, we stated that probability distributionOp, which is the game’s MSNE, also represents the pro-portions of the population adopting different strategies atany given stage of the game. CRNs are able to calculatethe total average payoff for the entire CRN population Nuof Equation (13) because it is based on common knowl-edge parameters: pn is the proportion of population thatselected channel n, while channel quality represented byun is also known to every CRN. In general, if selecting aparticular channel in a given time slot results in a higherthan total average payoff, then that channel will be selectedmore frequently in subsequent time slots, ultimatelyconverging to ESS.

5. FAIRNESS ANALYSIS OFDERIVED EQUILIBRIA

We now provide an analysis on the fairness of the NEderived in the preceding section. For the sake of clarityand ease of understanding, we consider the case of a two-channel heterogeneous spectrum sharing game, while thesame arguments can be applied for analyzing a K-channelscenario. The NE being considered as solutions for thespectrum sharing heterogeneous game are

� Two PSNE for the anti-coordination game are .ak, aj/

and .aj, ak/.� An MSNE defined by the probability distribution Op Df˛,ˇg given by Equations (7) and (8).

One of the ways to determine if entities receive a fairshare of the system’s resources is with Jain’s fairnessindex [23]. If there are N CRNs and every CRN’s utility isgiven as ui, then fairness of the derived NE can measuredby Jain’s equation as

J .u1, u2, : : : , uN/ D

�PNiD1 ui

�2

N.PN

iD1 u2i

(15)

As assumed previously in Section 3 for a two-channelscenario, channel k is of higher quality than channel j;therefore, uk > uj. Then from the payoff matrix in Table II,gaining access to channel k brings a larger payoff toa CRN, whereas being of comparatively lower quality,channel j brings a smaller payoff. There are two PSNE,.ak, aj/ and .aj, ak/; however, intuitively, both of them areunfair because uk ¤ uj and one player always obtains asmaller payoff than the other. This can be confirmed withEquation (15) as follows: whenever all ui are equal, then

the ratio�PN

iD1 ui

�2=PN

iD1 u2i in Equation (15) yields a

value equal to N and Jain’s index would be equal to 1,that is, the maximum, while for an unequal distributionof payoffs, it would be smaller than 1. Because PSNEdoes not result in equal payoff for all CRNs, it is not afair solution.

Let us now consider fairness of MSNE. According toDefinition 2, MSNE is a probability distribution over theset of strategies that makes the players indifferent abouttheir choice of strategies by making the payoffs equal eventhough the channels are of different quality. When all thepayoffs ui become equal, then from the same argumentof the preceding paragraph, Equation (15) yields an indexequal to 1, resulting in the MSNE’s resource distribution tobe fair.

6. SIMULATIONS AND RESULTS

6.1. Simulation preliminaries

We have conducted simulations to study the effectsof applying evolutionary game theoretic model for



self-coexistence with heterogeneous channels and to studyhow the channel selection strategies in MSNE are also theevolutionarily stable states. We first show the results ofsimulations in which the collocated CRNs have only twoavailable channels for which they contend and converge toan evolutionary stable state. Later, we show that our evolu-tionary game converges to ESS when there are more thantwo channels available for contention. To that end, we haveimplemented the replicator dynamics and provide resultsof our experiments with three, four, and five heterogeneouschannels as well. We also show that the evolutionary gamecan converge to new ESS when the network conditions maybe changing, requiring that the CRNs adjust to the newenvironments. As described in Section 4.2, ak means theaction of selecting channel k.

6.2. Results

Figure 2 represents the scenario in which CRNs contendfor two channels for secondary access. Figure 2(a) showshow CRNs select one out of two available channels withsome probability, where channel 1 is of better qualitythan channel 2. Any positive values for channel utilitieswould work however in case of simulations in Figure 2are assumed to be u1 D 9 and u2 D 7 for channels 1and 2, respectively, and its MSNE is p1 D 0.5625, p2 D

0.4375. Payoff from such strategic interactions is shownin Figure 2(b) based on which, CRNs modify the proba-bilities of selecting the same channels in subsequent timeslots/stages.

Let us first consider payoffs of CRNs that select chan-nels with smaller payoffs. As shown in Figure 2(b), CRNsthat select the lower-quality channel receive a larger aver-age payoff at t D 1 than CRNs that select higher-qualitychannel. This happens because more CRNs would wantto gain access to higher-quality channel, resulting in col-lisions and a zero payoff. Receiving higher payoff makesthe CRNs that selected smaller payoff channels to furtherincrease the probability of selecting the lower-quality chan-nel at t D 2 (Figure 2(a)). This however, results in loweraverage payoff for them at t D 2 than at t D 1. This hap-pens because the higher-quality channels are accessed witha relatively smaller probability at t D 2 because in previoustime slot, it had resulted in smaller payoff. A smaller payoffat t D 2 compared with higher payoff at t D 1 from access-ing channel 2 is still greater than the total average payoff ofthe entire CRN, which results in an even greater probabil-ity of selecting lower-quality channel in subsequent stages.A similar yet opposite pattern can be seen for CRNs thatselect higher quality channels with higher probabilities.Stated in another way, the proportion of CRNs selecting aparticular channel increases if its payoff is bigger than totalaverage payoff of the entire population and vice versa.

Cognitive radio networks keep modifying their channelselection probabilities in the same manner until their pay-offs converge and they reach the ESS, which in the case inFigure 2(a) is p1 D 0.5625, p2 D 0.4375 at around t D 25.The amount of time taken to converge to ESS is impor-tant as it would determine spectrum wastage because ofcollisions and is demonstrated in subsequent simulations.The average payoff uk of selecting a given channel k is

Figure 2. Channel access probabilities and average payoffs when the number of channels available for contention is K D 2. (a)Channel access probability and (b) average payoffs when the initial probabilities are unequal (c) and (d) initial probabilities are equal,

(e) and (f) under changing network conditions, that is, quality of channel 1 becomes worse than channel 2 at t D 50.



calculated by having the initial payoff u0 of Equation (12)equal to 1. Figure 2(c) and (d) represents the case wheninitial channel selection probabilities are equal yet stillconverge to ESS. Figure 2(e) and (f) represents changingnetwork conditions; that is, quality of channel 1 becomesworse than channel 2 at t D 50, yet the channel selectionstrategies still converge to a new ESS.

Figure 3(a) demonstrates that the MSNE of the evolu-tionary game achieves a fair distribution of the heteroge-neous spectrum resources. For this simulation, there aretwo channels available for contention, that is, K D 2 andu1 D 9 and u2 D 7. As shown in Figure 2(a) and (c),CRNs are free to select any initial probabilities for the twochannels, but any selection results in convergence to ESS;however, the payoffs vary with every probability distribu-tion. For the purpose of Figure 3, the X-axis represents

the initial probability of players selecting channel 1, thatis, at the start of simulation, whereas every data point onthe Y-axis represents total payoffs from selecting differ-ent probability distributions for channel selection until theyreach ESS. With the given utilities, MSNE of the game isp1 D 0.5625 and p2 D 0.4375. The figure shows that thetotal payoff for both channels becomes equal when prob-ability of selecting channel 1 equals p1 D 0.5625, andtherefore, p2 D 0.4375, which is the game’s MSNE aswell as the ESS as shown in Figure 2, making it the onlyprobability distribution of selecting the two channels thatis fair.

We have also carried out simulations to demonstratethe robustness of our game to evolve an ESS even whenthe initial estimates of the players regarding heteroge-neous channels are incorrect. To do so, we initialize the

Figure 3. (a) Total payoff from both channels becomes equal when initial probability of selecting channel 1 equals mixed strategyNash equilibria p1 D 0.5625, that is, the evolutionarily stable strategy probability. (b) Channel access probability and (c) average

payoffs when the initial probabilities are equal for a three-channel scenario.

Figure 4. Channel access probabilities and average payoffs when the number of channels available for contention is K D 3 and K D 4.(a),(e) Channel access probability and (b),(f) average payoffs when the initial probabilities are unequal, (c) Channel access probability

and (d) average payoffs when the initial probabilities are equal.



Figure 5. Channel access probabilities and average payoffs when the number of channels available for contention is K D 5. (a)Channel access probability and (b) average payoffs when the initial probabilities are equal.

Figure 6. Channel access probabilities and average payoffs when the number of channels available for contention is K D 5. (a)Channel access probability and (b) average payoffs when the initial probabilities are unequal.

players’s probabilities of accessing the channels to equaland unequal values and show that the strategies still con-verge to ESS. Also, we show that the game evolves itsESS even when the number of available channels is var-ied arbitrarily. However, the rate of convergence to ESSdepends on the number of channels, difference in their rel-ative quality, and the accuracy of players’ estimates abouttheir quality depicted by their choice of assigning initialaccess probabilities.

Figures 3(b) and (c), 4, 5, and 6 show the conver-gence of channel selection probabilities to ESS along withtheir respective average payoffs in cases where the num-ber of channels is increased to 3, 4, and 5, respectively,and channel utilities are varied between values such as 9and 4. The initial channel selection probabilities may beequal or unequal, yet in any case, the game always con-verges to the ESS for any given set of channel utilities.Another important observation is that the convergence rateto ESS decreases with the increase in number of channelsand how accurate the initial probabilities are as comparedwith the ESS.

7. CONCLUSION

Coexistence protocols employed by CRNs do not takeinto consideration the fact that spectrum bands vary sig-nificantly with regard to channel quality, thereby making

some channels of the spectrum bands more attractive toCRNs than others. In this paper, we aimed at answering thefundamental question of how CRNs should share hetero-geneous spectrum bands in a distributed yet fair mannerand proposed an evolutionary game theoretic framework toachieve that. We derived equilibrium strategies for CRNsspectrum sharing game for selecting particular spectrumbands and proved that the MSNE derived in the processare ESS while also being fair. We also derived the mech-anism of replicator dynamics with which players learnfrom payoff outcomes of their strategic interactions andmodify their strategies at every stage of the evolutionarygame. Because all players approach the ESS based solelyupon the common knowledge payoff observations, our pro-posed evolutionary framework can be implemented in adistributed manner.

REFERENCES

1. U.S. FCC, ET Docket 04-186. Notice of proposed rule

making, in the matter of unlicensed operation in the TV

broadcast bands, May 25, 2004.

2. Taher TM, Bacchus RB, Zdunek KJ, Roberson DA.

Long-term spectral occupancy findings in Chicago.

In IEEE Symposium on New Frontiers in Dynamic



Spectrum Access Networks (DySPAN), Aachen Ger-many, 2011; 100–107.

3. IEEE 802.22TM 2011 Standard for Wireless RegionalArea Networks in TV Whitespaces. http://www.ieee.org/22 [accessed on October 2015].

4. Feng X, Wang H, Wang X. A game approach forcooperative spectrum sharing in cognitive radio net-works. Wireless Communications and Mobile Comput-ing 2015; 15(3): 538–551.

5. Al-Tous H, Barhumi I. Resource allocation formultiple-users AF-OFDMA systems using the auctionframework. IEEE Transactions on Wireless Communi-cations 2014; 14(5): 2377–2393.

6. Wang B, Liu KJR, Clancy TC. Evolutionary coop-erative spectrum sensing game: how to collaborate?IEEE Transactions on Communications 2010; 58 (3):890–900.

7. Sroka P, Kliks A. Distributed interference mitigationin two-tier wireless networks using correlated equilib-rium and regret-matching learning. In European Con-ference on Networks and Communications (EuCNC),Bologna, Italy, 2014; 1–5.

8. Zheng J, Cai Y, Wu D. Subcarrier allocation based oncorrelated equilibrium in multi-cell OFDMA systems.EURASIP Journal on Wireless Communications andNetworking 2012; 2012: 1–12.

9. Huang JW, Krishnamurthy V. Cognitive base sta-tions in LTE/3GPP femtocells: a correlated equilibriumgame-theoretic approach. IEEE Transactions on Com-munications 2011; 59(12): 3485–3493.

10. Maharjan S, Zhang Y, Yuen C, Gjessing S. Distributedspectrum sensing in cognitive radio networks withfairness consideration: efficiency of correlated equi-librium. In IEEE Mobile Adhoc and Sensor Systems(MASS), Valencia Spain, 2011; 540–549.

11. Wang B, Han Z, Liu KJR. Peer-to-peer file sharinggame using correlated equilibrium. In 43rd AnnualConference on Information Sciences and Systems,IEEE CISS, Baltimore USA, 2009; 729–734.

12. Sengupta S, Chandramouli R, Brahma S, ChatterjeeM. A game theoretic framework for distributed self-coexistence among IEEE 802.22 networks. In IEEEGLOBECOM, New Orleans USA, 2008; 1–6.

13. Etkin R, Parekh A, Tse D. Spectrum sharing for unli-censed bands. IEEE Journal on Selected Areas inCommunications (JSAC) 2007; 25(3): 517–528.

14. Jiang C, Chen Y, Gao Y, Liu KJR. Joint spectrum sens-ing and access evolutionary game in cognitive radionetworks. IEEE Transactions on Wireless Communica-tions 2013; 12(5): 2470–2483.

15. Faisal Amjad M, Chatterjee M, Zou CC. Inducingvoluntary cooperation for optimal coexistence in cog-nitive radio networks: a game theoretic approach. In

IEEE Military Communications Conference (Milcom),

Baltimore USA, 2014; 955–961.

16. Sengupta S, Brahma S, Chatterjee M, Shankar N. Self-

coexistence among interference-aware IEEE 802.22

networks with enhanced air-interface. Pervasive and

Mobile Computing 2013; 9(4): 454–471.

17. Han Z, Pandana C, Liu KJR. Distributive opportunis-

tic spectrum access for cognitive radio using corre-

lated equilibrium and no-regret learning. In Wireless

Communications and Networking Conference, IEEE

WCNC, Kowloon, Hong Kong, 2007; 11–15.

18. Google, Inc.’s TV Bands Database System for Opera-

tion, ET Docket No. 04-186. http://www.google.com/

get/spectrumdatabase/channel/ [accessed on October

2015].

19. Show My White Space TVWS database from Spec-

trum Bridge Inc. http://whitespaces.spectrumbridge.

com/whitespaces/home.aspx [accessed on October

2015].

20. Shoham Y, Leyton-Brown K. Multiagent Systems:

Algorithmic, Game-Theoretic, and Logical Founda-

tions. Cambridge University Press, 2008.

21. Fudenburg D, Tirole J. Game Theory. The MIT press,

1991.

22. Maynard Smith J. Evolution and the Theory of Games.

Cambridge University Press, 1982.

23. Jain R, Hawe W, Chiu D. A quantitative measure of

fairness and discrimination for resource allocation in

shared computer systems, DEC-TR-301, September 26,

1984.

AUTHORS’ BIOGRAPHY

Muhammad Faisal Amjad is anAssistant Professor in the Departmentof Software Engineering, NationalUniversity of Sciences and Technol-ogy Pakistan. He received his PhDdegree in Computer Science from theUniversity of Central Florida USA in2015. His current research focuses on

dynamic spectrum access and defense against security vul-nerabilities in cognitive radio networks, wireless sensornetworks, game theory, and multi-agent systems. Muham-mad is a recipient of Fulbright PhD scholarship, HigherEducation Commission of Pakistan scholarship, and Presi-dent’s Gold Medal in master’s degree.


http://www.ieee.org/22

http://www.ieee.org/22

http://www.google.com/get/spectrumdatabase/channel/

http://www.google.com/get/spectrumdatabase/channel/

http://whitespaces.spectrumbridge.com/whitespaces/home.aspx

http://whitespaces.spectrumbridge.com/whitespaces/home.aspx


Mainak Chatterjee is an AssociateProfessor in the Department of Elec-trical Engineering and Computer Sci-ence, University of Central Florida,Orlando. He received his BSc degreein Physics (Hons.) from the Univer-sity of Calcutta; ME degree in Electri-cal Communication Engineering from

the Indian Institute of Science, Bangalore; and the PhDdegree from the Department of Computer Science andEngineering from the University of Texas at Arlington.His research interests include economic issues in wirelessnetworks, applied game theory, cognitive radio networks,dynamic spectrum access, and mobile video delivery. Hehas published over 150 conferences and journal papers.He obtained the Best Paper Awards in IEEE Globecom2008 and IEEE PIMRC 2011. He is the recipient ofthe AFOSR sponsored Young Investigator Program (YIP)Award. He co-founded the ACM Workshop on MobileVideo (MoVid). He serves on the editorial board ofElsevier’s Computer Communications and Pervasive andMobile Computing journals. He has served as the TPC Co-Chair of several conferences including ICDCN 2014, IEEEWoWMoM 2011, WONS 2010, IEEE MoVid 2009, Cog-nitive Radio Networks Track of IEEE Globecom 2009, andICCCN 2008. He also serves on the executive and technicalprogram committee of several international conferences.

Omar Nakhila is a PhD Candidatein the Department of Electrical Engi-neering and Computer Science, Uni-versity of Central Florida. He receivedhis MSc degree from the Departmentof Computer Engineering, Universityof Mosul, Mosul, Iraq, in 2007. Hisresearch focuses on the network pri-

vacy and security of computers, mobiles, and Internetof Things.

Cliff C. Zou is an associate professorin the Department of Electrical Engi-neering and Computer Science, Uni-versity of Central Florida. He receivedhis PhD degree from the Department ofElectrical and Computer Engineering,University of Massachusetts, Amherst,MA, in 2005. His research interests

include computer and network security, computer net-working, and performance evaluation. He is a seniormember of The Institute of Electrical and ElectronicsEngineers (IEEE).


evolutionary non-cooperative spectrum sharing game: long ...mainak/papers/faisal-wcmc.pdfmuhammad...

Documents